Sun Jun 2 16:19:27 UTC 2024 I: starting to build swift/unstable/amd64 on jenkins on '2024-06-02 16:19' Sun Jun 2 16:19:27 UTC 2024 I: The jenkins build log is/was available at https://jenkins.debian.net/userContent/reproducible/debian/build_service/amd64_25/8008/console.log Sun Jun 2 16:19:27 UTC 2024 I: Downloading source for unstable/swift=2.33.0-4 --2024-06-02 16:19:27-- http://deb.debian.org/debian/pool/main/s/swift/swift_2.33.0-4.dsc Connecting to 46.16.76.132:3128... connected. Proxy request sent, awaiting response... 200 OK Length: 3127 (3.1K) [text/prs.lines.tag] Saving to: ‘swift_2.33.0-4.dsc’ 0K ... 100% 338M=0s 2024-06-02 16:19:27 (338 MB/s) - ‘swift_2.33.0-4.dsc’ saved [3127/3127] Sun Jun 2 16:19:27 UTC 2024 I: swift_2.33.0-4.dsc -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 3.0 (quilt) Source: swift Binary: python3-swift, swift, swift-account, swift-container, swift-doc, swift-drive-audit, swift-object, swift-object-expirer, swift-proxy Architecture: all Version: 2.33.0-4 Maintainer: Debian OpenStack Uploaders: Thomas Goirand , Michal Arbet , Homepage: https://github.com/openstack/swift Standards-Version: 4.5.0 Vcs-Browser: https://salsa.debian.org/openstack-team/services/swift Vcs-Git: https://salsa.debian.org/openstack-team/services/swift.git Testsuite: autopkgtest, autopkgtest-pkg-python Testsuite-Triggers: @builddeps@ Build-Depends: debhelper-compat (= 12), dh-exec, dh-python, openstack-pkg-tools, python3-all, python3-pbr, python3-setuptools, python3-sphinx Build-Depends-Indep: python3-boto3, python3-cffi, python3-coverage, python3-cryptography, python3-eventlet, python3-fixtures, python3-greenlet, python3-keystoneclient, python3-keystonemiddleware, python3-lxml, python3-mock, python3-openssl, python3-openstackclient, python3-openstackdocstheme, python3-os-api-ref, python3-os-testr, python3-oslo.config, python3-oslo.log, python3-oslosphinx, python3-pastedeploy, python3-pyeclib, python3-requests, python3-requests-mock, python3-six, python3-stestr, python3-swiftclient, python3-xattr, subunit Package-List: python3-swift deb python optional arch=all swift deb net optional arch=all swift-account deb net optional arch=all swift-container deb net optional arch=all swift-doc deb doc optional arch=all swift-drive-audit deb net optional arch=all swift-object deb net optional arch=all swift-object-expirer deb net optional arch=all swift-proxy deb net optional arch=all Checksums-Sha1: 5fd704526efb11ab810225aa420f10b391461b7a 2640380 swift_2.33.0.orig.tar.xz 6783a70f4fe076f4d63c0d51e203a72050dd3952 29828 swift_2.33.0-4.debian.tar.xz Checksums-Sha256: aef4d1c25e58f792fa94ab26965f9d7dc491813cef149706d904034429e19ab8 2640380 swift_2.33.0.orig.tar.xz b4803f209f6dd509ec87d589b4481f024e889c3fa74a1bb8e832701169c557c0 29828 swift_2.33.0-4.debian.tar.xz Files: e6c241a62ad6f97ad61527c41fa78483 2640380 swift_2.33.0.orig.tar.xz 4a0e22615b39e926240bf75d9cc0d82d 29828 swift_2.33.0-4.debian.tar.xz -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEoLGp81CJVhMOekJc1BatFaxrQ/4FAmYmV/YACgkQ1BatFaxr Q/5AOhAAkC+L4IK1bjhluEdU/ocoVqZs56V93FFCXmM4LPsoc6isLrTR27j2v/ey keYDHczzgNkTMp6hicqxLi/WnWtv8j6aWjc8Bb4zW2VsFIHwzdvUEkMO5iJ7htaI Gd8Z8r+6DV3+Zt8dJCSqL3uObgeri6HhBWLDxD5tFTMNt+tbjYIqt1EL9ZEPQrKx lXFiUNU5pH/5yeKOBLIvAkKcehcS6SU4yvWgGMv6S9p1KDofLaEmDaiZy84bOjHM jxICAu5dhVA/Hf7uSbbXsq1oyZ2xWVYpD/OKw1n1HYJh/olrD6XOsUEFz2Wiki8g eiIbe6kFmYpbmbOIqv0bucpJqXTW1GhsdPgafl/xUR+EYp2SmzgfGQcYD6AxCYxk ohI0Aw/VHfUgVATPgePBDqiGDpD7KFbF+T+LYQKYfhhuSv8kEyvLFlgUKHzKLsX6 Q4dDPnsOOiXtz+3QNmFBWQbt4iFBQrjQM06AOmjm0DQNPCJMpiBt4Dd++nO39+ut TEnAel3/zrFfRfbvBPvFz36SffGgjc2vcqGOOkFgzXAwVXI4XCb3T/WeISYiiQ2b JpccejmG00rZsq8SVIIrdelplzOEG8/N69y61U7DS8aJZ8pqwknS4RV8l4HbIZaq FptghAQslkecoNaCHHaxg5xNS5Fj4YX1QsOPFwjqBfBGFbn3dp8= =a+gF -----END PGP SIGNATURE----- Sun Jun 2 16:19:27 UTC 2024 I: Checking whether the package is not for us Sun Jun 2 16:19:27 UTC 2024 I: Starting 1st build on remote node ionos1-amd64.debian.net. Sun Jun 2 16:19:27 UTC 2024 I: Preparing to do remote build '1' on ionos1-amd64.debian.net. Sun Jun 2 16:34:10 UTC 2024 I: Deleting $TMPDIR on ionos1-amd64.debian.net. I: pbuilder: network access will be disabled during build I: Current time: Sun Jun 2 04:19:29 -12 2024 I: pbuilder-time-stamp: 1717345169 I: Building the build Environment I: extracting base tarball [/var/cache/pbuilder/unstable-reproducible-base.tgz] I: copying local configuration W: --override-config is not set; not updating apt.conf Read the manpage for details. I: mounting /proc filesystem I: mounting /sys filesystem I: creating /{dev,run}/shm I: mounting /dev/pts filesystem I: redirecting /dev/ptmx to /dev/pts/ptmx I: policy-rc.d already exists I: Copying source file I: copying [swift_2.33.0-4.dsc] I: copying [./swift_2.33.0.orig.tar.xz] I: copying [./swift_2.33.0-4.debian.tar.xz] I: Extracting source gpgv: Signature made Mon Apr 22 12:28:38 2024 gpgv: using RSA key A0B1A9F3508956130E7A425CD416AD15AC6B43FE gpgv: Can't check signature: No public key dpkg-source: warning: cannot verify inline signature for ./swift_2.33.0-4.dsc: no acceptable signature found dpkg-source: info: extracting swift in swift-2.33.0 dpkg-source: info: unpacking swift_2.33.0.orig.tar.xz dpkg-source: info: unpacking swift_2.33.0-4.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying syslog_log_name.patch dpkg-source: info: applying disable_sphinxcontrib_rsvgconverter.patch dpkg-source: info: applying set-default-workers-value.patch dpkg-source: info: applying Add_tempurl_path_prefix_configuration_option.patch dpkg-source: info: applying swift-recon-only-query-object-servers-once.patch dpkg-source: info: applying Clean_up_watchdog_threads.patch dpkg-source: info: applying drive-full-checker.patch I: Not using root during the build. I: Installing the build-deps I: user script /srv/workspace/pbuilder/3531167/tmp/hooks/D02_print_environment starting I: set BUILDDIR='/build/reproducible-path' BUILDUSERGECOS='first user,first room,first work-phone,first home-phone,first other' BUILDUSERNAME='pbuilder1' BUILD_ARCH='amd64' DEBIAN_FRONTEND='noninteractive' DEB_BUILD_OPTIONS='buildinfo=+all reproducible=+all parallel=20 ' DISTRIBUTION='unstable' HOME='/root' HOST_ARCH='amd64' IFS=' ' INVOCATION_ID='82ef47bbb4f941aaa3fdcabd659f02cf' LANG='C' LANGUAGE='en_US:en' LC_ALL='C' MAIL='/var/mail/root' OPTIND='1' PATH='/usr/sbin:/usr/bin:/sbin:/bin:/usr/games' PBCURRENTCOMMANDLINEOPERATION='build' PBUILDER_OPERATION='build' PBUILDER_PKGDATADIR='/usr/share/pbuilder' PBUILDER_PKGLIBDIR='/usr/lib/pbuilder' PBUILDER_SYSCONFDIR='/etc' PPID='3531167' PS1='# ' PS2='> ' PS4='+ ' PWD='/' SHELL='/bin/bash' SHLVL='2' SUDO_COMMAND='/usr/bin/timeout -k 18.1h 18h /usr/bin/ionice -c 3 /usr/bin/nice /usr/sbin/pbuilder --build --configfile /srv/reproducible-results/rbuild-debian/r-b-build.CLcTtdy7/pbuilderrc_S6uI --distribution unstable --hookdir /etc/pbuilder/first-build-hooks --debbuildopts -b --basetgz /var/cache/pbuilder/unstable-reproducible-base.tgz --buildresult /srv/reproducible-results/rbuild-debian/r-b-build.CLcTtdy7/b1 --logfile b1/build.log swift_2.33.0-4.dsc' SUDO_GID='110' SUDO_UID='105' SUDO_USER='jenkins' TERM='unknown' TZ='/usr/share/zoneinfo/Etc/GMT+12' USER='root' _='/usr/bin/systemd-run' http_proxy='http://46.16.76.132:3128' I: uname -a Linux ionos1-amd64 6.1.0-21-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64 GNU/Linux I: ls -l /bin lrwxrwxrwx 1 root root 7 Jun 1 07:43 /bin -> usr/bin I: user script /srv/workspace/pbuilder/3531167/tmp/hooks/D02_print_environment finished -> Attempting to satisfy build-dependencies -> Creating pbuilder-satisfydepends-dummy package Package: pbuilder-satisfydepends-dummy Version: 0.invalid.0 Architecture: amd64 Maintainer: Debian Pbuilder Team Description: Dummy package to satisfy dependencies with aptitude - created by pbuilder This package was created automatically by pbuilder to satisfy the build-dependencies of the package being currently built. Depends: debhelper-compat (= 12), dh-exec, dh-python, openstack-pkg-tools, python3-all, python3-pbr, python3-setuptools, python3-sphinx, python3-boto3, python3-cffi, python3-coverage, python3-cryptography, python3-eventlet, python3-fixtures, python3-greenlet, python3-keystoneclient, python3-keystonemiddleware, python3-lxml, python3-mock, python3-openssl, python3-openstackclient, python3-openstackdocstheme, python3-os-api-ref, python3-os-testr, python3-oslo.config, python3-oslo.log, python3-oslosphinx, python3-pastedeploy, python3-pyeclib, python3-requests, python3-requests-mock, python3-six, python3-stestr, python3-swiftclient, python3-xattr, subunit dpkg-deb: building package 'pbuilder-satisfydepends-dummy' in '/tmp/satisfydepends-aptitude/pbuilder-satisfydepends-dummy.deb'. Selecting previously unselected package pbuilder-satisfydepends-dummy. (Reading database ... 19719 files and directories currently installed.) Preparing to unpack .../pbuilder-satisfydepends-dummy.deb ... Unpacking pbuilder-satisfydepends-dummy (0.invalid.0) ... dpkg: pbuilder-satisfydepends-dummy: dependency problems, but configuring anyway as you requested: pbuilder-satisfydepends-dummy depends on debhelper-compat (= 12); however: Package debhelper-compat is not installed. pbuilder-satisfydepends-dummy depends on dh-exec; however: Package dh-exec is not installed. pbuilder-satisfydepends-dummy depends on dh-python; however: Package dh-python is not installed. pbuilder-satisfydepends-dummy depends on openstack-pkg-tools; however: Package openstack-pkg-tools is not installed. pbuilder-satisfydepends-dummy depends on python3-all; however: Package python3-all is not installed. pbuilder-satisfydepends-dummy depends on python3-pbr; however: Package python3-pbr is not installed. pbuilder-satisfydepends-dummy depends on python3-setuptools; however: Package python3-setuptools is not installed. pbuilder-satisfydepends-dummy depends on python3-sphinx; however: Package python3-sphinx is not installed. pbuilder-satisfydepends-dummy depends on python3-boto3; however: Package python3-boto3 is not installed. pbuilder-satisfydepends-dummy depends on python3-cffi; however: Package python3-cffi is not installed. pbuilder-satisfydepends-dummy depends on python3-coverage; however: Package python3-coverage is not installed. pbuilder-satisfydepends-dummy depends on python3-cryptography; however: Package python3-cryptography is not installed. pbuilder-satisfydepends-dummy depends on python3-eventlet; however: Package python3-eventlet is not installed. pbuilder-satisfydepends-dummy depends on python3-fixtures; however: Package python3-fixtures is not installed. pbuilder-satisfydepends-dummy depends on python3-greenlet; however: Package python3-greenlet is not installed. pbuilder-satisfydepends-dummy depends on python3-keystoneclient; however: Package python3-keystoneclient is not installed. pbuilder-satisfydepends-dummy depends on python3-keystonemiddleware; however: Package python3-keystonemiddleware is not installed. pbuilder-satisfydepends-dummy depends on python3-lxml; however: Package python3-lxml is not installed. pbuilder-satisfydepends-dummy depends on python3-mock; however: Package python3-mock is not installed. pbuilder-satisfydepends-dummy depends on python3-openssl; however: Package python3-openssl is not installed. pbuilder-satisfydepends-dummy depends on python3-openstackclient; however: Package python3-openstackclient is not installed. pbuilder-satisfydepends-dummy depends on python3-openstackdocstheme; however: Package python3-openstackdocstheme is not installed. pbuilder-satisfydepends-dummy depends on python3-os-api-ref; however: Package python3-os-api-ref is not installed. pbuilder-satisfydepends-dummy depends on python3-os-testr; however: Package python3-os-testr is not installed. pbuilder-satisfydepends-dummy depends on python3-oslo.config; however: Package python3-oslo.config is not installed. pbuilder-satisfydepends-dummy depends on python3-oslo.log; however: Package python3-oslo.log is not installed. pbuilder-satisfydepends-dummy depends on python3-oslosphinx; however: Package python3-oslosphinx is not installed. pbuilder-satisfydepends-dummy depends on python3-pastedeploy; however: Package python3-pastedeploy is not installed. pbuilder-satisfydepends-dummy depends on python3-pyeclib; however: Package python3-pyeclib is not installed. pbuilder-satisfydepends-dummy depends on python3-requests; however: Package python3-requests is not installed. pbuilder-satisfydepends-dummy depends on python3-requests-mock; however: Package python3-requests-mock is not installed. pbuilder-satisfydepends-dummy depends on python3-six; however: Package python3-six is not installed. pbuilder-satisfydepends-dummy depends on python3-stestr; however: Package python3-stestr is not installed. pbuilder-satisfydepends-dummy depends on python3-swiftclient; however: Package python3-swiftclient is not installed. pbuilder-satisfydepends-dummy depends on python3-xattr; however: Package python3-xattr is not installed. pbuilder-satisfydepends-dummy depends on subunit; however: Package subunit is not installed. Setting up pbuilder-satisfydepends-dummy (0.invalid.0) ... Reading package lists... Building dependency tree... Reading state information... Initializing package states... Writing extended state information... Building tag database... pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) The following NEW packages will be installed: autoconf{a} automake{a} autopoint{a} autotools-dev{a} bsdextrautils{a} ca-certificates{a} debhelper{a} dh-autoreconf{a} dh-exec{a} dh-python{a} dh-strip-nondeterminism{a} docutils-common{a} dwz{a} file{a} fonts-font-awesome{a} fonts-glyphicons-halflings{a} gettext{a} gettext-base{a} groff-base{a} ieee-data{a} intltool-debian{a} jq{a} libarchive-zip-perl{a} libbsd0{a} libdebhelper-perl{a} libelf1t64{a} liberasurecode1{a} libexpat1{a} libfile-stripnondeterminism-perl{a} libice6{a} libicu72{a} libjq1{a} libjs-bootstrap{a} libjs-jquery{a} libjs-sphinxdoc{a} libjs-underscore{a} libjson-perl{a} libmagic-mgc{a} libmagic1t64{a} libonig5{a} libpipeline1{a} libpsl5t64{a} libpython3-stdlib{a} libpython3.11-minimal{a} libpython3.11-stdlib{a} libpython3.12-minimal{a} libpython3.12-stdlib{a} libreadline8t64{a} libsm6{a} libsubunit-perl{a} libtool{a} libuchardet0{a} libx11-6{a} libx11-data{a} libxau6{a} libxcb1{a} libxdmcp6{a} libxext6{a} libxml2{a} libxmu6{a} libxslt1.1{a} libxt6t64{a} libyaml-0-2{a} m4{a} man-db{a} media-types{a} netbase{a} openssl{a} openstack-pkg-tools{a} po-debconf{a} pycadf-common{a} python-babel-localedata{a} python-os-api-ref-common{a} python-oslosphinx-common{a} python3{a} python3-alabaster{a} python3-all{a} python3-attr{a} python3-autopage{a} python3-babel{a} python3-binary-memcached{a} python3-blinker{a} python3-boto3{a} python3-botocore{a} python3-certifi{a} python3-cffi{a} python3-cffi-backend{a} python3-chardet{a} python3-charset-normalizer{a} python3-cinderclient{a} python3-cliff{a} python3-cmd2{a} python3-coverage{a} python3-cryptography{a} python3-dateutil{a} python3-debtcollector{a} python3-decorator{a} python3-deprecation{a} python3-distutils{a} python3-dnspython{a} python3-docutils{a} python3-dogpile.cache{a} python3-dulwich{a} python3-eventlet{a} python3-extras{a} python3-fixtures{a} python3-glanceclient{a} python3-greenlet{a} python3-hatch-vcs{a} python3-hatchling{a} python3-idna{a} python3-imagesize{a} python3-importlib-metadata{a} python3-iso8601{a} python3-jaraco.classes{a} python3-jaraco.context{a} python3-jaraco.functools{a} python3-jeepney{a} python3-jinja2{a} python3-jmespath{a} python3-json-pointer{a} python3-jsonpatch{a} python3-jsonschema{a} python3-jsonschema-specifications{a} python3-jwt{a} python3-keyring{a} python3-keystoneauth1{a} python3-keystoneclient{a} python3-keystonemiddleware{a} python3-lib2to3{a} python3-lxml{a} python3-mako{a} python3-markupsafe{a} python3-memcache{a} python3-minimal{a} python3-mock{a} python3-more-itertools{a} python3-msgpack{a} python3-netaddr{a} python3-netifaces{a} python3-neutronclient{a} python3-novaclient{a} python3-oauthlib{a} python3-openssl{a} python3-openstackclient{a} python3-openstackdocstheme{a} python3-openstacksdk{a} python3-os-api-ref{a} python3-os-client-config{a} python3-os-service-types{a} python3-os-testr{a} python3-osc-lib{a} python3-oslo.cache{a} python3-oslo.config{a} python3-oslo.context{a} python3-oslo.i18n{a} python3-oslo.log{a} python3-oslo.serialization{a} python3-oslo.utils{a} python3-oslosphinx{a} python3-packaging{a} python3-paste{a} python3-pastedeploy{a} python3-pastedeploy-tpl{a} python3-pathspec{a} python3-pbr{a} python3-pip{a} python3-pkg-resources{a} python3-platformdirs{a} python3-pluggy{a} python3-ply{a} python3-prettytable{a} python3-pyasyncore{a} python3-pycadf{a} python3-pycparser{a} python3-pyeclib{a} python3-pygments{a} python3-pyinotify{a} python3-pyparsing{a} python3-pyperclip{a} python3-referencing{a} python3-requests{a} python3-requests-mock{a} python3-requestsexceptions{a} python3-rfc3986{a} python3-roman{a} python3-rpds-py{a} python3-s3transfer{a} python3-secretstorage{a} python3-setuptools{a} python3-setuptools-scm{a} python3-simplejson{a} python3-six{a} python3-snowballstemmer{a} python3-sphinx{a} python3-stestr{a} python3-stevedore{a} python3-subunit{a} python3-swiftclient{a} python3-tempita{a} python3-testtools{a} python3-tomlkit{a} python3-trove-classifiers{a} python3-typing-extensions{a} python3-tz{a} python3-uhashring{a} python3-urllib3{a} python3-voluptuous{a} python3-warlock{a} python3-wcwidth{a} python3-webob{a} python3-wheel{a} python3-wrapt{a} python3-xattr{a} python3-yaml{a} python3-zipp{a} python3.11{a} python3.11-minimal{a} python3.12{a} python3.12-minimal{a} readline-common{a} sensible-utils{a} sgml-base{a} sphinx-common{a} subunit{a} tzdata{a} wget{a} x11-common{a} xclip{a} xml-core{a} The following packages are RECOMMENDED but will NOT be installed: autopkgtest javascript-common libarchive-cpio-perl libjson-xs-perl libltdl-dev libmail-sendmail-perl libpaper-utils madison-lite pristine-tar publicsuffix python-openstackclient-doc python3-bs4 python3-cssselect python3-dev python3-fastimport python3-h2 python3-html5lib python3-httpcore python3-httpx python3-pastescript python3-pil python3-rfc3987 python3-uritemplate python3-webcolors xauth 0 packages upgraded, 230 newly installed, 0 to remove and 0 not upgraded. Need to get 69.8 MB of archives. After unpacking 379 MB will be used. Writing extended state information... Get: 1 http://deb.debian.org/debian unstable/main amd64 libpython3.11-minimal amd64 3.11.9-1 [817 kB] Get: 2 http://deb.debian.org/debian unstable/main amd64 libexpat1 amd64 2.6.2-1 [103 kB] Get: 3 http://deb.debian.org/debian unstable/main amd64 python3.11-minimal amd64 3.11.9-1 [1879 kB] Get: 4 http://deb.debian.org/debian unstable/main amd64 python3-minimal amd64 3.11.8-1 [26.3 kB] Get: 5 http://deb.debian.org/debian unstable/main amd64 media-types all 10.1.0 [26.9 kB] Get: 6 http://deb.debian.org/debian unstable/main amd64 netbase all 6.4 [12.8 kB] Get: 7 http://deb.debian.org/debian unstable/main amd64 tzdata all 2024a-4 [255 kB] Get: 8 http://deb.debian.org/debian unstable/main amd64 readline-common all 8.2-4 [69.3 kB] Get: 9 http://deb.debian.org/debian unstable/main amd64 libreadline8t64 amd64 8.2-4 [167 kB] Get: 10 http://deb.debian.org/debian unstable/main amd64 libpython3.11-stdlib amd64 3.11.9-1 [1792 kB] Get: 11 http://deb.debian.org/debian unstable/main amd64 python3.11 amd64 3.11.9-1 [602 kB] Get: 12 http://deb.debian.org/debian unstable/main amd64 libpython3-stdlib amd64 3.11.8-1 [9332 B] Get: 13 http://deb.debian.org/debian unstable/main amd64 python3 amd64 3.11.8-1 [27.4 kB] Get: 14 http://deb.debian.org/debian unstable/main amd64 libpython3.12-minimal amd64 3.12.3-1 [809 kB] Get: 15 http://deb.debian.org/debian unstable/main amd64 python3.12-minimal amd64 3.12.3-1 [2139 kB] Get: 16 http://deb.debian.org/debian unstable/main amd64 sgml-base all 1.31 [15.4 kB] Get: 17 http://deb.debian.org/debian unstable/main amd64 sensible-utils all 0.0.22 [22.4 kB] Get: 18 http://deb.debian.org/debian unstable/main amd64 openssl amd64 3.2.1-3 [1360 kB] Get: 19 http://deb.debian.org/debian unstable/main amd64 ca-certificates all 20240203 [158 kB] Get: 20 http://deb.debian.org/debian unstable/main amd64 libmagic-mgc amd64 1:5.45-3 [314 kB] Get: 21 http://deb.debian.org/debian unstable/main amd64 libmagic1t64 amd64 1:5.45-3 [105 kB] Get: 22 http://deb.debian.org/debian unstable/main amd64 file amd64 1:5.45-3 [42.9 kB] Get: 23 http://deb.debian.org/debian unstable/main amd64 gettext-base amd64 0.21-14+b1 [161 kB] Get: 24 http://deb.debian.org/debian unstable/main amd64 libuchardet0 amd64 0.0.8-1+b1 [68.8 kB] Get: 25 http://deb.debian.org/debian unstable/main amd64 groff-base amd64 1.23.0-4 [1180 kB] Get: 26 http://deb.debian.org/debian unstable/main amd64 bsdextrautils amd64 2.40.1-4 [95.7 kB] Get: 27 http://deb.debian.org/debian unstable/main amd64 libpipeline1 amd64 1.5.7-2 [38.0 kB] Get: 28 http://deb.debian.org/debian unstable/main amd64 man-db amd64 2.12.1-1 [1411 kB] Get: 29 http://deb.debian.org/debian unstable/main amd64 libpsl5t64 amd64 0.21.2-1.1 [56.8 kB] Get: 30 http://deb.debian.org/debian unstable/main amd64 wget amd64 1.24.5-1 [979 kB] Get: 31 http://deb.debian.org/debian unstable/main amd64 m4 amd64 1.4.19-4 [287 kB] Get: 32 http://deb.debian.org/debian unstable/main amd64 autoconf all 2.71-3 [332 kB] Get: 33 http://deb.debian.org/debian unstable/main amd64 autotools-dev all 20220109.1 [51.6 kB] Get: 34 http://deb.debian.org/debian unstable/main amd64 automake all 1:1.16.5-1.3 [823 kB] Get: 35 http://deb.debian.org/debian unstable/main amd64 autopoint all 0.21-14 [496 kB] Get: 36 http://deb.debian.org/debian unstable/main amd64 libdebhelper-perl all 13.15.3 [88.0 kB] Get: 37 http://deb.debian.org/debian unstable/main amd64 libtool all 2.4.7-7 [517 kB] Get: 38 http://deb.debian.org/debian unstable/main amd64 dh-autoreconf all 20 [17.1 kB] Get: 39 http://deb.debian.org/debian unstable/main amd64 libarchive-zip-perl all 1.68-1 [104 kB] Get: 40 http://deb.debian.org/debian unstable/main amd64 libfile-stripnondeterminism-perl all 1.14.0-1 [19.5 kB] Get: 41 http://deb.debian.org/debian unstable/main amd64 dh-strip-nondeterminism all 1.14.0-1 [8448 B] Get: 42 http://deb.debian.org/debian unstable/main amd64 libelf1t64 amd64 0.191-1+b1 [189 kB] Get: 43 http://deb.debian.org/debian unstable/main amd64 dwz amd64 0.15-1+b1 [110 kB] Get: 44 http://deb.debian.org/debian unstable/main amd64 libicu72 amd64 72.1-4+b1 [9395 kB] Get: 45 http://deb.debian.org/debian unstable/main amd64 libxml2 amd64 2.12.7+dfsg-2 [670 kB] Get: 46 http://deb.debian.org/debian unstable/main amd64 gettext amd64 0.21-14+b1 [1301 kB] Get: 47 http://deb.debian.org/debian unstable/main amd64 intltool-debian all 0.35.0+20060710.6 [22.9 kB] Get: 48 http://deb.debian.org/debian unstable/main amd64 po-debconf all 1.0.21+nmu1 [248 kB] Get: 49 http://deb.debian.org/debian unstable/main amd64 debhelper all 13.15.3 [901 kB] Get: 50 http://deb.debian.org/debian unstable/main amd64 dh-exec amd64 0.30 [25.6 kB] Get: 51 http://deb.debian.org/debian unstable/main amd64 python3-pkg-resources all 68.1.2-2 [241 kB] Get: 52 http://deb.debian.org/debian unstable/main amd64 python3-lib2to3 all 3.12.3-3.1 [77.7 kB] Get: 53 http://deb.debian.org/debian unstable/main amd64 python3-distutils all 3.12.3-3.1 [131 kB] Get: 54 http://deb.debian.org/debian unstable/main amd64 python3-setuptools all 68.1.2-2 [468 kB] Get: 55 http://deb.debian.org/debian unstable/main amd64 dh-python all 6.20240422 [107 kB] Get: 56 http://deb.debian.org/debian unstable/main amd64 xml-core all 0.19 [20.1 kB] Get: 57 http://deb.debian.org/debian unstable/main amd64 docutils-common all 0.20.1+dfsg-3 [128 kB] Get: 58 http://deb.debian.org/debian unstable/main amd64 fonts-font-awesome all 5.0.10+really4.7.0~dfsg-4.1 [517 kB] Get: 59 http://deb.debian.org/debian unstable/main amd64 fonts-glyphicons-halflings all 1.009~3.4.1+dfsg-3 [162 kB] Get: 60 http://deb.debian.org/debian unstable/main amd64 ieee-data all 20220827.1 [2029 kB] Get: 61 http://deb.debian.org/debian unstable/main amd64 libonig5 amd64 6.9.9-1 [189 kB] Get: 62 http://deb.debian.org/debian unstable/main amd64 libjq1 amd64 1.7.1-3 [163 kB] Get: 63 http://deb.debian.org/debian unstable/main amd64 jq amd64 1.7.1-3 [77.6 kB] Get: 64 http://deb.debian.org/debian unstable/main amd64 libbsd0 amd64 0.12.2-1 [131 kB] Get: 65 http://deb.debian.org/debian unstable/main amd64 liberasurecode1 amd64 1.6.3-1 [41.9 kB] Get: 66 http://deb.debian.org/debian unstable/main amd64 x11-common all 1:7.7+23 [252 kB] Get: 67 http://deb.debian.org/debian unstable/main amd64 libice6 amd64 2:1.0.10-1+b1 [56.7 kB] Get: 68 http://deb.debian.org/debian unstable/main amd64 libjs-bootstrap all 3.4.1+dfsg-3 [172 kB] Get: 69 http://deb.debian.org/debian unstable/main amd64 libjs-jquery all 3.6.1+dfsg+~3.5.14-1 [326 kB] Get: 70 http://deb.debian.org/debian unstable/main amd64 libjs-underscore all 1.13.4~dfsg+~1.11.4-3 [116 kB] Get: 71 http://deb.debian.org/debian unstable/main amd64 libjs-sphinxdoc all 7.2.6-8 [150 kB] Get: 72 http://deb.debian.org/debian unstable/main amd64 libjson-perl all 4.10000-1 [87.5 kB] Get: 73 http://deb.debian.org/debian unstable/main amd64 libpython3.12-stdlib amd64 3.12.3-1 [1951 kB] Get: 74 http://deb.debian.org/debian unstable/main amd64 libsm6 amd64 2:1.2.3-1+b1 [33.6 kB] Get: 75 http://deb.debian.org/debian unstable/main amd64 libsubunit-perl all 1.4.2-3 [5724 B] Get: 76 http://deb.debian.org/debian unstable/main amd64 libxau6 amd64 1:1.0.9-1+b1 [18.1 kB] Get: 77 http://deb.debian.org/debian unstable/main amd64 libxdmcp6 amd64 1:1.1.2-3+b1 [24.3 kB] Get: 78 http://deb.debian.org/debian unstable/main amd64 libxcb1 amd64 1.17.0-2 [144 kB] Get: 79 http://deb.debian.org/debian unstable/main amd64 libx11-data all 2:1.8.7-1 [328 kB] Get: 80 http://deb.debian.org/debian unstable/main amd64 libx11-6 amd64 2:1.8.7-1+b1 [799 kB] Get: 81 http://deb.debian.org/debian unstable/main amd64 libxext6 amd64 2:1.3.4-1+b1 [52.9 kB] Get: 82 http://deb.debian.org/debian unstable/main amd64 libxt6t64 amd64 1:1.2.1-1.2 [187 kB] Get: 83 http://deb.debian.org/debian unstable/main amd64 libxmu6 amd64 2:1.1.3-3+b2 [58.7 kB] Get: 84 http://deb.debian.org/debian unstable/main amd64 libxslt1.1 amd64 1.1.35-1+b1 [232 kB] Get: 85 http://deb.debian.org/debian unstable/main amd64 libyaml-0-2 amd64 0.2.5-1+b1 [52.6 kB] Get: 86 http://deb.debian.org/debian unstable/main amd64 python3-wheel all 0.43.0-1 [52.5 kB] Get: 87 http://deb.debian.org/debian unstable/main amd64 python3-pip all 24.0+dfsg-2 [1517 kB] Get: 88 http://deb.debian.org/debian unstable/main amd64 openstack-pkg-tools all 126 [94.2 kB] Get: 89 http://deb.debian.org/debian unstable/main amd64 pycadf-common all 3.1.1-3 [4492 B] Get: 90 http://deb.debian.org/debian unstable/main amd64 python-babel-localedata all 2.14.0-1 [5701 kB] Get: 91 http://deb.debian.org/debian unstable/main amd64 python-os-api-ref-common all 2.3.0+ds1-3 [9548 B] Get: 92 http://deb.debian.org/debian unstable/main amd64 python-oslosphinx-common all 4.18.0-5 [30.0 kB] Get: 93 http://deb.debian.org/debian unstable/main amd64 python3-alabaster all 0.7.12-1 [20.8 kB] Get: 94 http://deb.debian.org/debian unstable/main amd64 python3.12 amd64 3.12.3-1 [659 kB] Get: 95 http://deb.debian.org/debian unstable/main amd64 python3-all amd64 3.11.8-1 [1056 B] Get: 96 http://deb.debian.org/debian unstable/main amd64 python3-typing-extensions all 4.12.0-1 [72.1 kB] Get: 97 http://deb.debian.org/debian unstable/main amd64 python3-more-itertools all 10.2.0-1 [59.9 kB] Get: 98 http://deb.debian.org/debian unstable/main amd64 python3-zipp all 1.0.0-6 [6696 B] Get: 99 http://deb.debian.org/debian unstable/main amd64 python3-importlib-metadata all 7.1.0-1 [19.2 kB] Get: 100 http://deb.debian.org/debian unstable/main amd64 python3-attr all 23.2.0-2 [65.5 kB] Get: 101 http://deb.debian.org/debian unstable/main amd64 python3-autopage all 0.4.0-3 [12.7 kB] Get: 102 http://deb.debian.org/debian unstable/main amd64 python3-tz all 2024.1-2 [30.9 kB] Get: 103 http://deb.debian.org/debian unstable/main amd64 python3-babel all 2.14.0-1 [111 kB] Get: 104 http://deb.debian.org/debian unstable/main amd64 python3-six all 1.16.0-6 [16.3 kB] Get: 105 http://deb.debian.org/debian unstable/main amd64 python3-memcache all 1.62-1 [26.0 kB] Get: 106 http://deb.debian.org/debian unstable/main amd64 python3-uhashring all 2.3-2 [12.3 kB] Get: 107 http://deb.debian.org/debian unstable/main amd64 python3-binary-memcached all 0.31.2+dfsg1-2 [13.7 kB] Get: 108 http://deb.debian.org/debian unstable/main amd64 python3-blinker all 1.8.2-1 [13.1 kB] Get: 109 http://deb.debian.org/debian unstable/main amd64 python3-dateutil all 2.9.0-2 [79.4 kB] Get: 110 http://deb.debian.org/debian unstable/main amd64 python3-jmespath all 1.0.1-1 [21.1 kB] Get: 111 http://deb.debian.org/debian unstable/main amd64 python3-urllib3 all 1.26.18-2 [116 kB] Get: 112 http://deb.debian.org/debian unstable/main amd64 python3-certifi all 2023.11.17-1 [155 kB] Get: 113 http://deb.debian.org/debian unstable/main amd64 python3-charset-normalizer all 3.3.2-1 [51.6 kB] Get: 114 http://deb.debian.org/debian unstable/main amd64 python3-idna all 3.6-2 [37.0 kB] Get: 115 http://deb.debian.org/debian unstable/main amd64 python3-chardet all 5.2.0+dfsg-1 [107 kB] Get: 116 http://deb.debian.org/debian unstable/main amd64 python3-requests all 2.31.0+dfsg-2 [68.7 kB] Get: 117 http://deb.debian.org/debian unstable/main amd64 python3-botocore all 1.34.46+repack-1 [6464 kB] Get: 118 http://deb.debian.org/debian unstable/main amd64 python3-s3transfer all 0.10.1-1 [55.6 kB] Get: 119 http://deb.debian.org/debian unstable/main amd64 python3-boto3 all 1.34.46+dfsg-1 [269 kB] Get: 120 http://deb.debian.org/debian unstable/main amd64 python3-cffi-backend amd64 1.16.0-2+b2 [112 kB] Get: 121 http://deb.debian.org/debian unstable/main amd64 python3-ply all 3.11-6 [63.7 kB] Get: 122 http://deb.debian.org/debian unstable/main amd64 python3-pycparser all 2.22-1 [78.0 kB] Get: 123 http://deb.debian.org/debian unstable/main amd64 python3-cffi all 1.16.0-2 [88.5 kB] Get: 124 http://deb.debian.org/debian unstable/main amd64 python3-iso8601 all 1.0.2-1 [15.7 kB] Get: 125 http://deb.debian.org/debian unstable/main amd64 python3-lxml amd64 5.2.1-1 [1995 kB] Get: 126 http://deb.debian.org/debian unstable/main amd64 python3-cryptography amd64 42.0.5-2 [1186 kB] Get: 127 http://deb.debian.org/debian unstable/main amd64 python3-jwt all 2.7.0-1 [29.7 kB] Get: 128 http://deb.debian.org/debian unstable/main amd64 python3-oauthlib all 3.2.2-1 [95.4 kB] Get: 129 http://deb.debian.org/debian unstable/main amd64 python3-pbr all 6.0.0-1 [55.7 kB] Get: 130 http://deb.debian.org/debian unstable/main amd64 python3-os-service-types all 1.7.0-3 [12.5 kB] Get: 131 http://deb.debian.org/debian unstable/main amd64 python3-stevedore all 5.2.0-2 [20.8 kB] Get: 132 http://deb.debian.org/debian unstable/main amd64 python3-keystoneauth1 all 5.6.0-2 [155 kB] Get: 133 http://deb.debian.org/debian unstable/main amd64 python3-oslo.i18n all 6.3.0-2 [19.1 kB] Get: 134 http://deb.debian.org/debian unstable/main amd64 python3-wrapt amd64 1.15.0-2+b1 [49.3 kB] Get: 135 http://deb.debian.org/debian unstable/main amd64 python3-debtcollector all 3.0.0-2 [13.1 kB] Get: 136 http://deb.debian.org/debian unstable/main amd64 python3-netaddr all 0.10.1-1 [315 kB] Get: 137 http://deb.debian.org/debian unstable/main amd64 python3-netifaces amd64 0.11.0-2+b2 [19.4 kB] Get: 138 http://deb.debian.org/debian unstable/main amd64 python3-packaging all 24.0-1 [45.5 kB] Get: 139 http://deb.debian.org/debian unstable/main amd64 python3-pyparsing all 3.1.2-1 [146 kB] Get: 140 http://deb.debian.org/debian unstable/main amd64 python3-yaml amd64 6.0.1-2 [177 kB] Get: 141 http://deb.debian.org/debian unstable/main amd64 python3-oslo.utils all 7.1.0-2 [59.7 kB] Get: 142 http://deb.debian.org/debian unstable/main amd64 python3-wcwidth all 0.2.13+dfsg1-1 [24.9 kB] Get: 143 http://deb.debian.org/debian unstable/main amd64 python3-prettytable all 3.6.0-1 [36.0 kB] Get: 144 http://deb.debian.org/debian unstable/main amd64 python3-cinderclient all 1:9.5.0-2 [134 kB] Get: 145 http://deb.debian.org/debian unstable/main amd64 xclip amd64 0.13-4 [21.3 kB] Get: 146 http://deb.debian.org/debian unstable/main amd64 python3-pyperclip all 1.8.2-2 [13.2 kB] Get: 147 http://deb.debian.org/debian unstable/main amd64 python3-cmd2 all 2.4.3+ds-2 [144 kB] Get: 148 http://deb.debian.org/debian unstable/main amd64 python3-roman all 3.3-3 [9880 B] Get: 149 http://deb.debian.org/debian unstable/main amd64 python3-docutils all 0.20.1+dfsg-3 [389 kB] Get: 150 http://deb.debian.org/debian unstable/main amd64 python3-cliff all 4.6.0-2 [42.3 kB] Get: 151 http://deb.debian.org/debian unstable/main amd64 python3-coverage amd64 7.2.7+dfsg1-1+b1 [170 kB] Get: 152 http://deb.debian.org/debian unstable/main amd64 python3-decorator all 5.1.1-5 [15.1 kB] Get: 153 http://deb.debian.org/debian unstable/main amd64 python3-deprecation all 2.1.0-3 [14.5 kB] Get: 154 http://deb.debian.org/debian unstable/main amd64 python3-dnspython all 2.6.1-1 [161 kB] Get: 155 http://deb.debian.org/debian unstable/main amd64 python3-markupsafe amd64 2.1.5-1 [14.5 kB] Get: 156 http://deb.debian.org/debian unstable/main amd64 python3-mako all 1.3.2-1 [82.4 kB] Get: 157 http://deb.debian.org/debian unstable/main amd64 python3-dogpile.cache all 1.3.1-1 [45.0 kB] Get: 158 http://deb.debian.org/debian unstable/main amd64 python3-dulwich amd64 0.21.6-1+b1 [328 kB] Get: 159 http://deb.debian.org/debian unstable/main amd64 python3-greenlet amd64 3.0.1-3 [177 kB] Get: 160 http://deb.debian.org/debian unstable/main amd64 python3-eventlet all 0.35.1-1 [175 kB] Get: 161 http://deb.debian.org/debian unstable/main amd64 python3-extras all 1.0.0-5 [7536 B] Get: 162 http://deb.debian.org/debian unstable/main amd64 python3-testtools all 2.7.1-3 [121 kB] Get: 163 http://deb.debian.org/debian unstable/main amd64 python3-fixtures all 4.1.0-2 [34.1 kB] Get: 164 http://deb.debian.org/debian unstable/main amd64 python3-openssl all 24.1.0-1 [51.9 kB] Get: 165 http://deb.debian.org/debian unstable/main amd64 python3-json-pointer all 2.4-2 [14.6 kB] Get: 166 http://deb.debian.org/debian unstable/main amd64 python3-jsonpatch all 1.32-3 [11.9 kB] Get: 167 http://deb.debian.org/debian unstable/main amd64 python3-pathspec all 0.12.1-1 [28.1 kB] Get: 168 http://deb.debian.org/debian unstable/main amd64 python3-pluggy all 1.5.0-1 [26.9 kB] Get: 169 http://deb.debian.org/debian unstable/main amd64 python3-trove-classifiers all 2024.1.31-1 [9968 B] Get: 170 http://deb.debian.org/debian unstable/main amd64 python3-hatchling all 1.24.2-1 [53.2 kB] Get: 171 http://deb.debian.org/debian unstable/main amd64 python3-setuptools-scm all 8.0.4-2 [39.5 kB] Get: 172 http://deb.debian.org/debian unstable/main amd64 python3-hatch-vcs all 0.4.0-1 [8336 B] Get: 173 http://deb.debian.org/debian unstable/main amd64 python3-rpds-py amd64 0.12.0-3 [276 kB] Get: 174 http://deb.debian.org/debian unstable/main amd64 python3-referencing all 0.31.0-1 [22.2 kB] Get: 175 http://deb.debian.org/debian unstable/main amd64 python3-jsonschema-specifications all 2023.12.1-1 [8976 B] Get: 176 http://deb.debian.org/debian unstable/main amd64 python3-jsonschema all 4.19.2-3 [71.3 kB] Get: 177 http://deb.debian.org/debian unstable/main amd64 python3-warlock all 2.0.1-4 [9224 B] Get: 178 http://deb.debian.org/debian unstable/main amd64 python3-glanceclient all 1:4.5.0-2 [129 kB] Get: 179 http://deb.debian.org/debian unstable/main amd64 python3-imagesize all 1.4.1-1 [6688 B] Get: 180 http://deb.debian.org/debian unstable/main amd64 python3-jaraco.classes all 3.2.1-4 [7232 B] Get: 181 http://deb.debian.org/debian unstable/main amd64 python3-jaraco.context all 4.3.0-1 [7616 B] Get: 182 http://deb.debian.org/debian unstable/main amd64 python3-jaraco.functools all 4.0.0-1 [10.6 kB] Get: 183 http://deb.debian.org/debian unstable/main amd64 python3-jeepney all 0.8.0-3 [34.0 kB] Get: 184 http://deb.debian.org/debian unstable/main amd64 python3-jinja2 all 3.1.3-1 [119 kB] Get: 185 http://deb.debian.org/debian unstable/main amd64 python3-secretstorage all 3.3.3-3 [16.1 kB] Get: 186 http://deb.debian.org/debian unstable/main amd64 python3-keyring all 25.2.1-1 [53.7 kB] Get: 187 http://deb.debian.org/debian unstable/main amd64 python3-oslo.context all 5.5.0-2 [13.3 kB] Get: 188 http://deb.debian.org/debian unstable/main amd64 python3-msgpack amd64 1.0.3-3+b1 [96.5 kB] Get: 189 http://deb.debian.org/debian unstable/main amd64 python3-oslo.serialization all 5.4.0-2 [14.7 kB] Get: 190 http://deb.debian.org/debian unstable/main amd64 python3-pyasyncore all 1.0.2-2 [10.0 kB] Get: 191 http://deb.debian.org/debian unstable/main amd64 python3-pyinotify all 0.9.6-2 [27.4 kB] Get: 192 http://deb.debian.org/debian unstable/main amd64 python3-oslo.log all 5.5.1-2 [44.0 kB] Get: 193 http://deb.debian.org/debian unstable/main amd64 python3-rfc3986 all 2.0.0-2 [21.6 kB] Get: 194 http://deb.debian.org/debian unstable/main amd64 python3-oslo.config all 1:9.4.0-2 [105 kB] Get: 195 http://deb.debian.org/debian unstable/main amd64 python3-keystoneclient all 1:5.4.0-2 [199 kB] Get: 196 http://deb.debian.org/debian unstable/main amd64 python3-oslo.cache all 3.7.0-2 [34.8 kB] Get: 197 http://deb.debian.org/debian unstable/main amd64 python3-pycadf all 3.1.1-3 [17.9 kB] Get: 198 http://deb.debian.org/debian unstable/main amd64 python3-webob all 1:1.8.7-1 [88.2 kB] Get: 199 http://deb.debian.org/debian unstable/main amd64 python3-keystonemiddleware all 10.6.0-2 [79.0 kB] Get: 200 http://deb.debian.org/debian unstable/main amd64 python3-mock all 5.1.0-1 [70.2 kB] Get: 201 http://deb.debian.org/debian unstable/main amd64 python3-novaclient all 2:18.6.0-3 [248 kB] Get: 202 http://deb.debian.org/debian unstable/main amd64 python3-platformdirs all 4.2.1-1 [16.0 kB] Get: 203 http://deb.debian.org/debian unstable/main amd64 python3-requestsexceptions all 1.4.0-3 [4188 B] Get: 204 http://deb.debian.org/debian unstable/main amd64 python3-openstacksdk all 3.0.0-3 [666 kB] Get: 205 http://deb.debian.org/debian unstable/main amd64 python3-simplejson amd64 3.19.2-1+b1 [69.4 kB] Get: 206 http://deb.debian.org/debian unstable/main amd64 python3-osc-lib all 3.0.1-2 [52.7 kB] Get: 207 http://deb.debian.org/debian unstable/main amd64 python3-openstackclient all 6.6.0-3 [494 kB] Get: 208 http://deb.debian.org/debian unstable/main amd64 python3-os-client-config all 2.1.0-2 [17.4 kB] Get: 209 http://deb.debian.org/debian unstable/main amd64 python3-neutronclient all 1:11.2.0-3 [120 kB] Get: 210 http://deb.debian.org/debian unstable/main amd64 sphinx-common all 7.2.6-8 [702 kB] Get: 211 http://deb.debian.org/debian unstable/main amd64 python3-pygments all 2.17.2+dfsg-1 [818 kB] Get: 212 http://deb.debian.org/debian unstable/main amd64 python3-snowballstemmer all 2.2.0-4 [58.0 kB] Get: 213 http://deb.debian.org/debian unstable/main amd64 python3-sphinx all 7.2.6-8 [552 kB] Get: 214 http://deb.debian.org/debian unstable/main amd64 python3-openstackdocstheme all 3.2.0-1 [684 kB] Get: 215 http://deb.debian.org/debian unstable/main amd64 python3-oslosphinx all 4.18.0-5 [34.5 kB] Get: 216 http://deb.debian.org/debian unstable/main amd64 python3-os-api-ref all 2.3.0+ds1-3 [16.9 kB] Get: 217 http://deb.debian.org/debian unstable/main amd64 python3-tomlkit all 0.12.5-1 [42.0 kB] Get: 218 http://deb.debian.org/debian unstable/main amd64 python3-voluptuous all 0.14.2-1 [45.8 kB] Get: 219 http://deb.debian.org/debian unstable/main amd64 python3-subunit all 1.4.2-3 [66.4 kB] Get: 220 http://deb.debian.org/debian unstable/main amd64 subunit all 1.4.2-3 [5248 B] Get: 221 http://deb.debian.org/debian unstable/main amd64 python3-stestr all 4.1.0-2 [56.7 kB] Get: 222 http://deb.debian.org/debian unstable/main amd64 python3-os-testr all 3.0.0-3 [17.9 kB] Get: 223 http://deb.debian.org/debian unstable/main amd64 python3-tempita all 0.5.2-7 [15.0 kB] Get: 224 http://deb.debian.org/debian unstable/main amd64 python3-paste all 3.10.0-1 [222 kB] Get: 225 http://deb.debian.org/debian unstable/main amd64 python3-pastedeploy-tpl all 3.1-1 [8268 B] Get: 226 http://deb.debian.org/debian unstable/main amd64 python3-pastedeploy all 3.1-1 [18.3 kB] Get: 227 http://deb.debian.org/debian unstable/main amd64 python3-pyeclib amd64 1.6.1-2 [29.0 kB] Get: 228 http://deb.debian.org/debian unstable/main amd64 python3-requests-mock all 1.11.0-1 [17.4 kB] Get: 229 http://deb.debian.org/debian unstable/main amd64 python3-swiftclient all 1:4.5.0-2 [76.1 kB] Get: 230 http://deb.debian.org/debian unstable/main amd64 python3-xattr amd64 0.10.1-1 [16.2 kB] Fetched 69.8 MB in 1s (71.9 MB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package libpython3.11-minimal:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 19719 files and directories currently installed.) Preparing to unpack .../libpython3.11-minimal_3.11.9-1_amd64.deb ... Unpacking libpython3.11-minimal:amd64 (3.11.9-1) ... Selecting previously unselected package libexpat1:amd64. Preparing to unpack .../libexpat1_2.6.2-1_amd64.deb ... Unpacking libexpat1:amd64 (2.6.2-1) ... Selecting previously unselected package python3.11-minimal. Preparing to unpack .../python3.11-minimal_3.11.9-1_amd64.deb ... Unpacking python3.11-minimal (3.11.9-1) ... Setting up libpython3.11-minimal:amd64 (3.11.9-1) ... Setting up libexpat1:amd64 (2.6.2-1) ... Setting up python3.11-minimal (3.11.9-1) ... Selecting previously unselected package python3-minimal. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 20035 files and directories currently installed.) Preparing to unpack .../0-python3-minimal_3.11.8-1_amd64.deb ... Unpacking python3-minimal (3.11.8-1) ... Selecting previously unselected package media-types. Preparing to unpack .../1-media-types_10.1.0_all.deb ... Unpacking media-types (10.1.0) ... Selecting previously unselected package netbase. Preparing to unpack .../2-netbase_6.4_all.deb ... Unpacking netbase (6.4) ... Selecting previously unselected package tzdata. Preparing to unpack .../3-tzdata_2024a-4_all.deb ... Unpacking tzdata (2024a-4) ... Selecting previously unselected package readline-common. Preparing to unpack .../4-readline-common_8.2-4_all.deb ... Unpacking readline-common (8.2-4) ... Selecting previously unselected package libreadline8t64:amd64. Preparing to unpack .../5-libreadline8t64_8.2-4_amd64.deb ... Adding 'diversion of /lib/x86_64-linux-gnu/libhistory.so.8 to /lib/x86_64-linux-gnu/libhistory.so.8.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libhistory.so.8.2 to /lib/x86_64-linux-gnu/libhistory.so.8.2.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libreadline.so.8 to /lib/x86_64-linux-gnu/libreadline.so.8.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libreadline.so.8.2 to /lib/x86_64-linux-gnu/libreadline.so.8.2.usr-is-merged by libreadline8t64' Unpacking libreadline8t64:amd64 (8.2-4) ... Selecting previously unselected package libpython3.11-stdlib:amd64. Preparing to unpack .../6-libpython3.11-stdlib_3.11.9-1_amd64.deb ... Unpacking libpython3.11-stdlib:amd64 (3.11.9-1) ... Selecting previously unselected package python3.11. Preparing to unpack .../7-python3.11_3.11.9-1_amd64.deb ... Unpacking python3.11 (3.11.9-1) ... Selecting previously unselected package libpython3-stdlib:amd64. Preparing to unpack .../8-libpython3-stdlib_3.11.8-1_amd64.deb ... Unpacking libpython3-stdlib:amd64 (3.11.8-1) ... Setting up python3-minimal (3.11.8-1) ... Selecting previously unselected package python3. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 21027 files and directories currently installed.) Preparing to unpack .../000-python3_3.11.8-1_amd64.deb ... Unpacking python3 (3.11.8-1) ... Selecting previously unselected package libpython3.12-minimal:amd64. Preparing to unpack .../001-libpython3.12-minimal_3.12.3-1_amd64.deb ... Unpacking libpython3.12-minimal:amd64 (3.12.3-1) ... Selecting previously unselected package python3.12-minimal. Preparing to unpack .../002-python3.12-minimal_3.12.3-1_amd64.deb ... Unpacking python3.12-minimal (3.12.3-1) ... Selecting previously unselected package sgml-base. Preparing to unpack .../003-sgml-base_1.31_all.deb ... Unpacking sgml-base (1.31) ... Selecting previously unselected package sensible-utils. Preparing to unpack .../004-sensible-utils_0.0.22_all.deb ... Unpacking sensible-utils (0.0.22) ... Selecting previously unselected package openssl. Preparing to unpack .../005-openssl_3.2.1-3_amd64.deb ... Unpacking openssl (3.2.1-3) ... Selecting previously unselected package ca-certificates. Preparing to unpack .../006-ca-certificates_20240203_all.deb ... Unpacking ca-certificates (20240203) ... Selecting previously unselected package libmagic-mgc. Preparing to unpack .../007-libmagic-mgc_1%3a5.45-3_amd64.deb ... Unpacking libmagic-mgc (1:5.45-3) ... Selecting previously unselected package libmagic1t64:amd64. Preparing to unpack .../008-libmagic1t64_1%3a5.45-3_amd64.deb ... Unpacking libmagic1t64:amd64 (1:5.45-3) ... Selecting previously unselected package file. Preparing to unpack .../009-file_1%3a5.45-3_amd64.deb ... Unpacking file (1:5.45-3) ... Selecting previously unselected package gettext-base. Preparing to unpack .../010-gettext-base_0.21-14+b1_amd64.deb ... Unpacking gettext-base (0.21-14+b1) ... Selecting previously unselected package libuchardet0:amd64. Preparing to unpack .../011-libuchardet0_0.0.8-1+b1_amd64.deb ... Unpacking libuchardet0:amd64 (0.0.8-1+b1) ... Selecting previously unselected package groff-base. Preparing to unpack .../012-groff-base_1.23.0-4_amd64.deb ... Unpacking groff-base (1.23.0-4) ... Selecting previously unselected package bsdextrautils. Preparing to unpack .../013-bsdextrautils_2.40.1-4_amd64.deb ... Unpacking bsdextrautils (2.40.1-4) ... Selecting previously unselected package libpipeline1:amd64. Preparing to unpack .../014-libpipeline1_1.5.7-2_amd64.deb ... Unpacking libpipeline1:amd64 (1.5.7-2) ... Selecting previously unselected package man-db. Preparing to unpack .../015-man-db_2.12.1-1_amd64.deb ... Unpacking man-db (2.12.1-1) ... Selecting previously unselected package libpsl5t64:amd64. Preparing to unpack .../016-libpsl5t64_0.21.2-1.1_amd64.deb ... Unpacking libpsl5t64:amd64 (0.21.2-1.1) ... Selecting previously unselected package wget. Preparing to unpack .../017-wget_1.24.5-1_amd64.deb ... Unpacking wget (1.24.5-1) ... Selecting previously unselected package m4. Preparing to unpack .../018-m4_1.4.19-4_amd64.deb ... Unpacking m4 (1.4.19-4) ... Selecting previously unselected package autoconf. Preparing to unpack .../019-autoconf_2.71-3_all.deb ... Unpacking autoconf (2.71-3) ... Selecting previously unselected package autotools-dev. Preparing to unpack .../020-autotools-dev_20220109.1_all.deb ... Unpacking autotools-dev (20220109.1) ... Selecting previously unselected package automake. Preparing to unpack .../021-automake_1%3a1.16.5-1.3_all.deb ... Unpacking automake (1:1.16.5-1.3) ... Selecting previously unselected package autopoint. Preparing to unpack .../022-autopoint_0.21-14_all.deb ... Unpacking autopoint (0.21-14) ... Selecting previously unselected package libdebhelper-perl. Preparing to unpack .../023-libdebhelper-perl_13.15.3_all.deb ... Unpacking libdebhelper-perl (13.15.3) ... Selecting previously unselected package libtool. Preparing to unpack .../024-libtool_2.4.7-7_all.deb ... Unpacking libtool (2.4.7-7) ... Selecting previously unselected package dh-autoreconf. Preparing to unpack .../025-dh-autoreconf_20_all.deb ... Unpacking dh-autoreconf (20) ... Selecting previously unselected package libarchive-zip-perl. Preparing to unpack .../026-libarchive-zip-perl_1.68-1_all.deb ... Unpacking libarchive-zip-perl (1.68-1) ... Selecting previously unselected package libfile-stripnondeterminism-perl. Preparing to unpack .../027-libfile-stripnondeterminism-perl_1.14.0-1_all.deb ... Unpacking libfile-stripnondeterminism-perl (1.14.0-1) ... Selecting previously unselected package dh-strip-nondeterminism. Preparing to unpack .../028-dh-strip-nondeterminism_1.14.0-1_all.deb ... Unpacking dh-strip-nondeterminism (1.14.0-1) ... Selecting previously unselected package libelf1t64:amd64. Preparing to unpack .../029-libelf1t64_0.191-1+b1_amd64.deb ... Unpacking libelf1t64:amd64 (0.191-1+b1) ... Selecting previously unselected package dwz. Preparing to unpack .../030-dwz_0.15-1+b1_amd64.deb ... Unpacking dwz (0.15-1+b1) ... Selecting previously unselected package libicu72:amd64. Preparing to unpack .../031-libicu72_72.1-4+b1_amd64.deb ... Unpacking libicu72:amd64 (72.1-4+b1) ... Selecting previously unselected package libxml2:amd64. Preparing to unpack .../032-libxml2_2.12.7+dfsg-2_amd64.deb ... Unpacking libxml2:amd64 (2.12.7+dfsg-2) ... Selecting previously unselected package gettext. Preparing to unpack .../033-gettext_0.21-14+b1_amd64.deb ... Unpacking gettext (0.21-14+b1) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../034-intltool-debian_0.35.0+20060710.6_all.deb ... Unpacking intltool-debian (0.35.0+20060710.6) ... Selecting previously unselected package po-debconf. Preparing to unpack .../035-po-debconf_1.0.21+nmu1_all.deb ... Unpacking po-debconf (1.0.21+nmu1) ... Selecting previously unselected package debhelper. Preparing to unpack .../036-debhelper_13.15.3_all.deb ... Unpacking debhelper (13.15.3) ... Selecting previously unselected package dh-exec. Preparing to unpack .../037-dh-exec_0.30_amd64.deb ... Unpacking dh-exec (0.30) ... Selecting previously unselected package python3-pkg-resources. Preparing to unpack .../038-python3-pkg-resources_68.1.2-2_all.deb ... Unpacking python3-pkg-resources (68.1.2-2) ... Selecting previously unselected package python3-lib2to3. Preparing to unpack .../039-python3-lib2to3_3.12.3-3.1_all.deb ... Unpacking python3-lib2to3 (3.12.3-3.1) ... Selecting previously unselected package python3-distutils. Preparing to unpack .../040-python3-distutils_3.12.3-3.1_all.deb ... Unpacking python3-distutils (3.12.3-3.1) ... Selecting previously unselected package python3-setuptools. Preparing to unpack .../041-python3-setuptools_68.1.2-2_all.deb ... Unpacking python3-setuptools (68.1.2-2) ... Selecting previously unselected package dh-python. Preparing to unpack .../042-dh-python_6.20240422_all.deb ... Unpacking dh-python (6.20240422) ... Selecting previously unselected package xml-core. Preparing to unpack .../043-xml-core_0.19_all.deb ... Unpacking xml-core (0.19) ... Selecting previously unselected package docutils-common. Preparing to unpack .../044-docutils-common_0.20.1+dfsg-3_all.deb ... Unpacking docutils-common (0.20.1+dfsg-3) ... Selecting previously unselected package fonts-font-awesome. Preparing to unpack .../045-fonts-font-awesome_5.0.10+really4.7.0~dfsg-4.1_all.deb ... Unpacking fonts-font-awesome (5.0.10+really4.7.0~dfsg-4.1) ... Selecting previously unselected package fonts-glyphicons-halflings. Preparing to unpack .../046-fonts-glyphicons-halflings_1.009~3.4.1+dfsg-3_all.deb ... Unpacking fonts-glyphicons-halflings (1.009~3.4.1+dfsg-3) ... Selecting previously unselected package ieee-data. Preparing to unpack .../047-ieee-data_20220827.1_all.deb ... Unpacking ieee-data (20220827.1) ... Selecting previously unselected package libonig5:amd64. Preparing to unpack .../048-libonig5_6.9.9-1_amd64.deb ... Unpacking libonig5:amd64 (6.9.9-1) ... Selecting previously unselected package libjq1:amd64. Preparing to unpack .../049-libjq1_1.7.1-3_amd64.deb ... Unpacking libjq1:amd64 (1.7.1-3) ... Selecting previously unselected package jq. Preparing to unpack .../050-jq_1.7.1-3_amd64.deb ... Unpacking jq (1.7.1-3) ... Selecting previously unselected package libbsd0:amd64. Preparing to unpack .../051-libbsd0_0.12.2-1_amd64.deb ... Unpacking libbsd0:amd64 (0.12.2-1) ... Selecting previously unselected package liberasurecode1:amd64. Preparing to unpack .../052-liberasurecode1_1.6.3-1_amd64.deb ... Unpacking liberasurecode1:amd64 (1.6.3-1) ... Selecting previously unselected package x11-common. Preparing to unpack .../053-x11-common_1%3a7.7+23_all.deb ... Unpacking x11-common (1:7.7+23) ... Selecting previously unselected package libice6:amd64. Preparing to unpack .../054-libice6_2%3a1.0.10-1+b1_amd64.deb ... Unpacking libice6:amd64 (2:1.0.10-1+b1) ... Selecting previously unselected package libjs-bootstrap. Preparing to unpack .../055-libjs-bootstrap_3.4.1+dfsg-3_all.deb ... Unpacking libjs-bootstrap (3.4.1+dfsg-3) ... Selecting previously unselected package libjs-jquery. Preparing to unpack .../056-libjs-jquery_3.6.1+dfsg+~3.5.14-1_all.deb ... Unpacking libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... Selecting previously unselected package libjs-underscore. Preparing to unpack .../057-libjs-underscore_1.13.4~dfsg+~1.11.4-3_all.deb ... Unpacking libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... Selecting previously unselected package libjs-sphinxdoc. Preparing to unpack .../058-libjs-sphinxdoc_7.2.6-8_all.deb ... Unpacking libjs-sphinxdoc (7.2.6-8) ... Selecting previously unselected package libjson-perl. Preparing to unpack .../059-libjson-perl_4.10000-1_all.deb ... Unpacking libjson-perl (4.10000-1) ... Selecting previously unselected package libpython3.12-stdlib:amd64. Preparing to unpack .../060-libpython3.12-stdlib_3.12.3-1_amd64.deb ... Unpacking libpython3.12-stdlib:amd64 (3.12.3-1) ... Selecting previously unselected package libsm6:amd64. Preparing to unpack .../061-libsm6_2%3a1.2.3-1+b1_amd64.deb ... Unpacking libsm6:amd64 (2:1.2.3-1+b1) ... Selecting previously unselected package libsubunit-perl. Preparing to unpack .../062-libsubunit-perl_1.4.2-3_all.deb ... Unpacking libsubunit-perl (1.4.2-3) ... Selecting previously unselected package libxau6:amd64. Preparing to unpack .../063-libxau6_1%3a1.0.9-1+b1_amd64.deb ... Unpacking libxau6:amd64 (1:1.0.9-1+b1) ... Selecting previously unselected package libxdmcp6:amd64. Preparing to unpack .../064-libxdmcp6_1%3a1.1.2-3+b1_amd64.deb ... Unpacking libxdmcp6:amd64 (1:1.1.2-3+b1) ... Selecting previously unselected package libxcb1:amd64. Preparing to unpack .../065-libxcb1_1.17.0-2_amd64.deb ... Unpacking libxcb1:amd64 (1.17.0-2) ... Selecting previously unselected package libx11-data. Preparing to unpack .../066-libx11-data_2%3a1.8.7-1_all.deb ... Unpacking libx11-data (2:1.8.7-1) ... Selecting previously unselected package libx11-6:amd64. Preparing to unpack .../067-libx11-6_2%3a1.8.7-1+b1_amd64.deb ... Unpacking libx11-6:amd64 (2:1.8.7-1+b1) ... Selecting previously unselected package libxext6:amd64. Preparing to unpack .../068-libxext6_2%3a1.3.4-1+b1_amd64.deb ... Unpacking libxext6:amd64 (2:1.3.4-1+b1) ... Selecting previously unselected package libxt6t64:amd64. Preparing to unpack .../069-libxt6t64_1%3a1.2.1-1.2_amd64.deb ... Unpacking libxt6t64:amd64 (1:1.2.1-1.2) ... Selecting previously unselected package libxmu6:amd64. Preparing to unpack .../070-libxmu6_2%3a1.1.3-3+b2_amd64.deb ... Unpacking libxmu6:amd64 (2:1.1.3-3+b2) ... Selecting previously unselected package libxslt1.1:amd64. Preparing to unpack .../071-libxslt1.1_1.1.35-1+b1_amd64.deb ... Unpacking libxslt1.1:amd64 (1.1.35-1+b1) ... Selecting previously unselected package libyaml-0-2:amd64. Preparing to unpack .../072-libyaml-0-2_0.2.5-1+b1_amd64.deb ... Unpacking libyaml-0-2:amd64 (0.2.5-1+b1) ... Selecting previously unselected package python3-wheel. Preparing to unpack .../073-python3-wheel_0.43.0-1_all.deb ... Unpacking python3-wheel (0.43.0-1) ... Selecting previously unselected package python3-pip. Preparing to unpack .../074-python3-pip_24.0+dfsg-2_all.deb ... Unpacking python3-pip (24.0+dfsg-2) ... Selecting previously unselected package openstack-pkg-tools. Preparing to unpack .../075-openstack-pkg-tools_126_all.deb ... Unpacking openstack-pkg-tools (126) ... Selecting previously unselected package pycadf-common. Preparing to unpack .../076-pycadf-common_3.1.1-3_all.deb ... Unpacking pycadf-common (3.1.1-3) ... Selecting previously unselected package python-babel-localedata. Preparing to unpack .../077-python-babel-localedata_2.14.0-1_all.deb ... Unpacking python-babel-localedata (2.14.0-1) ... Selecting previously unselected package python-os-api-ref-common. Preparing to unpack .../078-python-os-api-ref-common_2.3.0+ds1-3_all.deb ... Unpacking python-os-api-ref-common (2.3.0+ds1-3) ... Selecting previously unselected package python-oslosphinx-common. Preparing to unpack .../079-python-oslosphinx-common_4.18.0-5_all.deb ... Unpacking python-oslosphinx-common (4.18.0-5) ... Selecting previously unselected package python3-alabaster. Preparing to unpack .../080-python3-alabaster_0.7.12-1_all.deb ... Unpacking python3-alabaster (0.7.12-1) ... Selecting previously unselected package python3.12. Preparing to unpack .../081-python3.12_3.12.3-1_amd64.deb ... Unpacking python3.12 (3.12.3-1) ... Selecting previously unselected package python3-all. Preparing to unpack .../082-python3-all_3.11.8-1_amd64.deb ... Unpacking python3-all (3.11.8-1) ... Selecting previously unselected package python3-typing-extensions. Preparing to unpack .../083-python3-typing-extensions_4.12.0-1_all.deb ... Unpacking python3-typing-extensions (4.12.0-1) ... Selecting previously unselected package python3-more-itertools. Preparing to unpack .../084-python3-more-itertools_10.2.0-1_all.deb ... Unpacking python3-more-itertools (10.2.0-1) ... Selecting previously unselected package python3-zipp. Preparing to unpack .../085-python3-zipp_1.0.0-6_all.deb ... Unpacking python3-zipp (1.0.0-6) ... Selecting previously unselected package python3-importlib-metadata. Preparing to unpack .../086-python3-importlib-metadata_7.1.0-1_all.deb ... Unpacking python3-importlib-metadata (7.1.0-1) ... Selecting previously unselected package python3-attr. Preparing to unpack .../087-python3-attr_23.2.0-2_all.deb ... Unpacking python3-attr (23.2.0-2) ... Selecting previously unselected package python3-autopage. Preparing to unpack .../088-python3-autopage_0.4.0-3_all.deb ... Unpacking python3-autopage (0.4.0-3) ... Selecting previously unselected package python3-tz. Preparing to unpack .../089-python3-tz_2024.1-2_all.deb ... Unpacking python3-tz (2024.1-2) ... Selecting previously unselected package python3-babel. Preparing to unpack .../090-python3-babel_2.14.0-1_all.deb ... Unpacking python3-babel (2.14.0-1) ... Selecting previously unselected package python3-six. Preparing to unpack .../091-python3-six_1.16.0-6_all.deb ... Unpacking python3-six (1.16.0-6) ... Selecting previously unselected package python3-memcache. Preparing to unpack .../092-python3-memcache_1.62-1_all.deb ... Unpacking python3-memcache (1.62-1) ... Selecting previously unselected package python3-uhashring. Preparing to unpack .../093-python3-uhashring_2.3-2_all.deb ... Unpacking python3-uhashring (2.3-2) ... Selecting previously unselected package python3-binary-memcached. Preparing to unpack .../094-python3-binary-memcached_0.31.2+dfsg1-2_all.deb ... Unpacking python3-binary-memcached (0.31.2+dfsg1-2) ... Selecting previously unselected package python3-blinker. Preparing to unpack .../095-python3-blinker_1.8.2-1_all.deb ... Unpacking python3-blinker (1.8.2-1) ... Selecting previously unselected package python3-dateutil. Preparing to unpack .../096-python3-dateutil_2.9.0-2_all.deb ... Unpacking python3-dateutil (2.9.0-2) ... Selecting previously unselected package python3-jmespath. Preparing to unpack .../097-python3-jmespath_1.0.1-1_all.deb ... Unpacking python3-jmespath (1.0.1-1) ... Selecting previously unselected package python3-urllib3. Preparing to unpack .../098-python3-urllib3_1.26.18-2_all.deb ... Unpacking python3-urllib3 (1.26.18-2) ... Selecting previously unselected package python3-certifi. Preparing to unpack .../099-python3-certifi_2023.11.17-1_all.deb ... Unpacking python3-certifi (2023.11.17-1) ... Selecting previously unselected package python3-charset-normalizer. Preparing to unpack .../100-python3-charset-normalizer_3.3.2-1_all.deb ... Unpacking python3-charset-normalizer (3.3.2-1) ... Selecting previously unselected package python3-idna. Preparing to unpack .../101-python3-idna_3.6-2_all.deb ... Unpacking python3-idna (3.6-2) ... Selecting previously unselected package python3-chardet. Preparing to unpack .../102-python3-chardet_5.2.0+dfsg-1_all.deb ... Unpacking python3-chardet (5.2.0+dfsg-1) ... Selecting previously unselected package python3-requests. Preparing to unpack .../103-python3-requests_2.31.0+dfsg-2_all.deb ... Unpacking python3-requests (2.31.0+dfsg-2) ... Selecting previously unselected package python3-botocore. Preparing to unpack .../104-python3-botocore_1.34.46+repack-1_all.deb ... Unpacking python3-botocore (1.34.46+repack-1) ... Selecting previously unselected package python3-s3transfer. Preparing to unpack .../105-python3-s3transfer_0.10.1-1_all.deb ... Unpacking python3-s3transfer (0.10.1-1) ... Selecting previously unselected package python3-boto3. Preparing to unpack .../106-python3-boto3_1.34.46+dfsg-1_all.deb ... Unpacking python3-boto3 (1.34.46+dfsg-1) ... Selecting previously unselected package python3-cffi-backend:amd64. Preparing to unpack .../107-python3-cffi-backend_1.16.0-2+b2_amd64.deb ... Unpacking python3-cffi-backend:amd64 (1.16.0-2+b2) ... Selecting previously unselected package python3-ply. Preparing to unpack .../108-python3-ply_3.11-6_all.deb ... Unpacking python3-ply (3.11-6) ... Selecting previously unselected package python3-pycparser. Preparing to unpack .../109-python3-pycparser_2.22-1_all.deb ... Unpacking python3-pycparser (2.22-1) ... Selecting previously unselected package python3-cffi. Preparing to unpack .../110-python3-cffi_1.16.0-2_all.deb ... Unpacking python3-cffi (1.16.0-2) ... Selecting previously unselected package python3-iso8601. Preparing to unpack .../111-python3-iso8601_1.0.2-1_all.deb ... Unpacking python3-iso8601 (1.0.2-1) ... Selecting previously unselected package python3-lxml:amd64. Preparing to unpack .../112-python3-lxml_5.2.1-1_amd64.deb ... Unpacking python3-lxml:amd64 (5.2.1-1) ... Selecting previously unselected package python3-cryptography. Preparing to unpack .../113-python3-cryptography_42.0.5-2_amd64.deb ... Unpacking python3-cryptography (42.0.5-2) ... Selecting previously unselected package python3-jwt. Preparing to unpack .../114-python3-jwt_2.7.0-1_all.deb ... Unpacking python3-jwt (2.7.0-1) ... Selecting previously unselected package python3-oauthlib. Preparing to unpack .../115-python3-oauthlib_3.2.2-1_all.deb ... Unpacking python3-oauthlib (3.2.2-1) ... Selecting previously unselected package python3-pbr. Preparing to unpack .../116-python3-pbr_6.0.0-1_all.deb ... Unpacking python3-pbr (6.0.0-1) ... Selecting previously unselected package python3-os-service-types. Preparing to unpack .../117-python3-os-service-types_1.7.0-3_all.deb ... Unpacking python3-os-service-types (1.7.0-3) ... Selecting previously unselected package python3-stevedore. Preparing to unpack .../118-python3-stevedore_5.2.0-2_all.deb ... Unpacking python3-stevedore (5.2.0-2) ... Selecting previously unselected package python3-keystoneauth1. Preparing to unpack .../119-python3-keystoneauth1_5.6.0-2_all.deb ... Unpacking python3-keystoneauth1 (5.6.0-2) ... Selecting previously unselected package python3-oslo.i18n. Preparing to unpack .../120-python3-oslo.i18n_6.3.0-2_all.deb ... Unpacking python3-oslo.i18n (6.3.0-2) ... Selecting previously unselected package python3-wrapt. Preparing to unpack .../121-python3-wrapt_1.15.0-2+b1_amd64.deb ... Unpacking python3-wrapt (1.15.0-2+b1) ... Selecting previously unselected package python3-debtcollector. Preparing to unpack .../122-python3-debtcollector_3.0.0-2_all.deb ... Unpacking python3-debtcollector (3.0.0-2) ... Selecting previously unselected package python3-netaddr. Preparing to unpack .../123-python3-netaddr_0.10.1-1_all.deb ... Unpacking python3-netaddr (0.10.1-1) ... Selecting previously unselected package python3-netifaces:amd64. Preparing to unpack .../124-python3-netifaces_0.11.0-2+b2_amd64.deb ... Unpacking python3-netifaces:amd64 (0.11.0-2+b2) ... Selecting previously unselected package python3-packaging. Preparing to unpack .../125-python3-packaging_24.0-1_all.deb ... Unpacking python3-packaging (24.0-1) ... Selecting previously unselected package python3-pyparsing. Preparing to unpack .../126-python3-pyparsing_3.1.2-1_all.deb ... Unpacking python3-pyparsing (3.1.2-1) ... Selecting previously unselected package python3-yaml. Preparing to unpack .../127-python3-yaml_6.0.1-2_amd64.deb ... Unpacking python3-yaml (6.0.1-2) ... Selecting previously unselected package python3-oslo.utils. Preparing to unpack .../128-python3-oslo.utils_7.1.0-2_all.deb ... Unpacking python3-oslo.utils (7.1.0-2) ... Selecting previously unselected package python3-wcwidth. Preparing to unpack .../129-python3-wcwidth_0.2.13+dfsg1-1_all.deb ... Unpacking python3-wcwidth (0.2.13+dfsg1-1) ... Selecting previously unselected package python3-prettytable. Preparing to unpack .../130-python3-prettytable_3.6.0-1_all.deb ... Unpacking python3-prettytable (3.6.0-1) ... Selecting previously unselected package python3-cinderclient. Preparing to unpack .../131-python3-cinderclient_1%3a9.5.0-2_all.deb ... Unpacking python3-cinderclient (1:9.5.0-2) ... Selecting previously unselected package xclip. Preparing to unpack .../132-xclip_0.13-4_amd64.deb ... Unpacking xclip (0.13-4) ... Selecting previously unselected package python3-pyperclip. Preparing to unpack .../133-python3-pyperclip_1.8.2-2_all.deb ... Unpacking python3-pyperclip (1.8.2-2) ... Selecting previously unselected package python3-cmd2. Preparing to unpack .../134-python3-cmd2_2.4.3+ds-2_all.deb ... Unpacking python3-cmd2 (2.4.3+ds-2) ... Selecting previously unselected package python3-roman. Preparing to unpack .../135-python3-roman_3.3-3_all.deb ... Unpacking python3-roman (3.3-3) ... Selecting previously unselected package python3-docutils. Preparing to unpack .../136-python3-docutils_0.20.1+dfsg-3_all.deb ... Unpacking python3-docutils (0.20.1+dfsg-3) ... Selecting previously unselected package python3-cliff. Preparing to unpack .../137-python3-cliff_4.6.0-2_all.deb ... Unpacking python3-cliff (4.6.0-2) ... Selecting previously unselected package python3-coverage. Preparing to unpack .../138-python3-coverage_7.2.7+dfsg1-1+b1_amd64.deb ... Unpacking python3-coverage (7.2.7+dfsg1-1+b1) ... Selecting previously unselected package python3-decorator. Preparing to unpack .../139-python3-decorator_5.1.1-5_all.deb ... Unpacking python3-decorator (5.1.1-5) ... Selecting previously unselected package python3-deprecation. Preparing to unpack .../140-python3-deprecation_2.1.0-3_all.deb ... Unpacking python3-deprecation (2.1.0-3) ... Selecting previously unselected package python3-dnspython. Preparing to unpack .../141-python3-dnspython_2.6.1-1_all.deb ... Unpacking python3-dnspython (2.6.1-1) ... Selecting previously unselected package python3-markupsafe. Preparing to unpack .../142-python3-markupsafe_2.1.5-1_amd64.deb ... Unpacking python3-markupsafe (2.1.5-1) ... Selecting previously unselected package python3-mako. Preparing to unpack .../143-python3-mako_1.3.2-1_all.deb ... Unpacking python3-mako (1.3.2-1) ... Selecting previously unselected package python3-dogpile.cache. Preparing to unpack .../144-python3-dogpile.cache_1.3.1-1_all.deb ... Unpacking python3-dogpile.cache (1.3.1-1) ... Selecting previously unselected package python3-dulwich. Preparing to unpack .../145-python3-dulwich_0.21.6-1+b1_amd64.deb ... Unpacking python3-dulwich (0.21.6-1+b1) ... Selecting previously unselected package python3-greenlet. Preparing to unpack .../146-python3-greenlet_3.0.1-3_amd64.deb ... Unpacking python3-greenlet (3.0.1-3) ... Selecting previously unselected package python3-eventlet. Preparing to unpack .../147-python3-eventlet_0.35.1-1_all.deb ... Unpacking python3-eventlet (0.35.1-1) ... Selecting previously unselected package python3-extras. Preparing to unpack .../148-python3-extras_1.0.0-5_all.deb ... Unpacking python3-extras (1.0.0-5) ... Selecting previously unselected package python3-testtools. Preparing to unpack .../149-python3-testtools_2.7.1-3_all.deb ... Unpacking python3-testtools (2.7.1-3) ... Selecting previously unselected package python3-fixtures. Preparing to unpack .../150-python3-fixtures_4.1.0-2_all.deb ... Unpacking python3-fixtures (4.1.0-2) ... Selecting previously unselected package python3-openssl. Preparing to unpack .../151-python3-openssl_24.1.0-1_all.deb ... Unpacking python3-openssl (24.1.0-1) ... Selecting previously unselected package python3-json-pointer. Preparing to unpack .../152-python3-json-pointer_2.4-2_all.deb ... Unpacking python3-json-pointer (2.4-2) ... Selecting previously unselected package python3-jsonpatch. Preparing to unpack .../153-python3-jsonpatch_1.32-3_all.deb ... Unpacking python3-jsonpatch (1.32-3) ... Selecting previously unselected package python3-pathspec. Preparing to unpack .../154-python3-pathspec_0.12.1-1_all.deb ... Unpacking python3-pathspec (0.12.1-1) ... Selecting previously unselected package python3-pluggy. Preparing to unpack .../155-python3-pluggy_1.5.0-1_all.deb ... Unpacking python3-pluggy (1.5.0-1) ... Selecting previously unselected package python3-trove-classifiers. Preparing to unpack .../156-python3-trove-classifiers_2024.1.31-1_all.deb ... Unpacking python3-trove-classifiers (2024.1.31-1) ... Selecting previously unselected package python3-hatchling. Preparing to unpack .../157-python3-hatchling_1.24.2-1_all.deb ... Unpacking python3-hatchling (1.24.2-1) ... Selecting previously unselected package python3-setuptools-scm. Preparing to unpack .../158-python3-setuptools-scm_8.0.4-2_all.deb ... Unpacking python3-setuptools-scm (8.0.4-2) ... Selecting previously unselected package python3-hatch-vcs. Preparing to unpack .../159-python3-hatch-vcs_0.4.0-1_all.deb ... Unpacking python3-hatch-vcs (0.4.0-1) ... Selecting previously unselected package python3-rpds-py. Preparing to unpack .../160-python3-rpds-py_0.12.0-3_amd64.deb ... Unpacking python3-rpds-py (0.12.0-3) ... Selecting previously unselected package python3-referencing. Preparing to unpack .../161-python3-referencing_0.31.0-1_all.deb ... Unpacking python3-referencing (0.31.0-1) ... Selecting previously unselected package python3-jsonschema-specifications. Preparing to unpack .../162-python3-jsonschema-specifications_2023.12.1-1_all.deb ... Unpacking python3-jsonschema-specifications (2023.12.1-1) ... Selecting previously unselected package python3-jsonschema. Preparing to unpack .../163-python3-jsonschema_4.19.2-3_all.deb ... Unpacking python3-jsonschema (4.19.2-3) ... Selecting previously unselected package python3-warlock. Preparing to unpack .../164-python3-warlock_2.0.1-4_all.deb ... Unpacking python3-warlock (2.0.1-4) ... Selecting previously unselected package python3-glanceclient. Preparing to unpack .../165-python3-glanceclient_1%3a4.5.0-2_all.deb ... Unpacking python3-glanceclient (1:4.5.0-2) ... Selecting previously unselected package python3-imagesize. Preparing to unpack .../166-python3-imagesize_1.4.1-1_all.deb ... Unpacking python3-imagesize (1.4.1-1) ... Selecting previously unselected package python3-jaraco.classes. Preparing to unpack .../167-python3-jaraco.classes_3.2.1-4_all.deb ... Unpacking python3-jaraco.classes (3.2.1-4) ... Selecting previously unselected package python3-jaraco.context. Preparing to unpack .../168-python3-jaraco.context_4.3.0-1_all.deb ... Unpacking python3-jaraco.context (4.3.0-1) ... Selecting previously unselected package python3-jaraco.functools. Preparing to unpack .../169-python3-jaraco.functools_4.0.0-1_all.deb ... Unpacking python3-jaraco.functools (4.0.0-1) ... Selecting previously unselected package python3-jeepney. Preparing to unpack .../170-python3-jeepney_0.8.0-3_all.deb ... Unpacking python3-jeepney (0.8.0-3) ... Selecting previously unselected package python3-jinja2. Preparing to unpack .../171-python3-jinja2_3.1.3-1_all.deb ... Unpacking python3-jinja2 (3.1.3-1) ... Selecting previously unselected package python3-secretstorage. Preparing to unpack .../172-python3-secretstorage_3.3.3-3_all.deb ... Unpacking python3-secretstorage (3.3.3-3) ... Selecting previously unselected package python3-keyring. Preparing to unpack .../173-python3-keyring_25.2.1-1_all.deb ... Unpacking python3-keyring (25.2.1-1) ... Selecting previously unselected package python3-oslo.context. Preparing to unpack .../174-python3-oslo.context_5.5.0-2_all.deb ... Unpacking python3-oslo.context (5.5.0-2) ... Selecting previously unselected package python3-msgpack. Preparing to unpack .../175-python3-msgpack_1.0.3-3+b1_amd64.deb ... Unpacking python3-msgpack (1.0.3-3+b1) ... Selecting previously unselected package python3-oslo.serialization. Preparing to unpack .../176-python3-oslo.serialization_5.4.0-2_all.deb ... Unpacking python3-oslo.serialization (5.4.0-2) ... Selecting previously unselected package python3-pyasyncore. Preparing to unpack .../177-python3-pyasyncore_1.0.2-2_all.deb ... Unpacking python3-pyasyncore (1.0.2-2) ... Selecting previously unselected package python3-pyinotify. Preparing to unpack .../178-python3-pyinotify_0.9.6-2_all.deb ... Unpacking python3-pyinotify (0.9.6-2) ... Selecting previously unselected package python3-oslo.log. Preparing to unpack .../179-python3-oslo.log_5.5.1-2_all.deb ... Unpacking python3-oslo.log (5.5.1-2) ... Selecting previously unselected package python3-rfc3986. Preparing to unpack .../180-python3-rfc3986_2.0.0-2_all.deb ... Unpacking python3-rfc3986 (2.0.0-2) ... Selecting previously unselected package python3-oslo.config. Preparing to unpack .../181-python3-oslo.config_1%3a9.4.0-2_all.deb ... Unpacking python3-oslo.config (1:9.4.0-2) ... Selecting previously unselected package python3-keystoneclient. Preparing to unpack .../182-python3-keystoneclient_1%3a5.4.0-2_all.deb ... Unpacking python3-keystoneclient (1:5.4.0-2) ... Selecting previously unselected package python3-oslo.cache. Preparing to unpack .../183-python3-oslo.cache_3.7.0-2_all.deb ... Unpacking python3-oslo.cache (3.7.0-2) ... Selecting previously unselected package python3-pycadf. Preparing to unpack .../184-python3-pycadf_3.1.1-3_all.deb ... Unpacking python3-pycadf (3.1.1-3) ... Selecting previously unselected package python3-webob. Preparing to unpack .../185-python3-webob_1%3a1.8.7-1_all.deb ... Unpacking python3-webob (1:1.8.7-1) ... Selecting previously unselected package python3-keystonemiddleware. Preparing to unpack .../186-python3-keystonemiddleware_10.6.0-2_all.deb ... Unpacking python3-keystonemiddleware (10.6.0-2) ... Selecting previously unselected package python3-mock. Preparing to unpack .../187-python3-mock_5.1.0-1_all.deb ... Unpacking python3-mock (5.1.0-1) ... Selecting previously unselected package python3-novaclient. Preparing to unpack .../188-python3-novaclient_2%3a18.6.0-3_all.deb ... Unpacking python3-novaclient (2:18.6.0-3) ... Selecting previously unselected package python3-platformdirs. Preparing to unpack .../189-python3-platformdirs_4.2.1-1_all.deb ... Unpacking python3-platformdirs (4.2.1-1) ... Selecting previously unselected package python3-requestsexceptions. Preparing to unpack .../190-python3-requestsexceptions_1.4.0-3_all.deb ... Unpacking python3-requestsexceptions (1.4.0-3) ... Selecting previously unselected package python3-openstacksdk. Preparing to unpack .../191-python3-openstacksdk_3.0.0-3_all.deb ... Unpacking python3-openstacksdk (3.0.0-3) ... Selecting previously unselected package python3-simplejson. Preparing to unpack .../192-python3-simplejson_3.19.2-1+b1_amd64.deb ... Unpacking python3-simplejson (3.19.2-1+b1) ... Selecting previously unselected package python3-osc-lib. Preparing to unpack .../193-python3-osc-lib_3.0.1-2_all.deb ... Unpacking python3-osc-lib (3.0.1-2) ... Selecting previously unselected package python3-openstackclient. Preparing to unpack .../194-python3-openstackclient_6.6.0-3_all.deb ... Unpacking python3-openstackclient (6.6.0-3) ... Selecting previously unselected package python3-os-client-config. Preparing to unpack .../195-python3-os-client-config_2.1.0-2_all.deb ... Unpacking python3-os-client-config (2.1.0-2) ... Selecting previously unselected package python3-neutronclient. Preparing to unpack .../196-python3-neutronclient_1%3a11.2.0-3_all.deb ... Unpacking python3-neutronclient (1:11.2.0-3) ... Selecting previously unselected package sphinx-common. Preparing to unpack .../197-sphinx-common_7.2.6-8_all.deb ... Unpacking sphinx-common (7.2.6-8) ... Selecting previously unselected package python3-pygments. Preparing to unpack .../198-python3-pygments_2.17.2+dfsg-1_all.deb ... Unpacking python3-pygments (2.17.2+dfsg-1) ... Selecting previously unselected package python3-snowballstemmer. Preparing to unpack .../199-python3-snowballstemmer_2.2.0-4_all.deb ... Unpacking python3-snowballstemmer (2.2.0-4) ... Selecting previously unselected package python3-sphinx. Preparing to unpack .../200-python3-sphinx_7.2.6-8_all.deb ... Unpacking python3-sphinx (7.2.6-8) ... Selecting previously unselected package python3-openstackdocstheme. Preparing to unpack .../201-python3-openstackdocstheme_3.2.0-1_all.deb ... Unpacking python3-openstackdocstheme (3.2.0-1) ... Selecting previously unselected package python3-oslosphinx. Preparing to unpack .../202-python3-oslosphinx_4.18.0-5_all.deb ... Unpacking python3-oslosphinx (4.18.0-5) ... Selecting previously unselected package python3-os-api-ref. Preparing to unpack .../203-python3-os-api-ref_2.3.0+ds1-3_all.deb ... Unpacking python3-os-api-ref (2.3.0+ds1-3) ... Selecting previously unselected package python3-tomlkit. Preparing to unpack .../204-python3-tomlkit_0.12.5-1_all.deb ... Unpacking python3-tomlkit (0.12.5-1) ... Selecting previously unselected package python3-voluptuous. Preparing to unpack .../205-python3-voluptuous_0.14.2-1_all.deb ... Unpacking python3-voluptuous (0.14.2-1) ... Selecting previously unselected package python3-subunit. Preparing to unpack .../206-python3-subunit_1.4.2-3_all.deb ... Unpacking python3-subunit (1.4.2-3) ... Selecting previously unselected package subunit. Preparing to unpack .../207-subunit_1.4.2-3_all.deb ... Unpacking subunit (1.4.2-3) ... Selecting previously unselected package python3-stestr. Preparing to unpack .../208-python3-stestr_4.1.0-2_all.deb ... Unpacking python3-stestr (4.1.0-2) ... Selecting previously unselected package python3-os-testr. Preparing to unpack .../209-python3-os-testr_3.0.0-3_all.deb ... Unpacking python3-os-testr (3.0.0-3) ... Selecting previously unselected package python3-tempita. Preparing to unpack .../210-python3-tempita_0.5.2-7_all.deb ... Unpacking python3-tempita (0.5.2-7) ... Selecting previously unselected package python3-paste. Preparing to unpack .../211-python3-paste_3.10.0-1_all.deb ... Unpacking python3-paste (3.10.0-1) ... Selecting previously unselected package python3-pastedeploy-tpl. Preparing to unpack .../212-python3-pastedeploy-tpl_3.1-1_all.deb ... Unpacking python3-pastedeploy-tpl (3.1-1) ... Selecting previously unselected package python3-pastedeploy. Preparing to unpack .../213-python3-pastedeploy_3.1-1_all.deb ... Unpacking python3-pastedeploy (3.1-1) ... Selecting previously unselected package python3-pyeclib. Preparing to unpack .../214-python3-pyeclib_1.6.1-2_amd64.deb ... Unpacking python3-pyeclib (1.6.1-2) ... Selecting previously unselected package python3-requests-mock. Preparing to unpack .../215-python3-requests-mock_1.11.0-1_all.deb ... Unpacking python3-requests-mock (1.11.0-1) ... Selecting previously unselected package python3-swiftclient. Preparing to unpack .../216-python3-swiftclient_1%3a4.5.0-2_all.deb ... Unpacking python3-swiftclient (1:4.5.0-2) ... Selecting previously unselected package python3-xattr. Preparing to unpack .../217-python3-xattr_0.10.1-1_amd64.deb ... Unpacking python3-xattr (0.10.1-1) ... Setting up media-types (10.1.0) ... Setting up libpipeline1:amd64 (1.5.7-2) ... Setting up libxau6:amd64 (1:1.0.9-1+b1) ... Setting up libsubunit-perl (1.4.2-3) ... Setting up libicu72:amd64 (72.1-4+b1) ... Setting up bsdextrautils (2.40.1-4) ... Setting up libmagic-mgc (1:5.45-3) ... Setting up libarchive-zip-perl (1.68-1) ... Setting up libyaml-0-2:amd64 (0.2.5-1+b1) ... Setting up fonts-glyphicons-halflings (1.009~3.4.1+dfsg-3) ... Setting up libdebhelper-perl (13.15.3) ... Setting up libmagic1t64:amd64 (1:5.45-3) ... Setting up libpython3.12-minimal:amd64 (3.12.3-1) ... Setting up x11-common (1:7.7+23) ... invoke-rc.d: could not determine current runlevel Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. Setting up libpsl5t64:amd64 (0.21.2-1.1) ... Setting up gettext-base (0.21-14+b1) ... Setting up m4 (1.4.19-4) ... Setting up file (1:5.45-3) ... Setting up libelf1t64:amd64 (0.191-1+b1) ... Setting up python-babel-localedata (2.14.0-1) ... Setting up tzdata (2024a-4) ... Current default time zone: 'Etc/UTC' Local time is now: Sun Jun 2 16:20:24 UTC 2024. Universal Time is now: Sun Jun 2 16:20:24 UTC 2024. Run 'dpkg-reconfigure tzdata' if you wish to change it. Setting up autotools-dev (20220109.1) ... Setting up libx11-data (2:1.8.7-1) ... Setting up python3-pastedeploy-tpl (3.1-1) ... Setting up autopoint (0.21-14) ... Setting up autoconf (2.71-3) ... Setting up liberasurecode1:amd64 (1.6.3-1) ... Setting up pycadf-common (3.1.1-3) ... Setting up dwz (0.15-1+b1) ... Setting up sensible-utils (0.0.22) ... Setting up libuchardet0:amd64 (0.0.8-1+b1) ... Setting up python-oslosphinx-common (4.18.0-5) ... Setting up libjson-perl (4.10000-1) ... Setting up netbase (6.4) ... Setting up sgml-base (1.31) ... Setting up libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... Setting up openssl (3.2.1-3) ... Setting up libbsd0:amd64 (0.12.2-1) ... Setting up readline-common (8.2-4) ... Setting up libxml2:amd64 (2.12.7+dfsg-2) ... Setting up fonts-font-awesome (5.0.10+really4.7.0~dfsg-4.1) ... Setting up libonig5:amd64 (6.9.9-1) ... Setting up libjs-bootstrap (3.4.1+dfsg-3) ... Setting up libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... Setting up automake (1:1.16.5-1.3) ... update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode Setting up libfile-stripnondeterminism-perl (1.14.0-1) ... Setting up python3.12-minimal (3.12.3-1) ... Setting up libice6:amd64 (2:1.0.10-1+b1) ... Setting up libxdmcp6:amd64 (1:1.1.2-3+b1) ... Setting up libxcb1:amd64 (1.17.0-2) ... Setting up gettext (0.21-14+b1) ... Setting up wget (1.24.5-1) ... Setting up libtool (2.4.7-7) ... Setting up libjq1:amd64 (1.7.1-3) ... Setting up intltool-debian (0.35.0+20060710.6) ... Setting up dh-autoreconf (20) ... Setting up ca-certificates (20240203) ... Updating certificates in /etc/ssl/certs... 146 added, 0 removed; done. Setting up python-os-api-ref-common (2.3.0+ds1-3) ... Setting up libjs-sphinxdoc (7.2.6-8) ... Setting up libreadline8t64:amd64 (8.2-4) ... Setting up dh-strip-nondeterminism (1.14.0-1) ... Setting up ieee-data (20220827.1) ... Setting up groff-base (1.23.0-4) ... Setting up xml-core (0.19) ... Setting up jq (1.7.1-3) ... Setting up libxslt1.1:amd64 (1.1.35-1+b1) ... Setting up libx11-6:amd64 (2:1.8.7-1+b1) ... Setting up libsm6:amd64 (2:1.2.3-1+b1) ... Setting up libpython3.12-stdlib:amd64 (3.12.3-1) ... Setting up po-debconf (1.0.21+nmu1) ... Setting up libpython3.11-stdlib:amd64 (3.11.9-1) ... Setting up python3.12 (3.12.3-1) ... Setting up libxext6:amd64 (2:1.3.4-1+b1) ... Setting up man-db (2.12.1-1) ... Not building database; man-db/auto-update is not 'true'. Setting up sphinx-common (7.2.6-8) ... Setting up libxt6t64:amd64 (1:1.2.1-1.2) ... Setting up libpython3-stdlib:amd64 (3.11.8-1) ... Setting up python3.11 (3.11.9-1) ... Setting up libxmu6:amd64 (2:1.1.3-3+b2) ... Setting up debhelper (13.15.3) ... Setting up python3 (3.11.8-1) ... Setting up dh-exec (0.30) ... Setting up python3-markupsafe (2.1.5-1) ... Setting up python3-wheel (0.43.0-1) ... Setting up python3-platformdirs (4.2.1-1) ... Setting up python3-tz (2024.1-2) ... Setting up python3-mock (5.1.0-1) ... Setting up python3-six (1.16.0-6) ... Setting up python3-simplejson (3.19.2-1+b1) ... Setting up python3-roman (3.3-3) ... Setting up python3-decorator (5.1.1-5) ... Setting up python3-jinja2 (3.1.3-1) ... Setting up python3-tempita (0.5.2-7) ... Setting up python3-packaging (24.0-1) ... Setting up python3-iso8601 (1.0.2-1) ... Setting up python3-wcwidth (0.2.13+dfsg1-1) ... Setting up python3-pyparsing (3.1.2-1) ... Setting up python3-certifi (2023.11.17-1) ... Setting up python3-snowballstemmer (2.2.0-4) ... Setting up python3-pyeclib (1.6.1-2) ... Setting up python3-jmespath (1.0.1-1) ... Setting up python3-greenlet (3.0.1-3) ... Setting up python3-idna (3.6-2) ... Setting up python3-wrapt (1.15.0-2+b1) ... Setting up xclip (0.13-4) ... Setting up python3-deprecation (2.1.0-3) ... Setting up python3-typing-extensions (4.12.0-1) ... Setting up python3-pip (24.0+dfsg-2) ... Setting up python3-jeepney (0.8.0-3) ... Setting up python3-urllib3 (1.26.18-2) ... Setting up python3-pluggy (1.5.0-1) ... Setting up python3-netifaces:amd64 (0.11.0-2+b2) ... Setting up python3-rpds-py (0.12.0-3) ... Setting up python3-netaddr (0.10.1-1) ... Setting up python3-trove-classifiers (2024.1.31-1) ... Setting up python3-json-pointer (2.4-2) ... Setting up python3-lxml:amd64 (5.2.1-1) ... Setting up python3-dnspython (2.6.1-1) ... Setting up python3-dulwich (0.21.6-1+b1) ... Setting up python3-dateutil (2.9.0-2) ... Setting up python3-msgpack (1.0.3-3+b1) ... Setting up python3-voluptuous (0.14.2-1) ... Setting up python3-pyasyncore (1.0.2-2) ... Setting up python3-prettytable (3.6.0-1) ... Setting up python3-tomlkit (0.12.5-1) ... Setting up python3-pathspec (0.12.1-1) ... Setting up python3-lib2to3 (3.12.3-3.1) ... Setting up python3-cffi-backend:amd64 (1.16.0-2+b2) ... Setting up python3-webob (1:1.8.7-1) ... Setting up python3-imagesize (1.4.1-1) ... Setting up python3-blinker (1.8.2-1) ... Setting up python3-pkg-resources (68.1.2-2) ... Setting up python3-distutils (3.12.3-3.1) ... python3.12: can't get files for byte-compilation Setting up python3-more-itertools (10.2.0-1) ... Setting up python3-attr (23.2.0-2) ... Setting up python3-jaraco.classes (3.2.1-4) ... Setting up python3-jaraco.functools (4.0.0-1) ... Setting up python3-setuptools (68.1.2-2) ... Setting up python3-jaraco.context (4.3.0-1) ... Setting up python3-jwt (2.7.0-1) ... Setting up python3-babel (2.14.0-1) ... update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode Setting up python3-pbr (6.0.0-1) ... Setting up python3-charset-normalizer (3.3.2-1) ... Setting up python3-alabaster (0.7.12-1) ... Setting up python3-jsonpatch (1.32-3) ... update-alternatives: using /usr/bin/json-patch-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode Setting up python3-ply (3.11-6) ... Setting up python3-autopage (0.4.0-3) ... Setting up python3-pyinotify (0.9.6-2) ... Setting up python3-extras (1.0.0-5) ... Setting up python3-memcache (1.62-1) ... Setting up python3-all (3.11.8-1) ... Setting up python3-requestsexceptions (1.4.0-3) ... Setting up python3-coverage (7.2.7+dfsg1-1+b1) ... Setting up python3-yaml (6.0.1-2) ... Setting up python3-pyperclip (1.8.2-2) ... Setting up python3-debtcollector (3.0.0-2) ... Setting up python3-zipp (1.0.0-6) ... Setting up python3-setuptools-scm (8.0.4-2) ... Setting up openstack-pkg-tools (126) ... Setting up python3-pycparser (2.22-1) ... Setting up python3-eventlet (0.35.1-1) ... Setting up python3-pygments (2.17.2+dfsg-1) ... Setting up python3-chardet (5.2.0+dfsg-1) ... Setting up python3-rfc3986 (2.0.0-2) ... Setting up python3-paste (3.10.0-1) ... Setting up python3-cmd2 (2.4.3+ds-2) ... Setting up python3-cryptography (42.0.5-2) ... Setting up python3-requests (2.31.0+dfsg-2) ... Setting up python3-oslosphinx (4.18.0-5) ... Setting up python3-oslo.context (5.5.0-2) ... Setting up python3-uhashring (2.3-2) ... Setting up python3-xattr (0.10.1-1) ... Setting up python3-os-service-types (1.7.0-3) ... Setting up python3-stevedore (5.2.0-2) ... Setting up python3-hatchling (1.24.2-1) ... Setting up python3-swiftclient (1:4.5.0-2) ... Setting up python3-binary-memcached (0.31.2+dfsg1-2) ... Setting up python3-oslo.i18n (6.3.0-2) ... Setting up python3-mako (1.3.2-1) ... Setting up python3-hatch-vcs (0.4.0-1) ... Setting up python3-referencing (0.31.0-1) ... Setting up dh-python (6.20240422) ... Setting up python3-importlib-metadata (7.1.0-1) ... Setting up python3-openssl (24.1.0-1) ... Setting up python3-pastedeploy (3.1-1) ... Setting up python3-cffi (1.16.0-2) ... Setting up python3-oslo.utils (7.1.0-2) ... Setting up python3-dogpile.cache (1.3.1-1) ... Setting up python3-oauthlib (3.2.2-1) ... Setting up python3-secretstorage (3.3.3-3) ... Setting up python3-requests-mock (1.11.0-1) ... Setting up python3-botocore (1.34.46+repack-1) ... Setting up python3-keyring (25.2.1-1) ... Setting up python3-jsonschema-specifications (2023.12.1-1) ... Setting up python3-keystoneauth1 (5.6.0-2) ... Setting up python3-s3transfer (0.10.1-1) ... Setting up python3-oslo.serialization (5.4.0-2) ... Setting up python3-novaclient (2:18.6.0-3) ... Setting up python3-jsonschema (4.19.2-3) ... Setting up python3-boto3 (1.34.46+dfsg-1) ... Setting up python3-openstacksdk (3.0.0-3) ... Setting up python3-cinderclient (1:9.5.0-2) ... Setting up python3-os-client-config (2.1.0-2) ... Setting up python3-warlock (2.0.1-4) ... Setting up python3-glanceclient (1:4.5.0-2) ... Setting up python3-testtools (2.7.1-3) ... Setting up python3-subunit (1.4.2-3) ... Setting up python3-fixtures (4.1.0-2) ... Setting up subunit (1.4.2-3) ... Processing triggers for libc-bin (2.38-12) ... Processing triggers for sgml-base (1.31) ... Setting up docutils-common (0.20.1+dfsg-3) ... Processing triggers for sgml-base (1.31) ... Setting up python3-docutils (0.20.1+dfsg-3) ... Setting up python3-cliff (4.6.0-2) ... Setting up python3-oslo.config (1:9.4.0-2) ... Setting up python3-keystoneclient (1:5.4.0-2) ... Setting up python3-osc-lib (3.0.1-2) ... Setting up python3-sphinx (7.2.6-8) ... Setting up python3-oslo.log (5.5.1-2) ... Setting up python3-pycadf (3.1.1-3) ... Setting up python3-stestr (4.1.0-2) ... Setting up python3-openstackclient (6.6.0-3) ... Setting up python3-openstackdocstheme (3.2.0-1) ... Setting up python3-neutronclient (1:11.2.0-3) ... Setting up python3-oslo.cache (3.7.0-2) ... Setting up python3-os-testr (3.0.0-3) ... Setting up python3-keystonemiddleware (10.6.0-2) ... Setting up python3-os-api-ref (2.3.0+ds1-3) ... Processing triggers for ca-certificates (20240203) ... Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done. Reading package lists... Building dependency tree... Reading state information... Reading extended state information... Initializing package states... Writing extended state information... Building tag database... -> Finished parsing the build-deps I: Building the package I: Running cd /build/reproducible-path/swift-2.33.0/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games" HOME="/nonexistent/first-build" dpkg-buildpackage -us -uc -b && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games" HOME="/nonexistent/first-build" dpkg-genchanges -S > ../swift_2.33.0-4_source.changes dpkg-buildpackage: info: source package swift dpkg-buildpackage: info: source version 2.33.0-4 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Thomas Goirand dpkg-source --before-build . dpkg-buildpackage: info: host architecture amd64 dpkg-source: info: using options from swift-2.33.0/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/ --extend-diff-ignore=^[.]gitreview$ debian/rules clean make: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh clean --buildsystem=pybuild --with python3,sphinxdoc debian/rules override_dh_auto_clean make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' rm -rf build .stestr *.egg-info .pybuild find . -iname '*.pyc' -delete for i in $(find . -type d -iname __pycache__) ; do rm -rf $i ; done make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_autoreconf_clean -O--buildsystem=pybuild dh_clean -O--buildsystem=pybuild debian/rules binary make: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh binary --buildsystem=pybuild --with python3,sphinxdoc dh_update_autotools_config -O--buildsystem=pybuild dh_autoreconf -O--buildsystem=pybuild dh_auto_configure -O--buildsystem=pybuild I: pybuild base:311: python3.12 setup.py config /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running config I: pybuild base:311: python3.11 setup.py config /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running config debian/rules override_dh_auto_build make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh_auto_build I: pybuild base:311: /usr/bin/python3.12 setup.py build /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running build running build_py creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kmip_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kms_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/encrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/crypto_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/decrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/reconciler.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/sync_store.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/sharder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy copying swift/proxy/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy copying swift/proxy/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/reaper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/bucket.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/tagging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/s3_acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/service.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_delete.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/location.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/object_lock.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_upload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/s3response.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/subresource.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_handlers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/etree.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3api.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/exception.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3token.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3request.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/html_viewer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/profile_model.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/shard-info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/drive_full_checker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/relinker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/container_deleter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/form_signature.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/recon_cron.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/reload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/manage_shard_ranges.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/ringcomposer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/ringbuilder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/dispersion_report.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/ring_builder_analyzer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/account.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/container.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/composite_builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/ring.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/middleware/keystoneauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/xprofile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/memcache.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/read_only.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/list_endpoints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/listing_formats.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/proxy_logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/copy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/container_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/symlink.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/staticweb.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/crossdomain.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/slo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/tempauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/container_sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/domain_remap.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/backend_ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/etag_quoter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/formpost.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/bulk.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/gatekeeper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/tempurl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/account_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/name_check.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/dlo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/catch_errors.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/healthcheck.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/cname_lookup.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/libc.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/timestamp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/ipaddrs.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/legacy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/object_versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/mem_diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/ssync_receiver.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/reconstructor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/mem_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/ssync_sender.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/expirer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/common/registry.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/container_sync_realms.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/db_auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/error_limiter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/daemon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/direct_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/constraints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/digest.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/swob.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/db_replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/header_key_dict.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/wsgi.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/storage_policy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/linkat.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/internal_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/bufferedhttp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/request_helpers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/db.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/splice.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/http_protocol.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/manager.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/base_storage_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/memcached.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/http.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj/watchers copying swift/obj/watchers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj/watchers copying swift/obj/watchers/dark_data.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj/watchers running egg_info creating swift.egg-info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Processing SOURCES.txt writing manifest file 'swift.egg-info/SOURCES.txt' warning: no files found matching 'ChangeLog' warning: no previously-included files found matching '.gitignore' warning: no previously-included files found matching '.gitreview' warning: no previously-included files matching '*.pyc' found anywhere in distribution reading manifest template 'MANIFEST.in' warning: no files found matching 'babel.cfg' adding license file 'LICENSE' adding license file 'AUTHORS' writing manifest file 'swift.egg-info/SOURCES.txt' /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/access_control_policy.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/bucket_logging_status.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/common.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_object_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_part_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/error.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_bucket_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_parts_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_versions_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/location_constraint.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/versioning_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/de creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/de/LC_MESSAGES copying swift/locale/de/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/de/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/en_GB creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/en_GB/LC_MESSAGES copying swift/locale/en_GB/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/en_GB/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/es creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/es/LC_MESSAGES copying swift/locale/es/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/es/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/fr creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/fr/LC_MESSAGES copying swift/locale/fr/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/fr/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/it creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/it/LC_MESSAGES copying swift/locale/it/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/it/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ja creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ja/LC_MESSAGES copying swift/locale/ja/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ja/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ko_KR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ko_KR/LC_MESSAGES copying swift/locale/ko_KR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ko_KR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/pt_BR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/pt_BR/LC_MESSAGES copying swift/locale/pt_BR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/pt_BR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ru creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ru/LC_MESSAGES copying swift/locale/ru/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ru/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/tr_TR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/tr_TR/LC_MESSAGES copying swift/locale/tr_TR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/tr_TR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_CN creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_CN/LC_MESSAGES copying swift/locale/zh_CN/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_CN/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_TW creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_TW/LC_MESSAGES copying swift/locale/zh_TW/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_TW/LC_MESSAGES running build_scripts creating build creating build/scripts-3.12 copying and adjusting bin/swift-account-audit -> build/scripts-3.12 copying and adjusting bin/swift-account-auditor -> build/scripts-3.12 copying and adjusting bin/swift-account-info -> build/scripts-3.12 copying and adjusting bin/swift-account-reaper -> build/scripts-3.12 copying and adjusting bin/swift-account-replicator -> build/scripts-3.12 copying and adjusting bin/swift-account-server -> build/scripts-3.12 copying and adjusting bin/swift-config -> build/scripts-3.12 copying and adjusting bin/swift-container-auditor -> build/scripts-3.12 copying and adjusting bin/swift-container-info -> build/scripts-3.12 copying and adjusting bin/swift-container-replicator -> build/scripts-3.12 copying and adjusting bin/swift-container-server -> build/scripts-3.12 copying and adjusting bin/swift-container-sharder -> build/scripts-3.12 copying and adjusting bin/swift-container-sync -> build/scripts-3.12 copying and adjusting bin/swift-container-updater -> build/scripts-3.12 copying and adjusting bin/swift-container-reconciler -> build/scripts-3.12 copying and adjusting bin/swift-reconciler-enqueue -> build/scripts-3.12 copying and adjusting bin/swift-dispersion-populate -> build/scripts-3.12 copying and adjusting bin/swift-dispersion-report -> build/scripts-3.12 copying and adjusting bin/swift-drive-audit -> build/scripts-3.12 copying and adjusting bin/swift-form-signature -> build/scripts-3.12 copying and adjusting bin/swift-get-nodes -> build/scripts-3.12 copying and adjusting bin/swift-init -> build/scripts-3.12 copying and adjusting bin/swift-object-auditor -> build/scripts-3.12 copying and adjusting bin/swift-object-expirer -> build/scripts-3.12 copying and adjusting bin/swift-object-info -> build/scripts-3.12 copying and adjusting bin/swift-object-replicator -> build/scripts-3.12 copying and adjusting bin/swift-object-reconstructor -> build/scripts-3.12 copying and adjusting bin/swift-object-relinker -> build/scripts-3.12 copying and adjusting bin/swift-object-server -> build/scripts-3.12 copying and adjusting bin/swift-object-updater -> build/scripts-3.12 copying and adjusting bin/swift-oldies -> build/scripts-3.12 copying and adjusting bin/swift-orphans -> build/scripts-3.12 copying and adjusting bin/swift-proxy-server -> build/scripts-3.12 copying and adjusting bin/swift-recon -> build/scripts-3.12 copying and adjusting bin/swift-recon-cron -> build/scripts-3.12 copying and adjusting bin/swift-ring-builder -> build/scripts-3.12 copying and adjusting bin/swift-ring-builder-analyzer -> build/scripts-3.12 copying and adjusting bin/swift-ring-composer -> build/scripts-3.12 changing mode of build/scripts-3.12/swift-account-audit from 644 to 755 changing mode of build/scripts-3.12/swift-account-auditor from 644 to 755 changing mode of build/scripts-3.12/swift-account-info from 644 to 755 changing mode of build/scripts-3.12/swift-account-reaper from 644 to 755 changing mode of build/scripts-3.12/swift-account-replicator from 644 to 755 changing mode of build/scripts-3.12/swift-account-server from 644 to 755 changing mode of build/scripts-3.12/swift-config from 644 to 755 changing mode of build/scripts-3.12/swift-container-auditor from 644 to 755 changing mode of build/scripts-3.12/swift-container-info from 644 to 755 changing mode of build/scripts-3.12/swift-container-replicator from 644 to 755 changing mode of build/scripts-3.12/swift-container-server from 644 to 755 changing mode of build/scripts-3.12/swift-container-sharder from 644 to 755 changing mode of build/scripts-3.12/swift-container-sync from 644 to 755 changing mode of build/scripts-3.12/swift-container-updater from 644 to 755 changing mode of build/scripts-3.12/swift-container-reconciler from 644 to 755 changing mode of build/scripts-3.12/swift-reconciler-enqueue from 644 to 755 changing mode of build/scripts-3.12/swift-dispersion-populate from 644 to 755 changing mode of build/scripts-3.12/swift-dispersion-report from 644 to 755 changing mode of build/scripts-3.12/swift-drive-audit from 644 to 755 changing mode of build/scripts-3.12/swift-form-signature from 644 to 755 changing mode of build/scripts-3.12/swift-get-nodes from 644 to 755 changing mode of build/scripts-3.12/swift-init from 644 to 755 changing mode of build/scripts-3.12/swift-object-auditor from 644 to 755 changing mode of build/scripts-3.12/swift-object-expirer from 644 to 755 changing mode of build/scripts-3.12/swift-object-info from 644 to 755 changing mode of build/scripts-3.12/swift-object-replicator from 644 to 755 changing mode of build/scripts-3.12/swift-object-reconstructor from 644 to 755 changing mode of build/scripts-3.12/swift-object-relinker from 644 to 755 changing mode of build/scripts-3.12/swift-object-server from 644 to 755 changing mode of build/scripts-3.12/swift-object-updater from 644 to 755 changing mode of build/scripts-3.12/swift-oldies from 644 to 755 changing mode of build/scripts-3.12/swift-orphans from 644 to 755 changing mode of build/scripts-3.12/swift-proxy-server from 644 to 755 changing mode of build/scripts-3.12/swift-recon from 644 to 755 changing mode of build/scripts-3.12/swift-recon-cron from 644 to 755 changing mode of build/scripts-3.12/swift-ring-builder from 644 to 755 changing mode of build/scripts-3.12/swift-ring-builder-analyzer from 644 to 755 changing mode of build/scripts-3.12/swift-ring-composer from 644 to 755 I: pybuild base:311: /usr/bin/python3 setup.py build /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running build running build_py creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/mem_diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/ssync_receiver.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/reconstructor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/mem_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/ssync_sender.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/expirer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/legacy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/object_versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/html_viewer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/profile_model.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj/watchers copying swift/obj/watchers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj/watchers copying swift/obj/watchers/dark_data.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj/watchers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy copying swift/proxy/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy copying swift/proxy/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/composite_builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/ring.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/middleware/keystoneauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/xprofile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/memcache.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/read_only.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/list_endpoints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/listing_formats.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/proxy_logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/copy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/container_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/symlink.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/staticweb.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/crossdomain.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/slo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/tempauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/container_sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/domain_remap.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/backend_ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/etag_quoter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/formpost.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/bulk.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/gatekeeper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/tempurl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/account_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/name_check.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/dlo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/catch_errors.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/healthcheck.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/cname_lookup.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kmip_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kms_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/encrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/crypto_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/decrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/account.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/container.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/common/registry.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/container_sync_realms.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/db_auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/error_limiter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/daemon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/direct_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/constraints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/digest.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/swob.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/db_replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/header_key_dict.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/wsgi.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/storage_policy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/linkat.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/internal_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/bufferedhttp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/request_helpers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/db.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/splice.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/http_protocol.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/manager.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/base_storage_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/memcached.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/http.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/bucket.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/tagging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/s3_acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/service.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_delete.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/location.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/object_lock.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_upload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/reaper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/common/middleware/s3api/s3response.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/subresource.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_handlers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/etree.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3api.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/exception.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3token.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3request.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/reconciler.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/sync_store.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/sharder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/shard-info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/drive_full_checker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/relinker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/container_deleter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/form_signature.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/recon_cron.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/reload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/manage_shard_ranges.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/ringcomposer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/ringbuilder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/dispersion_report.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/ring_builder_analyzer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/libc.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/timestamp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/ipaddrs.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils running egg_info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Reusing existing SOURCES.txt /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/de creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/de/LC_MESSAGES copying swift/locale/de/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/de/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/en_GB creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/en_GB/LC_MESSAGES copying swift/locale/en_GB/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/en_GB/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/es creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/es/LC_MESSAGES copying swift/locale/es/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/es/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/fr creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/fr/LC_MESSAGES copying swift/locale/fr/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/fr/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/it creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/it/LC_MESSAGES copying swift/locale/it/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/it/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ja creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ja/LC_MESSAGES copying swift/locale/ja/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ja/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ko_KR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ko_KR/LC_MESSAGES copying swift/locale/ko_KR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ko_KR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/pt_BR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/pt_BR/LC_MESSAGES copying swift/locale/pt_BR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/pt_BR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ru creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ru/LC_MESSAGES copying swift/locale/ru/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ru/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/tr_TR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/tr_TR/LC_MESSAGES copying swift/locale/tr_TR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/tr_TR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_CN creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_CN/LC_MESSAGES copying swift/locale/zh_CN/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_CN/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_TW creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_TW/LC_MESSAGES copying swift/locale/zh_TW/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_TW/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/access_control_policy.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/bucket_logging_status.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/common.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_object_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_part_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/error.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_bucket_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_parts_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_versions_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/location_constraint.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/versioning_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema running build_scripts creating build/scripts-3.11 copying and adjusting bin/swift-account-audit -> build/scripts-3.11 copying and adjusting bin/swift-account-auditor -> build/scripts-3.11 copying and adjusting bin/swift-account-info -> build/scripts-3.11 copying and adjusting bin/swift-account-reaper -> build/scripts-3.11 copying and adjusting bin/swift-account-replicator -> build/scripts-3.11 copying and adjusting bin/swift-account-server -> build/scripts-3.11 copying and adjusting bin/swift-config -> build/scripts-3.11 copying and adjusting bin/swift-container-auditor -> build/scripts-3.11 copying and adjusting bin/swift-container-info -> build/scripts-3.11 copying and adjusting bin/swift-container-replicator -> build/scripts-3.11 copying and adjusting bin/swift-container-server -> build/scripts-3.11 copying and adjusting bin/swift-container-sharder -> build/scripts-3.11 copying and adjusting bin/swift-container-sync -> build/scripts-3.11 copying and adjusting bin/swift-container-updater -> build/scripts-3.11 copying and adjusting bin/swift-container-reconciler -> build/scripts-3.11 copying and adjusting bin/swift-reconciler-enqueue -> build/scripts-3.11 copying and adjusting bin/swift-dispersion-populate -> build/scripts-3.11 copying and adjusting bin/swift-dispersion-report -> build/scripts-3.11 copying and adjusting bin/swift-drive-audit -> build/scripts-3.11 copying and adjusting bin/swift-form-signature -> build/scripts-3.11 copying and adjusting bin/swift-get-nodes -> build/scripts-3.11 copying and adjusting bin/swift-init -> build/scripts-3.11 copying and adjusting bin/swift-object-auditor -> build/scripts-3.11 copying and adjusting bin/swift-object-expirer -> build/scripts-3.11 copying and adjusting bin/swift-object-info -> build/scripts-3.11 copying and adjusting bin/swift-object-replicator -> build/scripts-3.11 copying and adjusting bin/swift-object-reconstructor -> build/scripts-3.11 copying and adjusting bin/swift-object-relinker -> build/scripts-3.11 copying and adjusting bin/swift-object-server -> build/scripts-3.11 copying and adjusting bin/swift-object-updater -> build/scripts-3.11 copying and adjusting bin/swift-oldies -> build/scripts-3.11 copying and adjusting bin/swift-orphans -> build/scripts-3.11 copying and adjusting bin/swift-proxy-server -> build/scripts-3.11 copying and adjusting bin/swift-recon -> build/scripts-3.11 copying and adjusting bin/swift-recon-cron -> build/scripts-3.11 copying and adjusting bin/swift-ring-builder -> build/scripts-3.11 copying and adjusting bin/swift-ring-builder-analyzer -> build/scripts-3.11 copying and adjusting bin/swift-ring-composer -> build/scripts-3.11 changing mode of build/scripts-3.11/swift-account-audit from 644 to 755 changing mode of build/scripts-3.11/swift-account-auditor from 644 to 755 changing mode of build/scripts-3.11/swift-account-info from 644 to 755 changing mode of build/scripts-3.11/swift-account-reaper from 644 to 755 changing mode of build/scripts-3.11/swift-account-replicator from 644 to 755 changing mode of build/scripts-3.11/swift-account-server from 644 to 755 changing mode of build/scripts-3.11/swift-config from 644 to 755 changing mode of build/scripts-3.11/swift-container-auditor from 644 to 755 changing mode of build/scripts-3.11/swift-container-info from 644 to 755 changing mode of build/scripts-3.11/swift-container-replicator from 644 to 755 changing mode of build/scripts-3.11/swift-container-server from 644 to 755 changing mode of build/scripts-3.11/swift-container-sharder from 644 to 755 changing mode of build/scripts-3.11/swift-container-sync from 644 to 755 changing mode of build/scripts-3.11/swift-container-updater from 644 to 755 changing mode of build/scripts-3.11/swift-container-reconciler from 644 to 755 changing mode of build/scripts-3.11/swift-reconciler-enqueue from 644 to 755 changing mode of build/scripts-3.11/swift-dispersion-populate from 644 to 755 changing mode of build/scripts-3.11/swift-dispersion-report from 644 to 755 changing mode of build/scripts-3.11/swift-drive-audit from 644 to 755 changing mode of build/scripts-3.11/swift-form-signature from 644 to 755 changing mode of build/scripts-3.11/swift-get-nodes from 644 to 755 changing mode of build/scripts-3.11/swift-init from 644 to 755 changing mode of build/scripts-3.11/swift-object-auditor from 644 to 755 changing mode of build/scripts-3.11/swift-object-expirer from 644 to 755 changing mode of build/scripts-3.11/swift-object-info from 644 to 755 changing mode of build/scripts-3.11/swift-object-replicator from 644 to 755 changing mode of build/scripts-3.11/swift-object-reconstructor from 644 to 755 changing mode of build/scripts-3.11/swift-object-relinker from 644 to 755 changing mode of build/scripts-3.11/swift-object-server from 644 to 755 changing mode of build/scripts-3.11/swift-object-updater from 644 to 755 changing mode of build/scripts-3.11/swift-oldies from 644 to 755 changing mode of build/scripts-3.11/swift-orphans from 644 to 755 changing mode of build/scripts-3.11/swift-proxy-server from 644 to 755 changing mode of build/scripts-3.11/swift-recon from 644 to 755 changing mode of build/scripts-3.11/swift-recon-cron from 644 to 755 changing mode of build/scripts-3.11/swift-ring-builder from 644 to 755 changing mode of build/scripts-3.11/swift-ring-builder-analyzer from 644 to 755 changing mode of build/scripts-3.11/swift-ring-composer from 644 to 755 /usr/share/openstack-pkg-tools/pkgos_insert_include pkgos_func python3-swift.postinst set -e ; set -x ; \ for i in debian/swift-account.swift-account-auditor.init.in \ debian/swift-account.swift-account-reaper.init.in \ debian/swift-account.swift-account-replicator.init.in \ debian/swift-container.swift-container-auditor.init.in \ debian/swift-container.swift-container-reconciler.init.in \ debian/swift-container.swift-container-replicator.init.in \ debian/swift-container.swift-container-sharder.init.in \ debian/swift-container.swift-container-sync.init.in \ debian/swift-container.swift-container-updater.init.in \ debian/swift-object-expirer.swift-object-expirer.init.in \ debian/swift-object.swift-object-auditor.init.in \ debian/swift-object.swift-object.init.in \ debian/swift-object.swift-object-reconstructor.init.in \ debian/swift-object.swift-object-replicator.init.in \ debian/swift-object.swift-object-updater.init.in \ ; do \ debian/gen-init $i ; \ done + debian/gen-init debian/swift-account.swift-account-auditor.init.in + debian/gen-init debian/swift-account.swift-account-reaper.init.in + debian/gen-init debian/swift-account.swift-account-replicator.init.in + debian/gen-init debian/swift-container.swift-container-auditor.init.in + debian/gen-init debian/swift-container.swift-container-reconciler.init.in + debian/gen-init debian/swift-container.swift-container-replicator.init.in + debian/gen-init debian/swift-container.swift-container-sharder.init.in + debian/gen-init debian/swift-container.swift-container-sync.init.in + debian/gen-init debian/swift-container.swift-container-updater.init.in + debian/gen-init debian/swift-object-expirer.swift-object-expirer.init.in + debian/gen-init debian/swift-object.swift-object-auditor.init.in + debian/gen-init debian/swift-object.swift-object.init.in + debian/gen-init debian/swift-object.swift-object-reconstructor.init.in + debian/gen-init debian/swift-object.swift-object-replicator.init.in + debian/gen-init debian/swift-object.swift-object-updater.init.in set -e ; set -x ; \ for i in debian/swift-proxy.swift-proxy.init \ debian/swift-account.swift-account.init \ debian/swift-container.swift-container.init \ ; do \ if [ $DEB_USE_UWSGI = "yes" ] ; then \ cp $i.in $i ; \ cat /usr/share/openstack-pkg-tools/init-script-template >>$i ; \ pkgos-gen-systemd-unit $i.in ; \ else \ CONFIG=`echo $i | cut -d/ -f2 | cut -d. -f1 | sed -e s/swift-//`-server debian/gen-init $i.in ; \ fi ; \ done + [ no = yes ] + echo debian/swift-proxy.swift-proxy.init + cut -d/ -f2 + + sedcut -d. -f1 -e s/swift-// + CONFIG=proxy-server debian/gen-init debian/swift-proxy.swift-proxy.init.in + [ no = yes ] + echo debian/swift-account.swift-account.init + cut -d/ -f2 + + cut -d. -f1 sed -e s/swift-// + CONFIG=account-server debian/gen-init debian/swift-account.swift-account.init.in + [ no = yes ] + echo debian/swift-container.swift-container.init + + cut -d. -f1 + sed -e s/swift-// cut -d/ -f2 + CONFIG=container-server debian/gen-init debian/swift-container.swift-container.init.in make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' debian/rules override_dh_auto_test make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' echo "Do nothing..." Do nothing... make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' create-stamp debian/debhelper-build-stamp dh_testroot -O--buildsystem=pybuild dh_prep -O--buildsystem=pybuild debian/rules override_dh_auto_install make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' pkgos-dh_auto_install --no-py2 --in-tmp + PKGOS_IN_TMP=no + echo WARNING: --no-py2 is deprecated and always on. WARNING: --no-py2 is deprecated and always on. + shift + PKGOS_IN_TMP=yes + shift + dpkg-parsechangelog -SSource + SRC_PKG_NAME=swift + + echo swift sed s/python-// + PY_MODULE_NAME=swift + py3versions -vr + PYTHON3S=3.12 3.11 + [ yes = yes ] + TARGET_DIR=tmp + pwd + python3.12 setup.py install --install-layout=deb --root /build/reproducible-path/swift-2.33.0/debian/tmp /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running install running build running build_py creating build/lib creating build/lib/swift creating build/lib/swift/common creating build/lib/swift/common/middleware creating build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/kmip_keymaster.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/__init__.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/kms_keymaster.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/keymaster.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/encrypter.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/crypto_utils.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/decrypter.py -> build/lib/swift/common/middleware/crypto creating build/lib/swift/common/ring copying swift/common/ring/builder.py -> build/lib/swift/common/ring copying swift/common/ring/__init__.py -> build/lib/swift/common/ring copying swift/common/ring/composite_builder.py -> build/lib/swift/common/ring copying swift/common/ring/utils.py -> build/lib/swift/common/ring copying swift/common/ring/ring.py -> build/lib/swift/common/ring creating build/lib/swift/container copying swift/container/reconciler.py -> build/lib/swift/container copying swift/container/updater.py -> build/lib/swift/container copying swift/container/__init__.py -> build/lib/swift/container copying swift/container/auditor.py -> build/lib/swift/container copying swift/container/backend.py -> build/lib/swift/container copying swift/container/sync_store.py -> build/lib/swift/container copying swift/container/server.py -> build/lib/swift/container copying swift/container/replicator.py -> build/lib/swift/container copying swift/container/sharder.py -> build/lib/swift/container copying swift/container/sync.py -> build/lib/swift/container creating build/lib/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/legacy.py -> build/lib/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/__init__.py -> build/lib/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/object_versioning.py -> build/lib/swift/common/middleware/versioned_writes creating build/lib/swift/common/utils copying swift/common/utils/libc.py -> build/lib/swift/common/utils copying swift/common/utils/timestamp.py -> build/lib/swift/common/utils copying swift/common/utils/__init__.py -> build/lib/swift/common/utils copying swift/common/utils/ipaddrs.py -> build/lib/swift/common/utils creating build/lib/swift/obj copying swift/obj/mem_diskfile.py -> build/lib/swift/obj copying swift/obj/ssync_receiver.py -> build/lib/swift/obj copying swift/obj/reconstructor.py -> build/lib/swift/obj copying swift/obj/mem_server.py -> build/lib/swift/obj copying swift/obj/ssync_sender.py -> build/lib/swift/obj copying swift/obj/updater.py -> build/lib/swift/obj copying swift/obj/__init__.py -> build/lib/swift/obj copying swift/obj/auditor.py -> build/lib/swift/obj copying swift/obj/diskfile.py -> build/lib/swift/obj copying swift/obj/expirer.py -> build/lib/swift/obj copying swift/obj/server.py -> build/lib/swift/obj copying swift/obj/replicator.py -> build/lib/swift/obj copying swift/common/registry.py -> build/lib/swift/common copying swift/common/container_sync_realms.py -> build/lib/swift/common copying swift/common/db_auditor.py -> build/lib/swift/common copying swift/common/error_limiter.py -> build/lib/swift/common copying swift/common/daemon.py -> build/lib/swift/common copying swift/common/direct_client.py -> build/lib/swift/common copying swift/common/constraints.py -> build/lib/swift/common copying swift/common/digest.py -> build/lib/swift/common copying swift/common/swob.py -> build/lib/swift/common copying swift/common/db_replicator.py -> build/lib/swift/common copying swift/common/header_key_dict.py -> build/lib/swift/common copying swift/common/wsgi.py -> build/lib/swift/common copying swift/common/storage_policy.py -> build/lib/swift/common copying swift/common/exceptions.py -> build/lib/swift/common copying swift/common/linkat.py -> build/lib/swift/common copying swift/common/__init__.py -> build/lib/swift/common copying swift/common/internal_client.py -> build/lib/swift/common copying swift/common/bufferedhttp.py -> build/lib/swift/common copying swift/common/request_helpers.py -> build/lib/swift/common copying swift/common/db.py -> build/lib/swift/common copying swift/common/splice.py -> build/lib/swift/common copying swift/common/http_protocol.py -> build/lib/swift/common copying swift/common/manager.py -> build/lib/swift/common copying swift/common/base_storage_server.py -> build/lib/swift/common copying swift/common/recon.py -> build/lib/swift/common copying swift/common/memcached.py -> build/lib/swift/common copying swift/common/http.py -> build/lib/swift/common creating build/lib/swift/account copying swift/account/reaper.py -> build/lib/swift/account copying swift/account/__init__.py -> build/lib/swift/account copying swift/account/auditor.py -> build/lib/swift/account copying swift/account/backend.py -> build/lib/swift/account copying swift/account/server.py -> build/lib/swift/account copying swift/account/utils.py -> build/lib/swift/account copying swift/account/replicator.py -> build/lib/swift/account creating build/lib/swift/proxy creating build/lib/swift/proxy/controllers copying swift/proxy/controllers/base.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/account.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/container.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/__init__.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/info.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/obj.py -> build/lib/swift/proxy/controllers creating build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3response.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/subresource.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_utils.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_handlers.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/__init__.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/etree.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3api.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/exception.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3token.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/utils.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3request.py -> build/lib/swift/common/middleware/s3api creating build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/html_viewer.py -> build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/exceptions.py -> build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/__init__.py -> build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/profile_model.py -> build/lib/swift/common/middleware/x_profile creating build/lib/swift/obj/watchers copying swift/obj/watchers/__init__.py -> build/lib/swift/obj/watchers copying swift/obj/watchers/dark_data.py -> build/lib/swift/obj/watchers copying swift/common/middleware/keystoneauth.py -> build/lib/swift/common/middleware copying swift/common/middleware/xprofile.py -> build/lib/swift/common/middleware copying swift/common/middleware/memcache.py -> build/lib/swift/common/middleware copying swift/common/middleware/read_only.py -> build/lib/swift/common/middleware copying swift/common/middleware/list_endpoints.py -> build/lib/swift/common/middleware copying swift/common/middleware/listing_formats.py -> build/lib/swift/common/middleware copying swift/common/middleware/proxy_logging.py -> build/lib/swift/common/middleware copying swift/common/middleware/copy.py -> build/lib/swift/common/middleware copying swift/common/middleware/container_quotas.py -> build/lib/swift/common/middleware copying swift/common/middleware/symlink.py -> build/lib/swift/common/middleware copying swift/common/middleware/acl.py -> build/lib/swift/common/middleware copying swift/common/middleware/staticweb.py -> build/lib/swift/common/middleware copying swift/common/middleware/crossdomain.py -> build/lib/swift/common/middleware copying swift/common/middleware/__init__.py -> build/lib/swift/common/middleware copying swift/common/middleware/slo.py -> build/lib/swift/common/middleware copying swift/common/middleware/tempauth.py -> build/lib/swift/common/middleware copying swift/common/middleware/container_sync.py -> build/lib/swift/common/middleware copying swift/common/middleware/domain_remap.py -> build/lib/swift/common/middleware copying swift/common/middleware/backend_ratelimit.py -> build/lib/swift/common/middleware copying swift/common/middleware/etag_quoter.py -> build/lib/swift/common/middleware copying swift/common/middleware/formpost.py -> build/lib/swift/common/middleware copying swift/common/middleware/bulk.py -> build/lib/swift/common/middleware copying swift/common/middleware/gatekeeper.py -> build/lib/swift/common/middleware copying swift/common/middleware/tempurl.py -> build/lib/swift/common/middleware copying swift/common/middleware/account_quotas.py -> build/lib/swift/common/middleware copying swift/common/middleware/name_check.py -> build/lib/swift/common/middleware copying swift/common/middleware/dlo.py -> build/lib/swift/common/middleware copying swift/common/middleware/catch_errors.py -> build/lib/swift/common/middleware copying swift/common/middleware/healthcheck.py -> build/lib/swift/common/middleware copying swift/common/middleware/cname_lookup.py -> build/lib/swift/common/middleware copying swift/common/middleware/recon.py -> build/lib/swift/common/middleware copying swift/common/middleware/ratelimit.py -> build/lib/swift/common/middleware copying swift/proxy/__init__.py -> build/lib/swift/proxy copying swift/proxy/server.py -> build/lib/swift/proxy creating build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/logging.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/bucket.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/base.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/acl.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/tagging.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/s3_acl.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/service.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_delete.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/__init__.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/location.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/obj.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/object_lock.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_upload.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/versioning.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/__init__.py -> build/lib/swift creating build/lib/swift/cli copying swift/cli/shard-info.py -> build/lib/swift/cli copying swift/cli/drive_full_checker.py -> build/lib/swift/cli copying swift/cli/relinker.py -> build/lib/swift/cli copying swift/cli/container_deleter.py -> build/lib/swift/cli copying swift/cli/__init__.py -> build/lib/swift/cli copying swift/cli/form_signature.py -> build/lib/swift/cli copying swift/cli/recon_cron.py -> build/lib/swift/cli copying swift/cli/reload.py -> build/lib/swift/cli copying swift/cli/info.py -> build/lib/swift/cli copying swift/cli/manage_shard_ranges.py -> build/lib/swift/cli copying swift/cli/ringcomposer.py -> build/lib/swift/cli copying swift/cli/ringbuilder.py -> build/lib/swift/cli copying swift/cli/dispersion_report.py -> build/lib/swift/cli copying swift/cli/recon.py -> build/lib/swift/cli copying swift/cli/ring_builder_analyzer.py -> build/lib/swift/cli running egg_info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Reusing existing SOURCES.txt /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) creating build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/access_control_policy.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/bucket_logging_status.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/common.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_object_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_part_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/error.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_bucket_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_parts_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_versions_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/location_constraint.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/versioning_configuration.rng -> build/lib/swift/common/middleware/s3api/schema creating build/lib/swift/locale creating build/lib/swift/locale/de creating build/lib/swift/locale/de/LC_MESSAGES copying swift/locale/de/LC_MESSAGES/swift.po -> build/lib/swift/locale/de/LC_MESSAGES creating build/lib/swift/locale/en_GB creating build/lib/swift/locale/en_GB/LC_MESSAGES copying swift/locale/en_GB/LC_MESSAGES/swift.po -> build/lib/swift/locale/en_GB/LC_MESSAGES creating build/lib/swift/locale/es creating build/lib/swift/locale/es/LC_MESSAGES copying swift/locale/es/LC_MESSAGES/swift.po -> build/lib/swift/locale/es/LC_MESSAGES creating build/lib/swift/locale/fr creating build/lib/swift/locale/fr/LC_MESSAGES copying swift/locale/fr/LC_MESSAGES/swift.po -> build/lib/swift/locale/fr/LC_MESSAGES creating build/lib/swift/locale/it creating build/lib/swift/locale/it/LC_MESSAGES copying swift/locale/it/LC_MESSAGES/swift.po -> build/lib/swift/locale/it/LC_MESSAGES creating build/lib/swift/locale/ja creating build/lib/swift/locale/ja/LC_MESSAGES copying swift/locale/ja/LC_MESSAGES/swift.po -> build/lib/swift/locale/ja/LC_MESSAGES creating build/lib/swift/locale/ko_KR creating build/lib/swift/locale/ko_KR/LC_MESSAGES copying swift/locale/ko_KR/LC_MESSAGES/swift.po -> build/lib/swift/locale/ko_KR/LC_MESSAGES creating build/lib/swift/locale/pt_BR creating build/lib/swift/locale/pt_BR/LC_MESSAGES copying swift/locale/pt_BR/LC_MESSAGES/swift.po -> build/lib/swift/locale/pt_BR/LC_MESSAGES creating build/lib/swift/locale/ru creating build/lib/swift/locale/ru/LC_MESSAGES copying swift/locale/ru/LC_MESSAGES/swift.po -> build/lib/swift/locale/ru/LC_MESSAGES creating build/lib/swift/locale/tr_TR creating build/lib/swift/locale/tr_TR/LC_MESSAGES copying swift/locale/tr_TR/LC_MESSAGES/swift.po -> build/lib/swift/locale/tr_TR/LC_MESSAGES creating build/lib/swift/locale/zh_CN creating build/lib/swift/locale/zh_CN/LC_MESSAGES copying swift/locale/zh_CN/LC_MESSAGES/swift.po -> build/lib/swift/locale/zh_CN/LC_MESSAGES creating build/lib/swift/locale/zh_TW creating build/lib/swift/locale/zh_TW/LC_MESSAGES copying swift/locale/zh_TW/LC_MESSAGES/swift.po -> build/lib/swift/locale/zh_TW/LC_MESSAGES running build_scripts running install_lib creating /build/reproducible-path/swift-2.33.0/debian/tmp creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3 creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_CN creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_CN/LC_MESSAGES copying build/lib/swift/locale/zh_CN/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_CN/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/it creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/it/LC_MESSAGES copying build/lib/swift/locale/it/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/it/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/en_GB creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/en_GB/LC_MESSAGES copying build/lib/swift/locale/en_GB/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/en_GB/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/es creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/es/LC_MESSAGES copying build/lib/swift/locale/es/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/es/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/de creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/de/LC_MESSAGES copying build/lib/swift/locale/de/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/de/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/fr creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/fr/LC_MESSAGES copying build/lib/swift/locale/fr/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/fr/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ja creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ja/LC_MESSAGES copying build/lib/swift/locale/ja/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ja/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/tr_TR creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/tr_TR/LC_MESSAGES copying build/lib/swift/locale/tr_TR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/tr_TR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/pt_BR creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/pt_BR/LC_MESSAGES copying build/lib/swift/locale/pt_BR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/pt_BR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ko_KR creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ko_KR/LC_MESSAGES copying build/lib/swift/locale/ko_KR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ko_KR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ru creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ru/LC_MESSAGES copying build/lib/swift/locale/ru/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ru/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_TW creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_TW/LC_MESSAGES copying build/lib/swift/locale/zh_TW/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_TW/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy copying build/lib/swift/proxy/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/base.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/account.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/container.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/info.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy copying build/lib/swift/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/reconciler.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/updater.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/backend.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/sync_store.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/sharder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/sync.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/registry.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/container_sync_realms.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/db_auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/error_limiter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/daemon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/direct_client.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/constraints.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/libc.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/timestamp.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/ipaddrs.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/digest.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/swob.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/db_replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/header_key_dict.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/wsgi.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/storage_policy.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/keystoneauth.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/xprofile.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/memcache.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/read_only.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/list_endpoints.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/listing_formats.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/proxy_logging.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/copy.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/container_quotas.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/symlink.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/acl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/staticweb.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/crossdomain.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/s3response.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/subresource.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/acl_utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/acl_handlers.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/etree.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/bucket_logging_status.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/common.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_versions_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/delete.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/delete_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/versioning_configuration.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/copy_object_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/copy_part_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/error.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/access_control_policy.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_parts_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/location_constraint.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_bucket_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/logging.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/bucket.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/base.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/acl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/tagging.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/s3_acl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/service.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/multi_delete.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/location.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/object_lock.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/multi_upload.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/versioning.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/s3api.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/exception.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/s3token.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/s3request.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/slo.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/tempauth.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/container_sync.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/domain_remap.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/backend_ratelimit.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/html_viewer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/exceptions.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/profile_model.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/etag_quoter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/formpost.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/bulk.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/gatekeeper.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/tempurl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/account_quotas.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/name_check.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/versioned_writes/legacy.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/versioned_writes/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/versioned_writes/object_versioning.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/dlo.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/catch_errors.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/healthcheck.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/kmip_keymaster.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/kms_keymaster.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/keymaster.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/encrypter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/crypto_utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/decrypter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/cname_lookup.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/recon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/ratelimit.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/exceptions.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/linkat.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/internal_client.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/bufferedhttp.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/request_helpers.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/db.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/builder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/composite_builder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/ring.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/splice.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/http_protocol.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/manager.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/base_storage_server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/recon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/memcached.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/http.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/reaper.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/backend.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/mem_diskfile.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers copying build/lib/swift/obj/watchers/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers copying build/lib/swift/obj/watchers/dark_data.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers copying build/lib/swift/obj/ssync_receiver.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/reconstructor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/mem_server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/ssync_sender.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/updater.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/diskfile.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/expirer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/shard-info.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/drive_full_checker.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/relinker.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/container_deleter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/form_signature.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/recon_cron.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/reload.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/info.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/manage_shard_ranges.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/ringcomposer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/ringbuilder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/dispersion_report.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/recon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/ring_builder_analyzer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/base.py to base.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/account.py to account.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/container.py to container.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/info.py to info.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/obj.py to obj.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/reconciler.py to reconciler.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/updater.py to updater.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/auditor.py to auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/backend.py to backend.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync_store.py to sync_store.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/replicator.py to replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sharder.py to sharder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync.py to sync.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/registry.py to registry.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/container_sync_realms.py to container_sync_realms.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_auditor.py to db_auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/error_limiter.py to error_limiter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/daemon.py to daemon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/direct_client.py to direct_client.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/constraints.py to constraints.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/libc.py to libc.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/timestamp.py to timestamp.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/ipaddrs.py to ipaddrs.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/digest.py to digest.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/swob.py to swob.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_replicator.py to db_replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/header_key_dict.py to header_key_dict.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/wsgi.py to wsgi.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/storage_policy.py to storage_policy.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/keystoneauth.py to keystoneauth.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/xprofile.py to xprofile.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/memcache.py to memcache.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/read_only.py to read_only.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/list_endpoints.py to list_endpoints.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/listing_formats.py to listing_formats.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/proxy_logging.py to proxy_logging.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/copy.py to copy.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_quotas.py to container_quotas.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/symlink.py to symlink.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/acl.py to acl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/staticweb.py to staticweb.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crossdomain.py to crossdomain.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3response.py to s3response.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/subresource.py to subresource.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_utils.py to acl_utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_handlers.py to acl_handlers.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/etree.py to etree.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/logging.py to logging.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/bucket.py to bucket.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/base.py to base.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/acl.py to acl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/tagging.py to tagging.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/s3_acl.py to s3_acl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/service.py to service.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_delete.py to multi_delete.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/location.py to location.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/obj.py to obj.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/object_lock.py to object_lock.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_upload.py to multi_upload.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/versioning.py to versioning.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3api.py to s3api.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/exception.py to exception.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3token.py to s3token.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/utils.py to utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3request.py to s3request.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/slo.py to slo.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempauth.py to tempauth.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_sync.py to container_sync.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/domain_remap.py to domain_remap.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/backend_ratelimit.py to backend_ratelimit.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/html_viewer.py to html_viewer.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/exceptions.py to exceptions.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/profile_model.py to profile_model.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/etag_quoter.py to etag_quoter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/formpost.py to formpost.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/bulk.py to bulk.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/gatekeeper.py to gatekeeper.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempurl.py to tempurl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/account_quotas.py to account_quotas.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/name_check.py to name_check.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/legacy.py to legacy.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/object_versioning.py to object_versioning.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/dlo.py to dlo.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/catch_errors.py to catch_errors.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/healthcheck.py to healthcheck.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kmip_keymaster.py to kmip_keymaster.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kms_keymaster.py to kms_keymaster.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/keymaster.py to keymaster.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/encrypter.py to encrypter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/crypto_utils.py to crypto_utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/decrypter.py to decrypter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/cname_lookup.py to cname_lookup.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/recon.py to recon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/ratelimit.py to ratelimit.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/exceptions.py to exceptions.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/linkat.py to linkat.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/internal_client.py to internal_client.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/bufferedhttp.py to bufferedhttp.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/request_helpers.py to request_helpers.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db.py to db.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/builder.py to builder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/composite_builder.py to composite_builder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/utils.py to utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/ring.py to ring.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/splice.py to splice.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http_protocol.py to http_protocol.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/manager.py to manager.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/base_storage_server.py to base_storage_server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/recon.py to recon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/memcached.py to memcached.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http.py to http.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/reaper.py to reaper.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/auditor.py to auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/backend.py to backend.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/utils.py to utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/replicator.py to replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_diskfile.py to mem_diskfile.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/dark_data.py to dark_data.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_receiver.py to ssync_receiver.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/reconstructor.py to reconstructor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_server.py to mem_server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_sender.py to ssync_sender.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/updater.py to updater.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/auditor.py to auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/diskfile.py to diskfile.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/expirer.py to expirer.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/replicator.py to replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/shard-info.py to shard-info.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/drive_full_checker.py to drive_full_checker.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/relinker.py to relinker.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/container_deleter.py to container_deleter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/form_signature.py to form_signature.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon_cron.py to recon_cron.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/reload.py to reload.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/info.py to info.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/manage_shard_ranges.py to manage_shard_ranges.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringcomposer.py to ringcomposer.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringbuilder.py to ringbuilder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/dispersion_report.py to dispersion_report.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon.py to recon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ring_builder_analyzer.py to ring_builder_analyzer.cpython-312.pyc running install_egg_info Copying swift.egg-info to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift-2.33.0.egg-info Skipping SOURCES.txt running install_scripts creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-sharder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-reconstructor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-ring-composer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-sync -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-config -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-recon -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-reconciler-enqueue -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-orphans -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-dispersion-populate -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-ring-builder-analyzer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-reaper -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-oldies -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-form-signature -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-ring-builder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-relinker -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-drive-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-recon-cron -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-init -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-get-nodes -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-dispersion-report -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-reconciler -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-expirer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-proxy-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sharder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-reconstructor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-composer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sync to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-config to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-reconciler-enqueue to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-orphans to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-populate to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder-analyzer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-reaper to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-oldies to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-form-signature to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-relinker to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-drive-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon-cron to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-init to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-get-nodes to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-report to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-reconciler to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-expirer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-proxy-server to 755 Installing swift-container-deleter script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-drive-full-checker script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-manage-shard-ranges script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-reload script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin + pwd + python3.11 setup.py install --install-layout=deb --root /build/reproducible-path/swift-2.33.0/debian/tmp /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running install running build running build_py running egg_info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Reusing existing SOURCES.txt /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) running build_scripts running install_lib byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/base.py to base.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/account.py to account.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/container.py to container.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/info.py to info.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/obj.py to obj.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/reconciler.py to reconciler.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/updater.py to updater.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/auditor.py to auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/backend.py to backend.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync_store.py to sync_store.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/replicator.py to replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sharder.py to sharder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync.py to sync.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/registry.py to registry.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/container_sync_realms.py to container_sync_realms.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_auditor.py to db_auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/error_limiter.py to error_limiter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/daemon.py to daemon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/direct_client.py to direct_client.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/constraints.py to constraints.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/libc.py to libc.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/timestamp.py to timestamp.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/ipaddrs.py to ipaddrs.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/digest.py to digest.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/swob.py to swob.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_replicator.py to db_replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/header_key_dict.py to header_key_dict.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/wsgi.py to wsgi.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/storage_policy.py to storage_policy.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/keystoneauth.py to keystoneauth.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/xprofile.py to xprofile.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/memcache.py to memcache.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/read_only.py to read_only.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/list_endpoints.py to list_endpoints.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/listing_formats.py to listing_formats.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/proxy_logging.py to proxy_logging.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/copy.py to copy.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_quotas.py to container_quotas.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/symlink.py to symlink.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/acl.py to acl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/staticweb.py to staticweb.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crossdomain.py to crossdomain.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3response.py to s3response.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/subresource.py to subresource.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_utils.py to acl_utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_handlers.py to acl_handlers.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/etree.py to etree.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/logging.py to logging.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/bucket.py to bucket.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/base.py to base.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/acl.py to acl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/tagging.py to tagging.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/s3_acl.py to s3_acl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/service.py to service.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_delete.py to multi_delete.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/location.py to location.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/obj.py to obj.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/object_lock.py to object_lock.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_upload.py to multi_upload.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/versioning.py to versioning.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3api.py to s3api.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/exception.py to exception.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3token.py to s3token.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/utils.py to utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3request.py to s3request.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/slo.py to slo.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempauth.py to tempauth.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_sync.py to container_sync.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/domain_remap.py to domain_remap.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/backend_ratelimit.py to backend_ratelimit.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/html_viewer.py to html_viewer.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/exceptions.py to exceptions.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/profile_model.py to profile_model.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/etag_quoter.py to etag_quoter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/formpost.py to formpost.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/bulk.py to bulk.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/gatekeeper.py to gatekeeper.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempurl.py to tempurl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/account_quotas.py to account_quotas.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/name_check.py to name_check.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/legacy.py to legacy.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/object_versioning.py to object_versioning.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/dlo.py to dlo.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/catch_errors.py to catch_errors.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/healthcheck.py to healthcheck.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kmip_keymaster.py to kmip_keymaster.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kms_keymaster.py to kms_keymaster.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/keymaster.py to keymaster.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/encrypter.py to encrypter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/crypto_utils.py to crypto_utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/decrypter.py to decrypter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/cname_lookup.py to cname_lookup.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/recon.py to recon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/ratelimit.py to ratelimit.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/exceptions.py to exceptions.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/linkat.py to linkat.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/internal_client.py to internal_client.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/bufferedhttp.py to bufferedhttp.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/request_helpers.py to request_helpers.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db.py to db.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/builder.py to builder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/composite_builder.py to composite_builder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/utils.py to utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/ring.py to ring.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/splice.py to splice.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http_protocol.py to http_protocol.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/manager.py to manager.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/base_storage_server.py to base_storage_server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/recon.py to recon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/memcached.py to memcached.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http.py to http.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/reaper.py to reaper.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/auditor.py to auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/backend.py to backend.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/utils.py to utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/replicator.py to replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_diskfile.py to mem_diskfile.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/dark_data.py to dark_data.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_receiver.py to ssync_receiver.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/reconstructor.py to reconstructor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_server.py to mem_server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_sender.py to ssync_sender.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/updater.py to updater.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/auditor.py to auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/diskfile.py to diskfile.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/expirer.py to expirer.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/replicator.py to replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/shard-info.py to shard-info.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/drive_full_checker.py to drive_full_checker.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/relinker.py to relinker.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/container_deleter.py to container_deleter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/form_signature.py to form_signature.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon_cron.py to recon_cron.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/reload.py to reload.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/info.py to info.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/manage_shard_ranges.py to manage_shard_ranges.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringcomposer.py to ringcomposer.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringbuilder.py to ringbuilder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/dispersion_report.py to dispersion_report.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon.py to recon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ring_builder_analyzer.py to ring_builder_analyzer.cpython-311.pyc running install_egg_info removing '/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift-2.33.0.egg-info' (and everything under it) Copying swift.egg-info to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift-2.33.0.egg-info Skipping SOURCES.txt running install_scripts copying build/scripts-3.11/swift-container-sharder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-reconstructor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-ring-composer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-sync -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-config -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-recon -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-reconciler-enqueue -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-orphans -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-dispersion-populate -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-ring-builder-analyzer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-reaper -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-oldies -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-form-signature -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-ring-builder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-relinker -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-drive-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-recon-cron -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-init -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-get-nodes -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-dispersion-report -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-reconciler -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-expirer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-proxy-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sharder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-reconstructor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-composer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sync to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-config to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-reconciler-enqueue to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-orphans to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-populate to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder-analyzer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-reaper to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-oldies to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-form-signature to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-relinker to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-drive-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon-cron to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-init to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-get-nodes to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-report to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-reconciler to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-expirer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-proxy-server to 755 Installing swift-container-deleter script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-drive-full-checker script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-manage-shard-ranges script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-reload script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin + pwd + rm -rf /build/reproducible-path/swift-2.33.0/debian/python*/usr/lib/python*/dist-packages/*.pth + pwd + rm -rf /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python*/dist-packages/*.pth rm debian/tmp/usr/bin/swift-init pkgos-dh_auto_test --no-py2 --test-path test/unit 'test\.unit\.(?!common\.test_utils\.TestUtils\.test_get_logger_sysloghandler_plumbing|common\.middleware\.test_cname_lookup\.TestCNAMELookup.*|common\.test_db\.TestDatabaseBroker\.test_get.*|container\.test_sync\.TestContainerSync\.test_init.*|common\.test_utils\.TestPunchHoleReally\.test_punch_a_hole.*|common\.test_utils\.Test_LibcWrapper\.test_argument_plumbing.*|common\.test_utils\.TestUtils\.test_load_pkg_resource_importlib.*|common\.test_db_replicator\.TestHandoffsOnly.*|common\.ring\.test_builder\.TestGetRequiredOverload\.test_perfect_four_zone_four_replica_bad_placement|common\.test_wsgi\.TestWSGI.*|obj\.test_expirer\.TestObjectExpirer\.test_delete_actual_object|common\.test_memcached\.TestMemcached\.test_get_conns_hostname6|common\.test_memcached\.TestMemcached\.test_get_conns_v6|common\.test_memcached\.TestMemcached\.test_get_conns_v6_default|obj\.test_reconstructor\.TestWorkerReconstructor\.test_run_forever_recon_aggregation|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_with_duplicate_but_sufficient_frag_indexes|proxy\.test_mem_server\.TestProxyServer\.test_statsd_prefix|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_read_timeout|proxy\.controllers\.test_base\.TestGetOrHeadHandler\.test_disconnected_logging|obj\.test_expirer\.TestObjectExpirer\.test_failed_delete_continues_on|obj\.test_replicator\.TestObjectReplicator\.test_update)' + PKGOS_TEST_PARALLEL=yes + PKGOS_TEST_SERIAL=no + PYTHONS=disabled + PYTHON3S=disabled + TEST_PARALLEL_OPT=--parallel + TEST_SERIAL_OPT= + echo WARNING: --no-py2 is deprecated, and always on. WARNING: --no-py2 is deprecated, and always on. + shift + py3versions -vr + PYTHON3S=3.12 3.11 + [ yes = no ] + [ no = yes ] + [ 3.12 = disabled ] + echo 3.12 + cut -d. -f1 + PYMAJOR=3 + echo ===> Testing with python (python3) ===> Testing with python (python3) + pwd + [ -d /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages ] + [ -z ] + pwd + echo Implicitly adding PYTHONPATH=/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages Implicitly adding PYTHONPATH=/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages + pwd + export PYTHONPATH=/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages + [ -e .stestr.conf ] + [ -x /usr/bin/python3-stestr ] + STESTR=stestr + rm -rf .stestr + PYTHON=python3.12 stestr run+ --parallel --subunit --test-path test/unit test\.unit\.(?!common\.test_utils\.TestUtils\.test_get_logger_sysloghandler_plumbing|common\.middleware\.test_cname_lookup\.TestCNAMELookup.*|common\.test_db\.TestDatabaseBroker\.test_get.*|container\.test_sync\.TestContainerSync\.test_init.*|common\.test_utils\.TestPunchHoleReally\.test_punch_a_hole.*|common\.test_utils\.Test_LibcWrapper\.test_argument_plumbing.*|common\.test_utils\.TestUtils\.test_load_pkg_resource_importlib.*|common\.test_db_replicator\.TestHandoffsOnly.*|common\.ring\.test_builder\.TestGetRequiredOverload\.test_perfect_four_zone_four_replica_bad_placement|common\.test_wsgi\.TestWSGI.*|obj\.test_expirer\.TestObjectExpirer\.test_delete_actual_object|common\.test_memcached\.TestMemcached\.test_get_conns_hostname6|common\.test_memcached\.TestMemcached\.test_get_conns_v6|common\.test_memcached\.TestMemcached\.test_get_conns_v6_default|obj\.test_reconstructor\.TestWorkerReconstructor\.test_run_forever_recon_aggregation|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_with_duplicate_but_sufficient_frag_indexes|proxy\.test_mem_server\.TestProxyServer\.test_statsd_prefix|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_read_timeout|proxy\.controllers\.test_base\.TestGetOrHeadHandler\.test_disconnected_logging|obj\.test_expirer\.TestObjectExpirer\.test_failed_delete_continues_on|obj\.test_replicator\.TestObjectReplicator\.test_update) subunit2pyunit /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. swift: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok swift: Failed to update sync_store /tmp/tmpe4l10tcz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/container/replicator.py", line 301, in _post_replicate_hook#012 self.sync_store.update_sync_store(broker)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#012AttributeError: 'NoneType' object has no attribute 'update_sync_store' swift: Failed to update sync_store /tmp/tmpe4l10tcz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/container/replicator.py", line 301, in _post_replicate_hook#012 self.sync_store.update_sync_store(broker)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#012AttributeError: 'NoneType' object has no attribute 'update_sync_store' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test INFO: - - - [02/Jun/2024:16:22:29 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.3435 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0222 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0354 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0366 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0315 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0096 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0113 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0101 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0095 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0075 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0097 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0128 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0116 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0106 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0098 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0167 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.1798 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0198 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0257 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0141 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0347 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0336 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0045 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0187 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0038 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0220 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0120 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0162 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0101 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0126 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0171 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0202 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0140 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0115 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0114 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0201 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0192 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0027 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0139 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0014 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0099 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0126 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0133 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0093 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0123 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0031 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0195 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0070 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0135 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0145 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0105 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0155 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0150 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0051 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0156 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0152 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0119 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0099 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0152 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0117 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0037 "-" 3541997 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml ... ok 2 RLock(s) were not greened, to fix this error make sure you run eventlet.monkey_patch() before importing any other modules. test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder ... ok test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/greenpool.py", line 87, in _spawn_n_impl func(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/x_profile/profile_model.py", line 274, in dump_profile profiler.dump_stats(tmpfn) File "/usr/lib/python3.12/profile.py", line 394, in dump_stats with open(file, 'wb') as f: ^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/tmp/log/swift/profile/default.profile3541995.tmp' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/greenpool.py", line 87, in _spawn_n_impl func(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/x_profile/profile_model.py", line 274, in dump_profile profiler.dump_stats(tmpfn) File "/usr/lib/python3.12/profile.py", line 394, in dump_stats with open(file, 'wb') as f: ^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/tmp/log/swift/profile/default.profile3541995.tmp' proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpnf4yiy71/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpnf4yiy71/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpnf4yiy71/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpnf4yiy71/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0872 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0250 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0199 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0298 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx35b71827f808474db9df5-00665c9c46" "proxy-server 3542011" 0.0027 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:43327/sdb1/1/a/c" "tx35b71827f808474db9df5-00665c9c46" "container-server 3542011" 0.0026 "-" 3542011 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:53601/sda1/1/a/c" "tx35b71827f808474db9df5-00665c9c46" "container-server 3542011" 0.0024 "-" 3542011 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx35b71827f808474db9df5-00665c9c46" "proxy-server 3542011" 0.0580 "-" 3542011 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx35b71827f808474db9df5-00665c9c46" "proxy-server 3542011" 0.0291 "-" 3542011 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/30 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx35b71827f808474db9df5-00665c9c46 - 0.0736 - - 1717345350.582975626 1717345350.656569004 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txe0c9973ee2f4485bbf5b9-00665c9c46" "proxy-server 3542011" 0.0026 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:43327/sdb1/1/a1/c1" "txe0c9973ee2f4485bbf5b9-00665c9c46" "container-server 3542011" 0.0030 "-" 3542011 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:53601/sda1/1/a1/c1" "txe0c9973ee2f4485bbf5b9-00665c9c46" "container-server 3542011" 0.0024 "-" 3542011 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe0c9973ee2f4485bbf5b9-00665c9c46" "proxy-server 3542011" 0.0541 "-" 3542011 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe0c9973ee2f4485bbf5b9-00665c9c46" "proxy-server 3542011" 0.0312 "-" 3542011 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/30 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txe0c9973ee2f4485bbf5b9-00665c9c46 - 0.0711 - - 1717345350.659515619 1717345350.730663061 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdbabff3d72dd43b4b396b-00665c9c46" "proxy-server 3542011" 0.0135 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:43327/sdb1/2/a/c1" "txdbabff3d72dd43b4b396b-00665c9c46" "container-server 3542011" 0.0150 "-" 3542011 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:53601/sda1/2/a/c1" "txdbabff3d72dd43b4b396b-00665c9c46" "container-server 3542011" 0.0023 "-" 3542011 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txdbabff3d72dd43b4b396b-00665c9c46" "proxy-server 3542011" 0.0668 "-" 3542011 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txdbabff3d72dd43b4b396b-00665c9c46" "proxy-server 3542011" 0.0472 "-" 3542011 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/30 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txdbabff3d72dd43b4b396b-00665c9c46 - 0.0948 - - 1717345350.733486652 1717345350.828334570 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd7c144eba82c4678a40b5-00665c9c46" "proxy-server 3542011" 0.0134 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:53601/sda1/1/a/c2" "txd7c144eba82c4678a40b5-00665c9c46" "container-server 3542011" 0.0134 "-" 3542011 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:43327/sdb1/1/a/c2" "txd7c144eba82c4678a40b5-00665c9c46" "container-server 3542011" 0.0019 "-" 3542011 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd7c144eba82c4678a40b5-00665c9c46" "proxy-server 3542011" 0.0410 "-" 3542011 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:30 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd7c144eba82c4678a40b5-00665c9c46" "proxy-server 3542011" 0.0662 "-" 3542011 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/30 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txd7c144eba82c4678a40b5-00665c9c46 - 0.0918 - - 1717345350.831097603 1717345350.922921419 2 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT ... ok test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl ... ok test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_response test.unit.account.test_utils.TestAccountUtils.test_account_listing_response ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty ... ok test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified ... ok test.unit.cli.test_recon.TestHelpers.test_size_suffix test.unit.cli.test_recon.TestHelpers.test_size_suffix ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded ... ok test.unit.cli.test_recon.TestScout.test_scout_http_error test.unit.cli.test_recon.TestScout.test_scout_http_error ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info ... ok test.unit.cli.test_recon.TestScout.test_scout_ok test.unit.cli.test_recon.TestScout.test_scout_ok ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes ... ok test.unit.cli.test_recon.TestScout.test_scout_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_socket_timeout ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix ... ok test.unit.cli.test_recon.TestScout.test_scout_url_error test.unit.cli.test_recon.TestScout.test_scout_url_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_container test.unit.common.middleware.test_bulk.TestUntar.test_bad_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller ... ok test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required ... ok test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0004 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0189 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0173 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0123 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0114 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0116 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0118 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0144 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0099 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0134 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0055 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0189 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0186 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0033 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0009 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0192 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0170 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0124 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0142 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0142 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0112 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0105 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0163 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0135 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0131 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0162 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0094 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0095 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0027 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0027 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0180 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0084 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0123 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0095 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0094 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0099 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0091 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0113 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0197 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0088 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0261 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0156 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0129 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0160 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0147 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0082 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0119 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0100 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0101 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0132 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0123 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0408 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0127 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0107 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0110 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0151 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0035 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0033 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0259 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3541997 - test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain ... ok test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_call test.unit.common.middleware.test_bulk.TestUntar.test_extract_call ... ok test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.common.ring.test_ring.TestRing.test_creation test.unit.common.ring.test_ring.TestRing.test_creation ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes test.unit.common.ring.test_ring.TestRing.test_get_more_nodes ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.common.ring.test_ring.TestRing.test_get_nodes test.unit.common.ring.test_ring.TestRing.test_get_nodes ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.common.ring.test_ring.TestRing.test_get_part test.unit.common.ring.test_ring.TestRing.test_get_part ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args ... ok test.unit.common.ring.test_ring.TestRing.test_get_part_nodes test.unit.common.ring.test_ring.TestRing.test_get_part_nodes ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor ... ok test.unit.common.ring.test_ring.TestRing.test_has_changed test.unit.common.ring.test_ring.TestRing.test_has_changed ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions ... ok test.unit.common.ring.test_ring.TestRing.test_reload test.unit.common.ring.test_ring.TestRing.test_reload ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured ... ok test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys ... ok test.unit.common.ring.test_ring.TestRing.test_reload_without_replication test.unit.common.ring.test_ring.TestRing.test_reload_without_replication ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors ... ok test.unit.common.ring.test_ring.TestRing.test_replica_count test.unit.common.ring.test_ring.TestRing.test_replica_count ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.common.test_manager.TestManager.test_create test.unit.common.test_manager.TestManager.test_create ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default ... ok test.unit.common.test_manager.TestManager.test_default_strict test.unit.common.test_manager.TestManager.test_default_strict ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params ... ok test.unit.common.test_manager.TestManager.test_force_reload test.unit.common.test_manager.TestManager.test_force_reload ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.common.test_manager.TestManager.test_get_command test.unit.common.test_manager.TestManager.test_get_command ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param ... ok test.unit.cli.test_recon.TestRecon.test_async_check test.unit.cli.test_recon.TestRecon.test_async_check ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass ... ok test.unit.common.test_manager.TestManager.test_iter test.unit.common.test_manager.TestManager.test_iter ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains ... ok test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do ... ok test.unit.common.test_manager.TestManager.test_list_commands test.unit.common.test_manager.TestManager.test_list_commands ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_del test.unit.common.test_swob.TestHeaderEnvironProxy.test_del ... ok test.unit.cli.test_recon.TestRecon.test_drive_audit_check test.unit.cli.test_recon.TestRecon.test_drive_audit_check ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.common.test_manager.TestManager.test_no_daemon test.unit.common.test_manager.TestManager.test_no_daemon ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys ... ok test.unit.cli.test_recon.TestRecon.test_gen_stats test.unit.cli.test_recon.TestRecon.test_gen_stats ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT ... ok test.unit.common.test_manager.TestManager.test_no_wait test.unit.common.test_manager.TestManager.test_no_wait ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys ... ok test.unit.cli.test_recon.TestRecon.test_get_error_ringnames test.unit.cli.test_recon.TestRecon.test_get_error_ringnames ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you ... ok test.unit.common.test_manager.TestManager.test_once test.unit.common.test_manager.TestManager.test_once ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold ... ok test.unit.cli.test_recon.TestRecon.test_get_hosts test.unit.cli.test_recon.TestRecon.test_get_hosts ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy ... ok test.unit.cli.test_recon.TestRecon.test_get_ring_names test.unit.cli.test_recon.TestRecon.test_get_ring_names ... ok test.unit.common.test_manager.TestManager.test_reload test.unit.common.test_manager.TestManager.test_reload ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided ... ok test.unit.cli.test_recon.TestRecon.test_get_ringmd5 test.unit.cli.test_recon.TestRecon.test_get_ringmd5 ... ok test.unit.common.test_manager.TestManager.test_restart test.unit.common.test_manager.TestManager.test_restart ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root ... ok test.unit.common.test_utils.TestDistributeEvenly.test_just_one test.unit.common.test_utils.TestDistributeEvenly.test_just_one ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object ... ok test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items ... ok test.unit.common.test_manager.TestManager.test_run_command test.unit.common.test_manager.TestManager.test_run_command ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta ... ok test.unit.common.test_utils.TestDistributeEvenly.test_uneven test.unit.common.test_utils.TestDistributeEvenly.test_uneven ... ok test.unit.common.test_manager.TestManager.test_shutdown test.unit.common.test_manager.TestManager.test_shutdown ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta ... ok test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type ... ok test.unit.common.test_manager.TestManager.test_start test.unit.common.test_manager.TestManager.test_start ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu ... ok test.unit.common.test_manager.TestManager.test_status test.unit.common.test_manager.TestManager.test_status ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter ... ok test.unit.cli.test_recon.TestRecon.test_ptime test.unit.cli.test_recon.TestRecon.test_ptime ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings ... ok test.unit.common.test_manager.TestManager.test_stop test.unit.common.test_manager.TestManager.test_stop ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata ... ok test.unit.cli.test_recon.TestRecon.test_quarantine_check test.unit.cli.test_recon.TestRecon.test_quarantine_check ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT ... ok test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout ... ok test.unit.cli.test_recon.TestRecon.test_umount_check test.unit.cli.test_recon.TestRecon.test_umount_check ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception ... ok test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT ... ok test.unit.cli.test_recon.TestReconCommands.test_disk_usage test.unit.cli.test_recon.TestReconCommands.test_disk_usage ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta ... ok test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers ... ok test.unit.common.test_swob.TestResponse.test_507 test.unit.common.test_swob.TestResponse.test_507 ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta ... ok test.unit.common.test_swob.TestResponse.test_app_iter test.unit.common.test_swob.TestResponse.test_app_iter ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta ... ok test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk ... ok test.unit.cli.test_recon.TestReconCommands.test_load_check test.unit.cli.test_recon.TestReconCommands.test_load_check ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta ... ok test.unit.common.test_swob.TestResponse.test_call_preserves_closeability test.unit.common.test_swob.TestResponse.test_call_preserves_closeability ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header ... ok test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_info test.unit.account.test_backend.TestCommonAccountBroker.test_get_info ... ok test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check ... ok test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row ... ok test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy ... ok test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist ... ok test.unit.common.test_swob.TestResponse.test_charset test.unit.common.test_swob.TestResponse.test_charset ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range ... ok test.unit.cli.test_recon.TestReconCommands.test_replication_check test.unit.cli.test_recon.TestReconCommands.test_replication_check ... ok test.unit.common.test_swob.TestResponse.test_charset_content_type test.unit.common.test_swob.TestResponse.test_charset_content_type ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get ... ok test.unit.cli.test_recon.TestReconCommands.test_server_type_check test.unit.cli.test_recon.TestReconCommands.test_server_type_check ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.cli.test_recon.TestReconCommands.test_sharding_check test.unit.cli.test_recon.TestReconCommands.test_sharding_check ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check test.unit.cli.test_recon.TestReconCommands.test_time_check ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking ... ok test.unit.common.test_swob.TestResponse.test_content_type test.unit.common.test_swob.TestResponse.test_content_type ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata ... ok test.unit.common.test_swob.TestResponse.test_empty_body test.unit.common.test_swob.TestResponse.test_empty_body ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req ... ok test.unit.common.test_swob.TestResponse.test_etag test.unit.common.test_swob.TestResponse.test_etag ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception ... ok test.unit.common.test_swob.TestResponse.test_host_url_default test.unit.common.test_swob.TestResponse.test_host_url_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check test.unit.cli.test_recon.TestReconCommands.test_version_check ... ok test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_https test.unit.common.test_swob.TestResponse.test_host_url_host_https ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check_differs test.unit.cli.test_recon.TestReconCommands.test_version_check_differs ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_override test.unit.common.test_swob.TestResponse.test_host_url_host_override ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_port_override test.unit.common.test_swob.TestResponse.test_host_url_host_port_override ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag ... ok test.unit.common.test_swob.TestResponse.test_host_url_https test.unit.common.test_swob.TestResponse.test_host_url_https ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_good test.unit.cli.test_reload.TestValidateManagerPid.test_good ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch ... ok test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_python test.unit.cli.test_reload.TestValidateManagerPid.test_non_python ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite test.unit.common.test_swob.TestResponse.test_location_no_rewrite ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_server test.unit.cli.test_reload.TestValidateManagerPid.test_non_server ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite test.unit.common.test_swob.TestResponse.test_location_rewrite ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete ... ok test.unit.common.test_swob.TestResponse.test_multi_range_body test.unit.common.test_swob.TestResponse.test_multi_range_body ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_open_error test.unit.cli.test_reload.TestValidateManagerPid.test_open_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_worker test.unit.cli.test_reload.TestValidateManagerPid.test_worker ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.test_swob.TestResponse.test_multi_response_iter test.unit.common.test_swob.TestResponse.test_multi_response_iter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format ... ok test.unit.common.test_swob.TestResponse.test_properties test.unit.common.test_swob.TestResponse.test_properties ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format ... ok test.unit.common.test_swob.TestResponse.test_range_body test.unit.common.test_swob.TestResponse.test_range_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable ... ok test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format ... ok test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body ... ok test.unit.common.test_swob.TestResponse.test_unicode_body test.unit.common.test_swob.TestResponse.test_unicode_body ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.test_utils.TestFileLikeIter.test_close test.unit.common.test_utils.TestFileLikeIter.test_close ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker ... ok test.unit.common.test_utils.TestFileLikeIter.test_get_hub test.unit.common.test_utils.TestFileLikeIter.test_get_hub ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart ... ok test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough ... ok test.unit.common.test_utils.TestFileLikeIter.test_next test.unit.common.test_utils.TestFileLikeIter.test_next ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring test.unit.cli.test_ringbuilder.TestCommands.test_create_ring ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting ... ok test.unit.common.test_utils.TestFileLikeIter.test_read test.unit.common.test_utils.TestFileLikeIter.test_read ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size test.unit.common.test_utils.TestFileLikeIter.test_read_with_size ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default test.unit.cli.test_ringbuilder.TestCommands.test_default ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline test.unit.common.test_utils.TestFileLikeIter.test_readline ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output test.unit.cli.test_ringbuilder.TestCommands.test_default_output ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline2 test.unit.common.test_utils.TestFileLikeIter.test_readline2 ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline3 test.unit.common.test_utils.TestFileLikeIter.test_readline3 ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines test.unit.common.test_utils.TestFileLikeIter.test_readlines ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response ... ok test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge ... ok test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor ... ok test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation ... ok test.unit.cli.test_ringbuilder.TestCommands.test_find_parts test.unit.cli.test_ringbuilder.TestCommands.test_find_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap ... ok test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty ... ok test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception ... ok test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates test.unit.common.test_utils.TestStatsdLogging.test_sample_rates ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching ... ok test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats ... ok test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail ... ok test.unit.common.test_utils.TestStatsdLogging.test_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_timing_stats ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate ... ok test.unit.account.test_reaper.TestReaper.test_basic_reap_account test.unit.account.test_reaper.TestReaper.test_basic_reap_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power ... ok test.unit.account.test_reaper.TestReaper.test_creation test.unit.account.test_reaper.TestReaper.test_creation ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type ... ok test.unit.account.test_reaper.TestReaper.test_reap_account test.unit.account.test_reaper.TestReaper.test_reap_account ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_no_container test.unit.account.test_reaper.TestReaper.test_reap_account_no_container ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.account.test_reaper.TestReaper.test_reap_container test.unit.account.test_reaper.TestReaper.test_reap_container ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id test.unit.common.ring.test_builder.TestRingBuilder.test_id ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed ... ok test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_init test.unit.common.ring.test_builder.TestRingBuilder.test_init ... ok test.unit.account.test_replicator.TestReplicatorSync.test_local_ids test.unit.account.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback ... ok test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance ... ok test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync test.unit.account.test_replicator.TestReplicatorSync.test_sync ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_load test.unit.common.ring.test_builder.TestRingBuilder.test_load ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error ... ok test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response ... ok test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration ... ok test.unit.account.test_backend.TestAccountBroker.test_chexor test.unit.account.test_backend.TestAccountBroker.test_chexor ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full ... ok test.unit.account.test_backend.TestAccountBroker.test_creation test.unit.account.test_backend.TestAccountBroker.test_creation ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status ... ok test.unit.account.test_backend.TestAccountBroker.test_delete_container test.unit.account.test_backend.TestAccountBroker.test_delete_container ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_delete_db_status test.unit.account.test_backend.TestAccountBroker.test_delete_db_status ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours ... ok test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty ... ok test.unit.account.test_backend.TestAccountBroker.test_empty test.unit.account.test_backend.TestAccountBroker.test_empty ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial ... ok test.unit.account.test_backend.TestAccountBroker.test_exception test.unit.account.test_backend.TestAccountBroker.test_exception ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_syncs test.unit.cli.test_info.TestCliInfo.test_print_db_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info test.unit.account.test_backend.TestAccountBroker.test_get_info ... ok test.unit.cli.test_info.TestCliInfo.test_print_info test.unit.cli.test_info.TestCliInfo.test_print_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_overload test.unit.common.ring.test_builder.TestRingBuilder.test_overload ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence ... ok test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially ... ok test.unit.account.test_backend.TestAccountBroker.test_initialize_fail test.unit.account.test_backend.TestAccountBroker.test_initialize_fail ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher ... ok test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail ... ok test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher ... ok test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items test.unit.account.test_backend.TestAccountBroker.test_merge_items ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_newid test.unit.account.test_backend.TestAccountBroker.test_newid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv ... ok test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade ... ok test.unit.account.test_backend.TestAccountBroker.test_put_container test.unit.account.test_backend.TestAccountBroker.test_put_container ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher ... ok test.unit.account.test_backend.TestAccountBroker.test_reclaim test.unit.account.test_backend.TestAccountBroker.test_reclaim ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success ... ok test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.account.test_server.TestAccountController.test_DELETE_empty test.unit.account.test_server.TestAccountController.test_DELETE_empty ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration ... ok test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_empty test.unit.account.test_server.TestAccountController.test_DELETE_not_empty ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_found test.unit.account.test_server.TestAccountController.test_DELETE_not_found ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_DELETE_now_empty test.unit.account.test_server.TestAccountController.test_DELETE_now_empty ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 ... ok test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save test.unit.common.ring.test_builder.TestRingBuilder.test_save ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save_load test.unit.common.ring.test_builder.TestRingBuilder.test_save_load ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_GET_accept_json test.unit.account.test_server.TestAccountController.test_GET_accept_json ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate test.unit.common.ring.test_builder.TestRingBuilder.test_validate ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_xml test.unit.account.test_server.TestAccountController.test_GET_accept_xml ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_json test.unit.account.test_server.TestAccountController.test_GET_empty_account_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match ... ok test.unit.common.test_db.TestDictFactory.test_normal_case test.unit.common.test_db.TestDictFactory.test_normal_case ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params ... ok test.unit.account.test_server.TestAccountController.test_GET_invalid_accept test.unit.account.test_server.TestAccountController.test_GET_invalid_accept ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata ... ok test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch ... ok test INFO: Disabling account_sdb test INFO: Disabling container_sdb test INFO: Disabling object_sdb test INFO: Enabling account_sdb test INFO: Enabling container_sdb test INFO: Enabling object_sdb Something useful Also useful Boring, but not secret test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards ... ok test.unit.account.test_server.TestAccountController.test_GET_log_requests_false test.unit.account.test_server.TestAccountController.test_GET_log_requests_false ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default test.unit.common.test_memcached.TestLoadMemcache.test_conf_default ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline ... ok test.unit.common.middleware.test_bulk.TestConfig.test_defaults test.unit.common.middleware.test_bulk.TestConfig.test_defaults ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn ... ok test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json ... ok test.unit.cli.test_info.TestPrintObj.test_missing_etag test.unit.cli.test_info.TestPrintObj.test_missing_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json ... ok test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink ... ok test.unit.cli.test_info.TestPrintObj.test_print_obj_valid test.unit.cli.test_info.TestPrintObj.test_print_obj_valid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync ... ok test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 199, in HEAD return self._handle_acl(app, 'DELETE') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 141, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst test.unit.common.test_utils.TestEventletRateLimiter.test_burst ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_init test.unit.common.test_utils.TestEventletRateLimiter.test_init ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole ... ok test.unit.common.test_utils.TestShardRangeList.test_filter test.unit.common.test_utils.TestShardRangeList.test_filter ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate ... ok test.unit.common.test_utils.TestShardRangeList.test_find_lower test.unit.common.test_utils.TestShardRangeList.test_find_lower ... ok test.unit.common.test_utils.TestShardRangeList.test_includes test.unit.common.test_utils.TestShardRangeList.test_includes ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestShardRangeList.test_init test.unit.common.test_utils.TestShardRangeList.test_init ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json ... ok test.unit.common.test_utils.TestShardRangeList.test_init_with_list test.unit.common.test_utils.TestShardRangeList.test_init_with_list ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl ... ok test.unit.common.test_utils.TestShardRangeList.test_pop test.unit.common.test_utils.TestShardRangeList.test_pop ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved ... ok test.unit.common.test_utils.TestShardRangeList.test_slice test.unit.common.test_utils.TestShardRangeList.test_slice ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container ... ok test.unit.common.test_utils.TestShardRangeList.test_states test.unit.common.test_utils.TestShardRangeList.test_states ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestShardRangeList.test_timestamps test.unit.common.test_utils.TestShardRangeList.test_timestamps ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version ... ok test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag ... ok test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set ... ok test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole ... ok test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 ... ok test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info test.unit.common.middleware.test_container_sync.TestContainerSync.test_info ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key ... ok test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.ResellerConfReader.test_defaults test.unit.common.test_utils.ResellerConfReader.test_defaults ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs ... ok test.unit.common.test_utils.ResellerConfReader.test_same_as_default test.unit.common.test_utils.ResellerConfReader.test_same_as_default ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.test_utils.ResellerConfReader.test_stray_comma test.unit.common.test_utils.ResellerConfReader.test_stray_comma ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_json test.unit.common.middleware.test_xprofile.TestStats2.test_to_json ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking test.unit.common.test_utils.TestPipeMutex.test_blocking ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestPipeMutex.test_nonblocking test.unit.common.test_utils.TestPipeMutex.test_nonblocking ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestPipeMutex.test_recursive test.unit.common.test_utils.TestPipeMutex.test_recursive ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version ... ok test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestPipeMutex.test_too_many_releases test.unit.common.test_utils.TestPipeMutex.test_too_many_releases ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range ... ok test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address ... ok test.unit.container.test_backend.TestContainerBroker.test_chexor test.unit.container.test_backend.TestContainerBroker.test_chexor ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix ... ok test.unit.container.test_backend.TestContainerBroker.test_create_broker test.unit.container.test_backend.TestContainerBroker.test_create_broker ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_creation test.unit.container.test_backend.TestContainerBroker.test_creation ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_db_file test.unit.container.test_backend.TestContainerBroker.test_db_file ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response ... ok test INFO: Begin account audit "once" mode test INFO: Account audit "once" mode completed: 0.00s test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/22/31 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0013 - - 1717345351.333141804 1717345351.334458351 0 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/22/31 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0014 - - 1717345351.336137295 1717345351.337550402 1 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_object test.unit.container.test_backend.TestContainerBroker.test_delete_object ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.container.test_backend.TestContainerBroker.test_empty test.unit.container.test_backend.TestContainerBroker.test_empty ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_enable_sharding test.unit.container.test_backend.TestContainerBroker.test_enable_sharding ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location ... ok test.unit.container.test_backend.TestContainerBroker.test_exception test.unit.container.test_backend.TestContainerBroker.test_exception ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.common.ring.test_ring.TestRingData.test_attrs test.unit.common.ring.test_ring.TestRingData.test_attrs ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline ... ok test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires ... ok test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change ... ok test.unit.common.ring.test_ring.TestRingData.test_load_closes_file test.unit.common.ring.test_ring.TestRingData.test_load_closes_file ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval ... ok test.unit.common.ring.test_ring.TestRingData.test_permissions test.unit.common.ring.test_ring.TestRingData.test_permissions ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel ... ok test.unit.common.ring.test_ring.TestRingData.test_replica_count test.unit.common.ring.test_ring.TestRingData.test_replica_count ... ok test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization ... ok test.unit.common.test_daemon.TestRunDaemon.test_fork_workers test.unit.common.test_daemon.TestRunDaemon.test_fork_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestRunDaemon.test_forked_worker test.unit.common.test_daemon.TestRunDaemon.test_forked_worker ... ok test.unit.common.test_daemon.TestRunDaemon.test_restart_workers test.unit.common.test_daemon.TestRunDaemon.test_restart_workers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestRunDaemon.test_run test.unit.common.test_daemon.TestRunDaemon.test_run ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestRunDaemon.test_run_daemon test.unit.common.test_daemon.TestRunDaemon.test_run_daemon ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid ... ok test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value ... ok test.unit.cli.test_relinker.TestRelinker.test_conf_file test.unit.cli.test_relinker.TestRelinker.test_conf_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices ... ok test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file ... ok test.unit.common.test_daemon.TestRunDaemon.test_signal test.unit.common.test_daemon.TestRunDaemon.test_signal ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.cli.test_relinker.TestRelinker.test_rehashing test.unit.cli.test_relinker.TestRelinker.test_rehashing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup ... ok test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_container_sync_realms.TestUtils.test_empty test.unit.common.test_container_sync_realms.TestUtils.test_empty ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container ... ok test.unit.common.test_db.TestHelperFunctions.test_zero_like test.unit.common.test_db.TestHelperFunctions.test_zero_like ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices ... ok test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_container_sync_realms.TestUtils.test_get_sig test.unit.common.test_container_sync_realms.TestUtils.test_get_sig ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.common.test_container_sync_realms.TestUtils.test_one_realm test.unit.common.test_container_sync_realms.TestUtils.test_one_realm ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_put test.unit.obj.test_server.TestObjectServer.test_expect_on_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.common.test_container_sync_realms.TestUtils.test_os_error test.unit.common.test_container_sync_realms.TestUtils.test_os_error ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestDaemon.test_create test.unit.common.test_daemon.TestDaemon.test_create ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestDaemon.test_stubs test.unit.common.test_daemon.TestDaemon.test_stubs ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_container_exists test.unit.common.test_internal_client.TestInternalClient.test_container_exists ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_account test.unit.common.test_internal_client.TestInternalClient.test_create_account ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container test.unit.common.test_internal_client.TestInternalClient.test_create_container ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_file test.unit.cli.test_relinker.TestRelinker.test_relink_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_account test.unit.common.test_internal_client.TestInternalClient.test_delete_account ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_delete_container test.unit.common.test_internal_client.TestInternalClient.test_delete_container ... ok test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container ... ok test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_object test.unit.common.test_internal_client.TestInternalClient.test_delete_object ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 ... ok test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_info test.unit.container.test_backend.TestCommonContainerBroker.test_get_info ... ok test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info test.unit.common.test_internal_client.TestInternalClient.test_get_account_info ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata ... ok test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs ... ok test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get ... ok test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.obj.test_server.TestObjectServer.test_not_found test.unit.obj.test_server.TestObjectServer.test_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_metadata ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_object test.unit.common.test_internal_client.TestInternalClient.test_get_object ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp ... ok test.unit.common.test_internal_client.TestInternalClient.test_init test.unit.common.test_internal_client.TestInternalClient.test_init ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at ... ok test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_containers test.unit.common.test_internal_client.TestInternalClient.test_iter_containers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items test.unit.common.test_internal_client.TestInternalClient.test_iter_items ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers ... ok test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_iter_objects test.unit.common.test_internal_client.TestInternalClient.test_iter_objects ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_load_from_config test.unit.common.test_internal_client.TestInternalClient.test_load_from_config ... ok test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf ... ok test.unit.cli.test_relinker.TestRelinker.test_state_file test.unit.cli.test_relinker.TestRelinker.test_state_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_workers_children test.unit.cli.test_relinker.TestRelinker.test_workers_children ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_workers_parent test.unit.cli.test_relinker.TestRelinker.test_workers_parent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestInternalClient.test_make_path test.unit.common.test_internal_client.TestInternalClient.test_make_path ... ok test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object test.unit.common.test_internal_client.TestInternalClient.test_upload_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing ... ok test.unit.common.test_linkat.TestLinkat.test_available test.unit.common.test_linkat.TestLinkat.test_available ... ok test.unit.common.test_linkat.TestLinkat.test_errno test.unit.common.test_linkat.TestLinkat.test_errno ... ok test.unit.common.test_linkat.TestLinkat.test_flags test.unit.common.test_linkat.TestLinkat.test_flags ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_success test.unit.common.test_linkat.TestLinkat.test_linkat_success ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable test.unit.common.test_linkat.TestLinkat.test_unavailable ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc ... ok test.unit.common.test_swob.TestStatusMap.test_status_map test.unit.common.test_swob.TestStatusMap.test_status_map ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific ... ok test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0142 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0148 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0100 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0116 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0034 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0124 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0153 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0115 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0139 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0034 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0190 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0150 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0124 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0195 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0143 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0218 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0029 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0184 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0172 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0129 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0138 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0220 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0004 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0005 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0002 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0285 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0038 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0203 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0258 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0099 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0102 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0155 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0113 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0133 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0157 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0032 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0101 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0069 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0301 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0121 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0095 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0129 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0190 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0079 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0077 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0188 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0089 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0064 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0161 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0031 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3541997 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Use tempauth middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Use of the auth_admin_prefix, auth_host, auth_port, auth_protocol, identity_uri, admin_token, admin_user, admin_password, and admin_tenant_name configuration options was deprecated in the Mitaka release in favor of an auth_plugin and its related options. This class may be removed in a future release. Configuring admin URI using auth fragments was deprecated in the Kilo release, and will be removed in the Newton release, use 'identity_uri' instead. Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint test DEBUG: Calling S3Api Middleware Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json ... ok Ring Builder file does not exist: /tmp/tmprc0_w4xm/tmpqfzs0e99 Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.2:6200R127.0.0.2:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.1:6200R127.0.0.1:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:6200R[3::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Ring Builder file does not exist: /tmp/tmp9d0sr57a/tmplg1fwgas Partition power increase in progress. You need to finish the increase first before adding devices. Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sda9_"" with 3.14159265359 weight got id 4 Device 4 already uses 127.0.0.1:6200/sda9. The on-disk ring builder is unchanged. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. /tmp/tmpigi1fc4z/tmpsux0t01j, build version 4, id 037f5000970f4c87919753b2ab113680 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmpigi1fc4z/tmpsux0t01j.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output_id_assigned Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 Device d4r1z2-127.0.0.5:6004R127.0.0.5:6004/sda5_"" with 100.0 weight got id 4 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d0r2z1-127.0.0.6:6005R127.0.0.6:6005/sdb6_"" with 100.0 weight got id 0 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_sorted_output Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. /build/reproducible-path/swift-2.33.0/test/unit/cli/test_ipv6_output Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 32 1 13 1 Partition Matches 6 1 7 1 8 1 19 1 20 1 29 1 34 1 36 1 41 1 46 1 47 1 48 1 50 1 51 1 57 1 59 1 2 1 3 1 9 1 11 1 18 1 21 1 22 1 30 1 31 1 35 1 40 1 49 1 52 1 58 1 60 1 62 1 1 1 4 1 5 1 12 1 16 1 17 1 24 1 26 1 27 1 32 1 33 1 39 1 42 1 43 1 45 1 56 1 Partition Matches 1 1 3 1 5 1 7 1 13 1 14 1 21 1 22 1 25 1 34 1 38 1 45 1 51 1 52 1 58 1 60 1 4 1 6 1 9 1 11 1 15 1 17 1 20 1 33 1 35 1 47 1 49 1 50 1 54 1 56 1 57 1 59 1 2 1 10 1 12 1 16 1 24 1 27 1 30 1 32 1 36 1 37 1 39 1 40 1 41 1 48 1 53 1 61 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 37 1 46 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 5 1 52 1 Specified builder file "/tmp/tmpmsrvb76n/tmpd64_iw4k" is not rebalanced yet. Please rebalance first. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error ... ok =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 5 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== [2024-06-02 16:22:30] Checking auditor stats =============================================================================== test ERROR: Error decrypting container listing: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: get_keys(): unknown key id: unknown_key test ERROR: Error decrypting container listing: unknown_key test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid ... ok test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj ... ok test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 370, in reap_container headers, objects = direct_get_container( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 171, in fake_direct_get_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.10.10.1:6202/sda3 test-reaper ERROR: ERROR: invalid storage policy index: 2 test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail ... ok test.unit.account.test_reaper.TestReaper.test_reap_delay test.unit.account.test_reaper.TestReaper.test_reap_delay ... ok test.unit.account.test_reaper.TestReaper.test_reap_device test.unit.account.test_reaper.TestReaper.test_reap_device ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts ... ok test.unit.account.test_reaper.TestReaper.test_reap_object test.unit.account.test_reaper.TestReaper.test_reap_object ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_fail test.unit.account.test_reaper.TestReaper.test_reap_object_fail ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_object_timeout ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set ... ok test.unit.account.test_reaper.TestReaper.test_reset_stats test.unit.account.test_reaper.TestReaper.test_reset_stats ... ok test.unit.account.test_reaper.TestReaper.test_run_forever test.unit.account.test_reaper.TestReaper.test_run_forever ... ok test.unit.account.test_reaper.TestReaper.test_run_once test.unit.account.test_reaper.TestReaper.test_run_once ... ok test.unit.cli.test_reload.TestMain.test_check_failed test.unit.cli.test_reload.TestMain.test_check_failed ... ok test.unit.cli.test_reload.TestMain.test_good test.unit.cli.test_reload.TestMain.test_good ... ok test.unit.cli.test_reload.TestMain.test_needs_pid test.unit.cli.test_reload.TestMain.test_needs_pid ... ok test.unit.cli.test_reload.TestMain.test_timeout test.unit.cli.test_reload.TestMain.test_timeout ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing ... ok test ERROR: Bad key for 'container': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: get_keys(): from callback: boom: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_crypto_utils.py", line 95, in callback raise Exception('boom') Exception: boom test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Missing key for 'container': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'container' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Did not get a keys dict: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ TypeError: list indices must be integers or slices, not str test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Cipher must be AES_CTR_256 test DEBUG: No decryption is necessary because of override test DEBUG: No decryption is necessary because of override test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_decrypter.py", line 902, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: Error decrypting header X-Object-Sysmeta-Crypto-Etag: Missing crypto meta in value fgBd78GL6kACmfmuAxTz9B7eJCFTfjxotcr7trw/5Qo= test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting object: Bad crypto meta: Missing 'cipher' test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Key must be length 32 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting object: Missing 'body_key' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr ... ok test.unit.common.test_manager.TestManagerModule.test_command_wrapper test.unit.common.test_manager.TestManagerModule.test_command_wrapper ... ok test.unit.common.test_manager.TestManagerModule.test_exc test.unit.common.test_manager.TestManagerModule.test_exc ... ok test.unit.common.test_manager.TestManagerModule.test_format_server_name test.unit.common.test_manager.TestManagerModule.test_format_server_name ... ok test.unit.common.test_manager.TestManagerModule.test_safe_kill test.unit.common.test_manager.TestManagerModule.test_safe_kill ... ok test.unit.common.test_manager.TestManagerModule.test_servers test.unit.common.test_manager.TestManagerModule.test_servers ... ok test.unit.common.test_manager.TestManagerModule.test_setup_env test.unit.common.test_manager.TestManagerModule.test_setup_env ... ok test.unit.common.test_manager.TestManagerModule.test_verify_server test.unit.common.test_manager.TestManagerModule.test_verify_server ... ok test.unit.common.test_manager.TestManagerModule.test_watch_server_pids test.unit.common.test_manager.TestManagerModule.test_watch_server_pids ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star test.unit.common.test_swob.TestConditionalIfMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_closes test.unit.common.test_utils.TestReiterate.test_reiterate_closes ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple ... ok test.unit.common.test_utils.TestShardName.test test.unit.common.test_utils.TestShardName.test ... ok test.unit.common.test_utils.TestShardName.test_bad_create test.unit.common.test_utils.TestShardName.test_bad_create ... ok test.unit.common.test_utils.TestShardName.test_bad_parse test.unit.common.test_utils.TestShardName.test_bad_parse ... ok test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names ... ok test.unit.common.test_utils.TestShardName.test_root_has_hyphens test.unit.common.test_utils.TestShardName.test_root_has_hyphens ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning ... ok test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0170 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0133 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0148 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0188 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0166 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0374 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0251 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0239 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0002 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0164 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0126 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0136 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0307 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0099 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0174 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0142 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0100 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0205 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0103 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0145 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0152 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0125 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0154 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0140 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0089 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0106 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0141 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0192 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0110 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0109 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0099 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0108 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0102 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0120 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0028 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0104 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0104 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0125 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0108 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0158 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0132 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0111 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0115 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0092 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0152 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0143 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0030 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0171 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0140 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0118 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0186 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0100 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0382 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0031 "-" 3542001 - test.unit.account.test_server.TestAccountController.test_GET_log_requests_true test.unit.account.test_server.TestAccountController.test_GET_log_requests_true ... ok test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_json test.unit.account.test_server.TestAccountController.test_GET_not_found_json ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_plain test.unit.account.test_server.TestAccountController.test_GET_not_found_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_xml test.unit.account.test_server.TestAccountController.test_GET_not_found_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_over_limit test.unit.account.test_server.TestAccountController.test_GET_over_limit ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order ... ok test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_encrypter.py", line 913, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: User tenant_id with role(s) rdr1,rdr2 has project reader authorizing test ERROR: Problem finding shard upper in '/tmp/tmpp33zeemk/test_container.db': : LockTimeout (Nones) test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_brokers test.unit.container.test_backend.TestContainerBroker.test_get_brokers ... ok test WARNING: More than one part in a single-part response? test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test NOTICE: Closing unnecessary sock for port 6006 (child pid 88) test NOTICE: Closing unnecessary sock for port 6006 (child pid 89) test NOTICE: Closing unnecessary sock for port 6006 (child pid 90) test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test NOTICE: Started child 0 (PID 94) for port 6009 test NOTICE: Started child 1 (PID 95) for port 6009 test NOTICE: Started child 2 (PID 96) for port 6009 test NOTICE: Started child 1 (PID 97) for port 6009 test NOTICE: Child 2 (PID 3542027, port 6007) exiting normally test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test DEBUG: set nice priority to 1 test DEBUG: set ionice class 1 priority 0 WARNING: Unable to modify I/O scheduling class and priority of process. Keeping unchanged! Check logs for more info. test ERROR: Unable to modify ionice priority: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/libc.py", line 326, in _ioprio_set io_class = IO_CLASS_ENUM[io_class] ~~~~~~~~~~~~~^^^^^^^^^^ KeyError: 'class_foo' test DEBUG: set ionice class 2 priority 4 test DEBUG: set nice priority to -15 test DEBUG: set ionice class 3 priority 6 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: Setting object count to 1032 (txn: tx40dd0a1844fc4e3f8a2c3-00665c9c4f) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: Setting object count to 48 (txn: tx0092a9dac8cb4542bd127-00665c9c4f) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txa15004c19d174263bc02c-00665c9c4f) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.container.test_backend.TestContainerBroker.test_get_db_state test.unit.container.test_backend.TestContainerBroker.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info test.unit.container.test_backend.TestContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBroker.test_get_items_since test.unit.container.test_backend.TestContainerBroker.test_get_items_since ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces test.unit.container.test_backend.TestContainerBroker.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects test.unit.container.test_backend.TestContainerBroker.test_get_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBroker.test_get_policy_info test.unit.container.test_backend.TestContainerBroker.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_replication_info test.unit.container.test_backend.TestContainerBroker.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_report_info test.unit.container.test_backend.TestContainerBroker.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBroker.test_is_deleted test.unit.container.test_backend.TestContainerBroker.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_last_modified_time test.unit.container.test_backend.TestContainerBroker.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items test.unit.container.test_backend.TestContainerBroker.test_merge_items ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBroker.test_newid test.unit.container.test_backend.TestContainerBroker.test_newid ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBroker.test_path test.unit.container.test_backend.TestContainerBroker.test_path ... ok test.unit.container.test_backend.TestContainerBroker.test_pending_file_name test.unit.container.test_backend.TestContainerBroker.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object test.unit.container.test_backend.TestContainerBroker.test_put_object ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim test.unit.container.test_backend.TestContainerBroker.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted ... ok test INFO: Begin account audit "once" mode test ERROR: Audit Failed for /tmp/tmp1jz4mj8x/sda/accounts/0/0/0/test.db: The total container_count for the account a (3) does not match the sum of container_count across policies (2) test INFO: Account audit "once" mode completed: 0.00s test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test NOTICE: Child 3542023 exiting normally test NOTICE: Started child 88 from parent 3542023 test NOTICE: Started child 89 from parent 3542023 test ERROR: Removing dead child 88 from parent 3542023 test NOTICE: Started child 90 from parent 3542023 test NOTICE: Started child 88 from parent 3542023 test NOTICE: Started child 89 from parent 3542023 test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (10.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3542023] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3542023] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/2 pid=pid-1] Starting object reconstructor in daemon mode. test INFO: [worker 1/2 pid=pid-1] Starting object reconstruction pass. test INFO: [worker 1/2 pid=pid-1] Object reconstruction complete. (0.10 minutes) test DEBUG: [worker 1/2 pid=pid-1] reconstruction sleeping for 30.0 seconds. test INFO: [worker 2/4 pid=20641] Running object reconstructor in script mode. test DEBUG: [worker 2/4 pid=20641] debug message test INFO: [worker 2/4 pid=20641] info message test WARNING: [worker 2/4 pid=20641] warning message test ERROR: [worker 2/4 pid=20641] error message test INFO: [worker 2/4 pid=20641] Object reconstruction complete (once). (0.00 minutes) test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'AUTH_a.example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9900>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9900>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/dontchange', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.aexample.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59faec0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59faec0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/ä½\xa0好', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/FOO_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers ... ok test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{}' test DEBUG: Error on keystone reply: 201 b'{}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'[]' test DEBUG: Error on keystone reply: 201 b'[]' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'null' test DEBUG: Error on keystone reply: 201 b'null' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'"foo"' test DEBUG: Error on keystone reply: 201 b'"foo"' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'1' test DEBUG: Error on keystone reply: 201 b'1' - rejecting request test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_chrome test.unit.common.middleware.test_formpost.TestFormPost.test_chrome ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_explorer test.unit.common.middleware.test_formpost.TestFormPost.test_explorer ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_firefox test.unit.common.middleware.test_formpost.TestFormPost.test_firefox ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_container test.unit.common.middleware.test_formpost.TestFormPost.test_no_container ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_safari test.unit.common.middleware.test_formpost.TestFormPost.test_safari ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree ... ok test.unit.common.ring.test_utils.TestUtils.test_dispersion_report test.unit.common.ring.test_utils.TestUtils.test_dispersion_report ... ok test.unit.common.ring.test_utils.TestUtils.test_is_local_device test.unit.common.ring.test_utils.TestUtils.test_is_local_device ... ok test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname ... ok test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_args test.unit.common.ring.test_utils.TestUtils.test_parse_args ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_value test.unit.common.ring.test_utils.TestUtils.test_parse_search_value ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_replication_defaults test.unit.common.ring.test_utils.TestUtils.test_replication_defaults ... ok test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args test.unit.common.ring.test_utils.TestUtils.test_validate_args ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact ... ok test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting ... ok test.unit.common.test_db.TestDatabaseBroker.test_commit_puts test.unit.common.test_db.TestDatabaseBroker.test_commit_puts ... ok test.unit.common.test_db.TestDatabaseBroker.test_delete_db test.unit.common.test_db.TestDatabaseBroker.test_delete_db ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate ... ok test.unit.common.test_db.TestDatabaseBroker.test_initialize test.unit.common.test_db.TestDatabaseBroker.test_initialize ... ok test.unit.common.test_db.TestDatabaseBroker.test_lock test.unit.common.test_db.TestDatabaseBroker.test_lock ... ok test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init ... ok test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata test.unit.common.test_db.TestDatabaseBroker.test_metadata ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_newid test.unit.common.test_db.TestDatabaseBroker.test_newid ... ok test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00062 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2z8sa4z4/sda/accounts/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00664 seconds (150.61733/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00069 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00052 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00042 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sda/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'eef74fee-7084-445d-b65c-b1fe608c2ab2-sda', '1717345354.56103', '1717345354.55618', '0', '') test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpeg81eq3x/sda/accounts/0/cca/7e85f39cfbd92b74b0b21a19f4188cca/7e85f39cfbd92b74b0b21a19f4188cca.db REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'e530b678-d0ad-4842-8775-d1dd4ed46bbb-sda', '1717345354.58736', '1717345354.58123', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('e530b678-d0ad-4842-8775-d1dd4ed46bbb-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'e530b678-d0ad-4842-8775-d1dd4ed46bbb-sda', '1717345354.58736', '1717345354.58123', '0', '') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('e530b678-d0ad-4842-8775-d1dd4ed46bbb-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06797 seconds (14.71198/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (1, 'ed3d6e3c0ee3532a00e7b1f38675d99b', '59f3dc76-d4b9-4c39-b6d6-90d5c143ef6b-sda', '1717345354.68344', '1717345354.67823', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, rsync_then_merge, ('59f3dc76-d4b9-4c39-b6d6-90d5c143ef6b-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (11, '41e7e9529d1486e478c2ce9732215c40', '77436533-79c4-45ab-bf27-0863aaeca13c-sda', '1717345354.76607', '1717345354.75603', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_items, ([{'ROWID': 1, 'name': '/a/c_0', 'put_timestamp': '1717345354.7873', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c_1', 'put_timestamp': '1717345354.78796', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c_2', 'put_timestamp': '1717345354.78888', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c_3', 'put_timestamp': '1717345354.78914', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c_4', 'put_timestamp': '1717345354.79038', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c_5', 'put_timestamp': '1717345354.79081', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c_6', 'put_timestamp': '1717345354.79133', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c_7', 'put_timestamp': '1717345354.79166', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c_8', 'put_timestamp': '1717345354.79202', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c_9', 'put_timestamp': '1717345354.79238', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c_missing', 'put_timestamp': '1717345354.79268', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}], '77436533-79c4-45ab-bf27-0863aaeca13c-sda') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_syncs, ([],) test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile123 Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile3542003 Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile123 Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile3542003 Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile456 30 function calls in 0.002 seconds Ordered by: call count ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :0(setprofile) 0 0.000 0.000 profile:0(profiler) Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile123 Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile3542003 test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_increment test.unit.common.test_error_limiter.TestErrorLimiter.test_increment ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_pending test.unit.common.test_utils.TestGreenAsyncPile.test_pending ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first ... ok test.unit.common.test_utils.TestLRUCache.test_maxsize test.unit.common.test_utils.TestLRUCache.test_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_maxtime test.unit.common.test_utils.TestLRUCache.test_maxtime ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxsize test.unit.common.test_utils.TestLRUCache.test_set_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxtime test.unit.common.test_utils.TestLRUCache.test_set_maxtime ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str test.unit.common.test_wsgi.TestPipelineWrapper.test_str ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_conf test.unit.container.test_sharder.TestContainerSharderConf.test_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_default test.unit.container.test_sharder.TestContainerSharderConf.test_default ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate test.unit.container.test_sharder.TestContainerSharderConf.test_validate ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_close_error test.unit.obj.test_diskfile.TestECDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit test.unit.obj.test_diskfile.TestECDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_delete test.unit.obj.test_diskfile.TestECDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments test.unit.obj.test_diskfile.TestECDiskFile.test_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestTpoolSize.test_default_config test.unit.obj.test_server.TestTpoolSize.test_default_config ... ok test.unit.obj.test_server.TestTpoolSize.test_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_sync test.unit.obj.test_ssync.TestSsyncReplication.test_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted ... ok test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing ... ok test.unit.obj.test_ssync_sender.TestSender.test_call test.unit.obj.test_ssync_sender.TestSender.test_call ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect test.unit.obj.test_ssync_sender.TestSender.test_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect test.unit.obj.test_ssync_sender.TestSender.test_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all test.unit.obj.test_ssync_sender.TestSender.test_readline_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete test.unit.obj.test_ssync_sender.TestSender.test_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post test.unit.obj.test_ssync_sender.TestSender.test_send_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put test.unit.obj.test_ssync_sender.TestSender.test_send_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges ... ok test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_post test.unit.obj.test_ssync_sender.TestSender.test_updates_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put test.unit.obj.test_ssync_sender.TestSender.test_updates_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBroker.test_remove_objects test.unit.container.test_backend.TestContainerBroker.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/22/31 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0021 - - 1717345351.458704233 1717345351.460774660 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/22/31 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0019 - - 1717345351.461981773 1717345351.463889837 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Invalid current //REALM/CLUSTER (foo) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Worker mock-pid-0 exited my-daemon DEBUG: Worker mock-pid-1 exited my-daemon DEBUG: Worker mock-pid-2 exited my-daemon NOTICE: Finished 3542011 my-daemon DEBUG: Forked worker 3542011 finished my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Daemon needs to change options, aborting workers my-daemon DEBUG: Cleaned up worker mock-pid-0 my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-3 with {} my-daemon DEBUG: Spawned worker mock-pid-4 with {} my-daemon DEBUG: Spawned worker mock-pid-5 with {} my-daemon DEBUG: Cleaned up worker mock-pid-3 my-daemon DEBUG: Cleaned up worker mock-pid-4 my-daemon DEBUG: Cleaned up worker mock-pid-5 my-daemon NOTICE: SIGTERM received () my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Worker mock-pid exited my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon NOTICE: Worker mock-pid died my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00052 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpbh7k2h8z/sda/fake/0/59e/33379ecb053aa5c9e356c68997cbb59e/33379ecb053aa5c9e356c68997cbb59e.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00395 seconds (253.10469/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00117 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00046 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00035 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-ic DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) fake-swift INFO: - - 02/Jun/2024/16/22/32 GET /cont/obj HTTP/1.0 200 - some_agent - - 19 - - - 0.0007 - - 1717345352.520273447 1717345352.520930052 - fake-swift INFO: - - 02/Jun/2024/16/22/32 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0003 - - 1717345352.522623062 1717345352.522890329 - fake-swift INFO: - - 02/Jun/2024/16/22/32 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717345352.523818016 1717345352.524015188 - fake-swift INFO: - - 02/Jun/2024/16/22/32 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717345352.524631977 1717345352.524823189 - test DEBUG: Before request test DEBUG: Inside of request (txn: foo) test DEBUG: After request fake-swift INFO: - - 02/Jun/2024/16/22/32 DELETE /container HTTP/1.0 409 - some_agent - - 19 - - - 0.0002 - - 1717345352.535404205 1717345352.535571575 - proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpao6dmh8h/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpao6dmh8h/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header ... ok test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file ... ok test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'true' test DEBUG: Error on keystone reply: 201 b'true' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, rejecting request with error: 400 Bad Request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, rejecting request with error: 401 Unauthorized test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. Quarantined /tmp/tmp4tlsk1qx/dev/dbs/par/pre/db to /tmp/tmp4tlsk1qx/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmp4tlsk1qx/dev/dbs/par/pre/db to /tmp/tmp4tlsk1qx/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmp4tlsk1qx/dev/dbs/par/pre/db to /tmp/tmp4tlsk1qx/dev/quarantined/tests/db due to corrupted database Quarantined /tmp/tmp4tlsk1qx/dev/dbs/par/pre/db to /tmp/tmp4tlsk1qx/dev/quarantined/tests/db due to disk error while accessing database test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors ... ok test.unit.common.test_db.TestDatabaseBroker.test_put_record test.unit.common.test_db.TestDatabaseBroker.test_put_record ... ok Quarantined /tmp/tmp3g8g4_sv/dev/dbs/par/pre/db to /tmp/tmp3g8g4_sv/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info ... ok test.unit.common.test_db.TestDatabaseBroker.test_skip_commits test.unit.common.test_db.TestDatabaseBroker.test_skip_commits ... ok Quarantined /tmp/tmpifg2cpgw/dev/dbs/par/pre/db to /tmp/tmpifg2cpgw/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata ... ok test.unit.common.test_utils.TestContextPool.test_close test.unit.common.test_utils.TestContextPool.test_close ... ok test.unit.common.test_utils.TestContextPool.test_context_manager test.unit.common.test_utils.TestContextPool.test_context_manager ... ok test.unit.common.test_utils.TestUTC.test_tzname test.unit.common.test_utils.TestUTC.test_tzname ... ok test1 STDOUT: test2 STDERR: test4 test5 STDERR: test6 test8 STDOUT: a#012b#012c STDOUT: d test.unit.common.test_utils.TestUtils.test_LoggerFileObject test.unit.common.test_utils.TestUtils.test_LoggerFileObject ... ok test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0158 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0254 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0072 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0072 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0149 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0155 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0201 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0092 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.0189 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0169 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0159 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0112 "-" 3541997 - test DEBUG: - - - [02/Jun/2024:16:22:33 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3541997 - test DEBUG: - - - [02/Jun/2024:16:22:33 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3541997 - test DEBUG: - - - [02/Jun/2024:16:22:33 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0012 "-" 3541997 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:22:33 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0092 "-" 3541997 - test DEBUG: - - - [02/Jun/2024:16:22:33 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0006 "-" 3541997 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0196 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0158 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0252 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0014 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0176 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0124 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0173 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0110 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0019 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0252 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0015 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0126 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0149 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0020 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0022 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0026 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0222 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0013 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3541997 3 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0108 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0101 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3541997 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test ERROR: Problem finding shard upper in '/tmp/tmpp33zeemk/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmps808mvcy/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmps808mvcy/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBroker.test_set_db_states test.unit.container.test_backend.TestContainerBroker.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBroker.test_sync_merging test.unit.container.test_backend.TestContainerBroker.test_sync_merging ... ok test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency test.unit.container.test_reconciler.TestReconciler.test_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency_config test.unit.container.test_reconciler.TestReconciler.test_concurrency_config ... ok test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi ... ok test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.0.0.0:1000/sda: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 490, in reap_object direct_delete_object( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 160, in fake_direct_delete_object raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.0.0.0:1000/sda test-reaper ERROR: Timeout Exception with 10.0.0.1:1001/sdb test-reaper ERROR: Timeout Exception with 10.0.0.2:1002/sdc test-reaper ERROR: Timeout Exception with 10.0.0.3:1003/sdd test-reaper DEBUG: Begin devices pass: thisdeviceisbad test-reaper ERROR: Exception in top-level account reaper loop: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 135, in run_once for device in os.listdir(self.devices): ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'thisdeviceisbad' test-reaper INFO: Devices pass completed: 0.00s Reloaded swift-proxy-server test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - deferring rejection downstream test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, deferring rejection based on error: 400 Bad Request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, deferring rejection based on error: 401 Unauthorized test WARNING: The following digest algorithms are configured but not supported: md5, not-a-valid-digest test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test INFO: - - 02/Jun/2024/16/22/32 GET / HTTP/1.0 200 - - - - 8 - - First:%201%0ASecond:%202 0.0001 - - 1717345352.655390978 1717345352.655496120 - test INFO: - - 02/Jun/2024/16/22/32 GET / HTTP/1.0 500 - - - - - - - - 0.0000 - - 1717345352.657183647 1717345352.657195568 - test INFO: - - 02/Jun/2024/16/22/32 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345352.658419609 1717345352.658487797 - test INFO: GET /v1/a/c 205 205 test INFO: GET /v1/a/c 209 205 test INFO: GET /v1/a/c 210 205 test INFO: 1.2.3.4 1.2.3.4 02/Jun/2024/16/22/32 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345352.667151690 1717345352.667202473 - test INFO: - - 02/Jun/2024/16/22/32 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717345352.668342352 1717345352.668370247 - test INFO: - - 02/Jun/2024/16/22/32 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717345352.669467211 1717345352.669481039 - test INFO: - - 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345353.251719713 1717345353.251804829 - test INFO: - - 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345353.253459692 1717345353.253520012 - test INFO: - - 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - - - 15 - - - 0.0010 - - 1717345353.253427744 1717345353.254398823 - test INFO: - - 26/Apr/1970/17/46/41 GET / HTTP/1.0 200 - - - - 8 - - - 1.0000 - - 10000000.000000000 10000001.000000000 - test INFO: GET /v1/a/c 503 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c True 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c parsing%20ok 503 test INFO: GET /v1/a/c 500 500 test INFO: GET /v1/a/c 500 568 test INFO: GET /v1/a/c 499 503 test INFO: GET /v1/a/c 499 503 test INFO: GET / 500 200 test INFO: 2001:db8:85a3:8d3:1319:8a2e:370:7348 2001:db8:85a3:8d3:1319:8a2e:370:7348 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345353.298432589 1717345353.298503399 - test INFO: - - 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345353.301909685 1717345353.301985979 - test INFO: - - 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - b05bf940-0464-4c... - 8 - - - 0.0001 - - 1717345353.303313971 1717345353.303386450 - test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart ... ok I like ham STDERR: --- Logging error --- STDERR: Traceback (most recent call last): STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 845, in emit raise socket.error(errno.ENOTCONN, "This is an ex-syslog") STDERR: OSError: [Errno 107] This is an ex-syslog STDERR: Call stack: STDERR: File "", line 198, in _run_module_as_main STDERR: File "", line 88, in _run_code STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 88, in main() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 82, in main program.TestProgram( STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 197, in __init__ self.runTests() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 247, in runTests self.result = testRunner.run(self.test) STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 51, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/case.py", line 690, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/case.py", line 634, in run self._callTestMethod(testMethod) STDERR: File "/usr/lib/python3.12/unittest/case.py", line 589, in _callTestMethod if method() is not None: STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 877, in test_LoggerFileObject_recursion logger.info("I like ham") STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1539, in info self._log(INFO, msg, args, **kwargs) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1684, in _log self.handle(record) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1700, in handle self.callHandlers(record) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1762, in callHandlers hdlr.handle(record) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1028, in handle self.emit(record) STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 847, in emit self.handleError(record) STDERR: Message: 'I like ham' Arguments: () test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion ... ok test.unit.common.test_utils.TestUtils.test_NullLogger test.unit.common.test_utils.TestUtils.test_NullLogger ... ok test.unit.common.test_utils.TestUtils.test_TRUE_VALUES test.unit.common.test_utils.TestUtils.test_TRUE_VALUES ... ok test.unit.common.test_utils.TestUtils.test_backwards test.unit.common.test_utils.TestUtils.test_backwards ... ok test.unit.common.test_utils.TestUtils.test_cache_from_env test.unit.common.test_utils.TestUtils.test_cache_from_env ... ok test.unit.common.test_utils.TestUtils.test_cap_length test.unit.common.test_utils.TestUtils.test_cap_length ... ok test.unit.common.test_utils.TestUtils.test_capture_stdio test.unit.common.test_utils.TestUtils.test_capture_stdio ... ok test.unit.common.test_utils.TestUtils.test_clean_content_type test.unit.common.test_utils.TestUtils.test_clean_content_type ... ok test.unit.common.test_utils.TestUtils.test_clean_logger_exception test.unit.common.test_utils.TestUtils.test_clean_logger_exception ... ok test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene ... ok test.unit.common.test_utils.TestUtils.test_config_auto_int_value test.unit.common.test_utils.TestUtils.test_config_auto_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_fallocate_value test.unit.common.test_utils.TestUtils.test_config_fallocate_value ... ok test.unit.common.test_utils.TestUtils.test_config_float_value test.unit.common.test_utils.TestUtils.test_config_float_value ... ok test.unit.common.test_utils.TestUtils.test_config_percent_value test.unit.common.test_utils.TestUtils.test_config_percent_value ... ok test.unit.common.test_utils.TestUtils.test_config_positive_int_value test.unit.common.test_utils.TestUtils.test_config_positive_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_request_node_count_value test.unit.common.test_utils.TestUtils.test_config_request_node_count_value ... ok test.unit.common.test_utils.TestUtils.test_config_true_value test.unit.common.test_utils.TestUtils.test_config_true_value ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close test.unit.common.test_utils.TestUtils.test_drain_and_close ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges test.unit.common.test_utils.TestUtils.test_drop_privileges ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache test.unit.common.test_utils.TestUtils.test_dump_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner ... ok test.unit.common.test_utils.TestUtils.test_extract_swift_bytes test.unit.common.test_utils.TestUtils.test_extract_swift_bytes ... ok test.unit.common.test_utils.TestUtils.test_find_namespace test.unit.common.test_utils.TestUtils.test_find_namespace ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_large_body test.unit.common.test_utils.TestUtils.test_friendly_close_large_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_small_body test.unit.common.test_utils.TestUtils.test_friendly_close_small_body ... ok test.unit.common.test_utils.TestUtils.test_fsync_dir test.unit.common.test_utils.TestUtils.test_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_generate_trans_id test.unit.common.test_utils.TestUtils.test_generate_trans_id ... ok test.unit.common.test_utils.TestUtils.test_get_db_files test.unit.common.test_utils.TestUtils.test_get_db_files ... ok test.unit.common.test_utils.TestUtils.test_get_log_line test.unit.common.test_utils.TestUtils.test_get_log_line ... ok test.unit.common.test_utils.TestUtils.test_get_logger test.unit.common.test_utils.TestUtils.test_get_logger ... ok test.unit.common.test_utils.TestUtils.test_get_logger_console test.unit.common.test_utils.TestUtils.test_get_logger_console ... ok test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route ... ok test.unit.common.test_utils.TestUtils.test_get_partition_for_hash test.unit.common.test_utils.TestUtils.test_get_partition_for_hash ... ok test.unit.common.test_utils.TestUtils.test_get_partition_from_path test.unit.common.test_utils.TestUtils.test_get_partition_from_path ... ok test.unit.common.test_utils.TestUtils.test_get_policy_index test.unit.common.test_utils.TestUtils.test_get_policy_index ... ok test.unit.common.test_utils.TestUtils.test_get_redirect_data test.unit.common.test_utils.TestUtils.test_get_redirect_data ... ok test.unit.common.test_utils.TestUtils.test_get_trans_id_time test.unit.common.test_utils.TestUtils.test_get_trans_id_time ... ok test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str ... ok test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string ... ok test.unit.common.test_utils.TestUtils.test_hash_path test.unit.common.test_utils.TestUtils.test_hash_path ... ok test.unit.common.test_utils.TestUtils.test_human_readable test.unit.common.test_utils.TestUtils.test_human_readable ... ok test.unit.common.test_utils.TestUtils.test_is_file_older test.unit.common.test_utils.TestUtils.test_is_file_older ... ok test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_error test.unit.common.test_utils.TestUtils.test_ismount_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_root test.unit.common.test_utils.TestUtils.test_ismount_path_is_root ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_dev test.unit.common.test_utils.TestUtils.test_ismount_successes_dev ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_ino test.unit.common.test_utils.TestUtils.test_ismount_successes_ino ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists ... ok test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/22/31 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0017 - - 1717345351.503759861 1717345351.505482197 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/22/31 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0014 - - 1717345351.506831169 1717345351.508270502 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/service.py", line 35, in GET resp = req.get_response(self.app, query={'format': 'json'}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523d6c0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523d6c0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523d6c0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl ... ok test.unit.common.test_db_auditor.TestAuditor.test_database_auditor test.unit.common.test_db_auditor.TestAuditor.test_database_auditor ... ok test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass ... ok test ERROR: ERROR: swift.cache could not be found in env! test ERROR: ERROR: swift.cache could not be found in env! test WARNING: Unable to perform fsync() on directory /tmp/tmppjnxguv9: Bad file descriptor Unable to locate some_not_real_function in libc. Leaving as a no-op. test.unit.common.test_utils.TestUtils.test_load_libc_function test.unit.common.test_utils.TestUtils.test_load_libc_function ... ok test.unit.common.test_utils.TestUtils.test_load_pkg_resource test.unit.common.test_utils.TestUtils.test_load_pkg_resource ... skipped 'pkg_resources loading is only available on python 3.7 and earlier' test.unit.common.test_utils.TestUtils.test_load_recon_cache test.unit.common.test_utils.TestUtils.test_load_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_lock_file test.unit.common.test_utils.TestUtils.test_lock_file ... ok test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink ... ok test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail ... ok test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open ... ok test.unit.common.test_utils.TestUtils.test_lock_path test.unit.common.test_utils.TestUtils.test_lock_path ... ok test.unit.common.test_utils.TestUtils.test_lock_path_class test.unit.common.test_utils.TestUtils.test_lock_path_class ... ok test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit ... ok test.unit.common.test_utils.TestUtils.test_lock_path_name test.unit.common.test_utils.TestUtils.test_lock_path_name ... ok test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps ... ok test.unit.common.test_utils.TestUtils.test_log_string_formatter test.unit.common.test_utils.TestUtils.test_log_string_formatter ... ok test.unit.common.test_utils.TestUtils.test_majority_size test.unit.common.test_utils.TestUtils.test_majority_size ... ok test.unit.common.test_utils.TestUtils.test_make_db_file_path test.unit.common.test_utils.TestUtils.test_make_db_file_path ... ok test.unit.common.test_utils.TestUtils.test_makedirs_count test.unit.common.test_utils.TestUtils.test_makedirs_count ... ok test.unit.common.test_utils.TestUtils.test_md5_with_data test.unit.common.test_utils.TestUtils.test_md5_with_data ... ok test.unit.common.test_utils.TestUtils.test_md5_without_data test.unit.common.test_utils.TestUtils.test_md5_without_data ... ok test.unit.common.test_utils.TestUtils.test_mkdirs test.unit.common.test_utils.TestUtils.test_mkdirs ... ok test.unit.common.test_utils.TestUtils.test_monkey_patch test.unit.common.test_utils.TestUtils.test_monkey_patch ... ok test.unit.common.test_utils.TestUtils.test_nested_prefixlogger test.unit.common.test_utils.TestUtils.test_nested_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_node_to_string test.unit.common.test_utils.TestUtils.test_node_to_string ... ok test.unit.common.test_utils.TestUtils.test_non_negative_float test.unit.common.test_utils.TestUtils.test_non_negative_float ... ok test.unit.common.test_utils.TestUtils.test_non_negative_int test.unit.common.test_utils.TestUtils.test_non_negative_int ... ok test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision ... ok test.unit.common.test_utils.TestUtils.test_normalize_timestamp test.unit.common.test_utils.TestUtils.test_normalize_timestamp ... ok test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_content_type test.unit.common.test_utils.TestUtils.test_parse_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_db_filename test.unit.common.test_utils.TestUtils.test_parse_db_filename ... ok test.unit.common.test_utils.TestUtils.test_parse_options test.unit.common.test_utils.TestUtils.test_parse_options ... ok test.unit.common.test_utils.TestUtils.test_parse_options_errors test.unit.common.test_utils.TestUtils.test_parse_options_errors ... ok test.unit.common.test_utils.TestUtils.test_parse_override_options test.unit.common.test_utils.TestUtils.test_parse_override_options ... ok test.unit.common.test_utils.TestUtils.test_prefixlogger test.unit.common.test_utils.TestUtils.test_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_quorum_size test.unit.common.test_utils.TestUtils.test_quorum_size ... ok test.unit.common.test_utils.TestUtils.test_quote_bytes test.unit.common.test_utils.TestUtils.test_quote_bytes ... ok test.unit.common.test_utils.TestUtils.test_quote_unicode test.unit.common.test_utils.TestUtils.test_quote_unicode ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep ... ok test.unit.common.test_utils.TestUtils.test_readconf test.unit.common.test_utils.TestUtils.test_readconf ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir test.unit.common.test_utils.TestUtils.test_readconf_dir ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files ... ok test.unit.common.test_utils.TestUtils.test_readconf_raw test.unit.common.test_utils.TestUtils.test_readconf_raw ... ok test.unit.common.test_utils.TestUtils.test_remove_directory test.unit.common.test_utils.TestUtils.test_remove_directory ... ok test.unit.common.test_utils.TestUtils.test_remove_file test.unit.common.test_utils.TestUtils.test_remove_file ... ok test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false ... ok test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_replace_partition_in_path test.unit.common.test_utils.TestUtils.test_replace_partition_in_path ... ok test.unit.common.test_utils.TestUtils.test_round_robin_iter test.unit.common.test_utils.TestUtils.test_round_robin_iter ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip ... ok test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation ... ok test.unit.common.test_utils.TestUtils.test_safe_json_loads test.unit.common.test_utils.TestUtils.test_safe_json_loads ... ok test.unit.common.test_utils.TestUtils.test_search_tree test.unit.common.test_utils.TestUtils.test_search_tree ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match ... ok test.unit.common.test_utils.TestUtils.test_select_node_ip test.unit.common.test_utils.TestUtils.test_select_node_ip ... ok test.unit.common.test_utils.TestUtils.test_split_path test.unit.common.test_utils.TestUtils.test_split_path ... ok test.unit.common.test_utils.TestUtils.test_storage_directory test.unit.common.test_utils.TestUtils.test_storage_directory ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer test.unit.common.test_utils.TestUtils.test_str_anonymizer ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness ... ok test.unit.common.test_utils.TestUtils.test_str_format_time test.unit.common.test_utils.TestUtils.test_str_format_time ... ok test.unit.common.test_utils.TestUtils.test_streq_const_time test.unit.common.test_utils.TestUtils.test_streq_const_time ... ok test.unit.common.test_utils.TestUtils.test_strict_b64decode test.unit.common.test_utils.TestUtils.test_strict_b64decode ... ok test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter test.unit.common.test_utils.TestUtils.test_swift_log_formatter ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length ... ok test.unit.common.test_utils.TestUtils.test_systemd_notify test.unit.common.test_utils.TestUtils.test_systemd_notify ... ok test.unit.common.test_utils.TestUtils.test_validate_device_partition test.unit.common.test_utils.TestUtils.test_validate_device_partition ... ok test.unit.common.test_utils.TestUtils.test_validate_hash_conf test.unit.common.test_utils.TestUtils.test_validate_hash_conf ... ok test.unit.common.test_utils.TestUtils.test_validate_sync_to test.unit.common.test_utils.TestUtils.test_validate_sync_to ... ok test.unit.common.test_utils.TestUtils.test_write_file test.unit.common.test_utils.TestUtils.test_write_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission ... ok test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: Setting object count to 1064 (txn: txa15004c19d174263bc02c-00665c9c4f) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txaf56584757804795b478b-00665c9c4f) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txbb04fd9db9d0476ab0d80-00665c9c4f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Setting object count to 1065 (txn: txfe9fcb89673b49ecaccbc-00665c9c52) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Setting object count to 1065 (txn: tx44ee21afc8704f78a2677-00665c9c55) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx2b05eee1da144d609ae14-00665c9c58) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: tx2b05eee1da144d609ae14-00665c9c58) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error ... ok test ERROR: Invalid swift_bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4161, in override_bytes_from_content_type listing_dict['bytes'] = int(swift_bytes) ^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: 'hey' Unlinked retiring db '/tmp/tmpmfakfldm/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx4faa46b8ea6040ce89224-00665c9c4f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txced16614c8cb4b37ab74e-00665c9c4f) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpm47b4382/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpm47b4382/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpm47b4382/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpm47b4382/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0119 "-" 3542023 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0095 "-" 3542023 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0143 "-" 3542023 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0120 "-" 3542023 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6dbfc8e6348543e4b3647-00665c9c4f" "proxy-server 3542023" 0.0019 "-" 3542023 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:41617/sda1/1/a/c" "tx6dbfc8e6348543e4b3647-00665c9c4f" "container-server 3542023" 0.0016 "-" 3542023 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:50077/sdb1/1/a/c" "tx6dbfc8e6348543e4b3647-00665c9c4f" "container-server 3542023" 0.0015 "-" 3542023 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6dbfc8e6348543e4b3647-00665c9c4f" "proxy-server 3542023" 0.0410 "-" 3542023 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6dbfc8e6348543e4b3647-00665c9c4f" "proxy-server 3542023" 0.0240 "-" 3542023 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx6dbfc8e6348543e4b3647-00665c9c4f - 0.0527 - - 1717345359.515233517 1717345359.567904711 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx586dc08cc15d4a69a1b56-00665c9c4f" "proxy-server 3542023" 0.0016 "-" 3542023 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:50077/sdb1/1/a1/c1" "tx586dc08cc15d4a69a1b56-00665c9c4f" "container-server 3542023" 0.0017 "-" 3542023 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:41617/sda1/1/a1/c1" "tx586dc08cc15d4a69a1b56-00665c9c4f" "container-server 3542023" 0.0027 "-" 3542023 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx586dc08cc15d4a69a1b56-00665c9c4f" "proxy-server 3542023" 0.0229 "-" 3542023 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx586dc08cc15d4a69a1b56-00665c9c4f" "proxy-server 3542023" 0.0383 "-" 3542023 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx586dc08cc15d4a69a1b56-00665c9c4f - 0.0482 - - 1717345359.569923878 1717345359.618169069 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9724f641f8794ff38a3ac-00665c9c4f" "proxy-server 3542023" 0.0168 "-" 3542023 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:41617/sda1/2/a/c1" "tx9724f641f8794ff38a3ac-00665c9c4f" "container-server 3542023" 0.0041 "-" 3542023 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:50077/sdb1/2/a/c1" "tx9724f641f8794ff38a3ac-00665c9c4f" "container-server 3542023" 0.0183 "-" 3542023 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx9724f641f8794ff38a3ac-00665c9c4f" "proxy-server 3542023" 0.0414 "-" 3542023 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx9724f641f8794ff38a3ac-00665c9c4f" "proxy-server 3542023" 0.0581 "-" 3542023 1 test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' Unlinked retiring db '/tmp/tmp0r_jdm48/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container ... ok test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' Unlinked retiring db '/tmp/tmpcsanngw1/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. user quit No .MockServer object at 0x7f9a5cedbf80> running .MockServerFactory.MockServer object at 0x7f9a5c4ec6b0> (1) appears to have stopped No .MockServerFactory.MockServer object at 0x7f9a5c5c2f30> running No .MockServerFactory.MockServer object at 0x7f9a5c4ec6b0> running Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f9a5c5c2f30> to die; giving up Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f9a5c4ec560> to die; killing Signal .MockServerFactory.MockServer object at 0x7f9a5c4ec560> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f9a5c4ec620> to die; killing Signal .MockServerFactory.MockServer object at 0x7f9a5c4ec620> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f9a5c4ec560> to die; killing Signal .MockServerFactory.MockServer object at 0x7f9a5c4ec560> pid: None signal: 9 test WARNING: Error sending UDP message to ('some.host.com', 8125): [Errno 1] test errno 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0111 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0164 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0020 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0227 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0235 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0092 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0164 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0146 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0325 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0162 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0227 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0116 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0169 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0301 "-" 3542007 2 test ERROR: ERROR Account update failed with 127.0.0.1:55747/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0136 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0163 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0174 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0010 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0226 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0241 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:32 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0023 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0328 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0280 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0210 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0115 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0120 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0092 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0123 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0102 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:32 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0053 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0060 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0107 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0038 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter ... ok test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0153 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0137 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0124 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0133 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0148 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0144 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0193 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0107 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0130 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0025 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0025 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0030 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0183 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0324 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0110 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0125 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0114 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0116 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0122 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0091 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0119 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0161 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0140 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0098 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0154 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0133 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0077 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0029 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0192 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0036 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0108 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0103 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0105 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0103 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0106 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0101 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0230 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0135 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0094 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0127 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0134 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0144 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0032 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0030 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0151 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0092 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0105 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0102 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0121 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0102 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0088 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0156 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0104 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0089 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0117 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0132 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0100 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0034 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0142 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3542001 - test.unit.account.test_server.TestAccountController.test_GET_with_containers_json test.unit.account.test_server.TestAccountController.test_GET_with_containers_json ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.account.test_server.TestAccountController.test_HEAD_empty_account test.unit.account.test_server.TestAccountController.test_HEAD_empty_account ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_HEAD_not_found test.unit.account.test_server.TestAccountController.test_HEAD_not_found ... ok test.unit.account.test_server.TestAccountController.test_HEAD_with_containers test.unit.account.test_server.TestAccountController.test_HEAD_with_containers ... ok test.unit.account.test_server.TestAccountController.test_OPTIONS test.unit.account.test_server.TestAccountController.test_OPTIONS ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata ... ok test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found ... ok test.unit.account.test_server.TestAccountController.test_POST_insufficient_space test.unit.account.test_server.TestAccountController.test_POST_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_POST_invalid_partition test.unit.account.test_server.TestAccountController.test_POST_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float ... ok test.unit.account.test_server.TestAccountController.test_PUT test.unit.account.test_server.TestAccountController.test_PUT ... ok proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx9724f641f8794ff38a3ac-00665c9c4f - 0.0850 - - 1717345359.620586157 1717345359.705572128 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9363af1b961548288932d-00665c9c4f" "proxy-server 3542023" 0.0081 "-" 3542023 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:41617/sda1/1/a/c2" "tx9363af1b961548288932d-00665c9c4f" "container-server 3542023" 0.0018 "-" 3542023 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:50077/sdb1/1/a/c2" "tx9363af1b961548288932d-00665c9c4f" "container-server 3542023" 0.0122 "-" 3542023 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx9363af1b961548288932d-00665c9c4f" "proxy-server 3542023" 0.0475 "-" 3542023 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx9363af1b961548288932d-00665c9c4f" "proxy-server 3542023" 0.0342 "-" 3542023 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx9363af1b961548288932d-00665c9c4f - 0.0664 - - 1717345359.707821369 1717345359.774263859 2 test ERROR: Problem finding shard upper in '/tmp/tmpv5_mfflx/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpv5_mfflx/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpo5kct1we/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpo5kct1we/containers/part/suffix/hash/container.db'] Unlinked retiring db '/tmp/tmptzph_s55/containers/part/suffix/hash/hash.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default ... ok Unlinked retiring db '/tmp/tmp7i6oe6qr/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted ... ok test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' fake INFO: - - 02/Jun/2024/16/22/39 GET /v1/a/c/o%3Ftemp_url_sig%3Ddeadbeefdeadbeef...%26temp_url_expires%3D1717346359 HTTP/1.0 404 - - - - - - - - 0.0002 - - 1717345359.763694048 1717345359.763919115 - test ERROR: Error in '/tmp/tmpobls4u9i/container-sync-realms.conf' with mtime_check_interval: could not convert string to float: 'invalid' test ERROR: Could not load '/tmp/tmpqmw4hasl/container-sync-realms.conf': File contains no section headers. file: '/tmp/tmpqmw4hasl/container-sync-realms.conf', line: 1 'invalid' test DEBUG: Could not load 'ba1a43fe0a9640e6adb2d5cf1bc48413': [Errno 2] No such file or directory: 'ba1a43fe0a9640e6adb2d5cf1bc48413' test ERROR: Could not load '/tmp/tmp068hjmul/container-sync-realms.conf': [Errno 13] Permission denied: '/tmp/tmp068hjmul/container-sync-realms.conf' test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00041 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmplkxzpzor/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00721 seconds (138.68485/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test DEBUG: Successfully deleted db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test DEBUG: Not deleting db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (2/3 success) test DEBUG: Not deleting db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (requires sharding, state unsharded) test DEBUG: Not deleting db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345360.36947.db (requires sharding, state sharding) test DEBUG: Not deleting db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345360.36947.db (2/3 success) test DEBUG: Successfully deleted db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345360.36947.db test DEBUG: Successfully deleted db /tmp/tmpz9arw0py/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '492c46354211255d442c968cf5982434', '41b5c4e2-2172-463b-bf23-a895055ea417-sda', '1717345360.45756', '1717345360.00000', '0', '', '1717345360.00000', 51, 0) test DEBUG: /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o0', 'created_at': '1717345361.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o1', 'created_at': '1717345362.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o2', 'created_at': '1717345363.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o3', 'created_at': '1717345364.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o4', 'created_at': '1717345365.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o5', 'created_at': '1717345366.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o6', 'created_at': '1717345367.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o7', 'created_at': '1717345368.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o8', 'created_at': '1717345369.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o9', 'created_at': '1717345370.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '41b5c4e2-2172-463b-bf23-a895055ea417-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 11, 'name': 'o10', 'created_at': '1717345371.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 12, 'name': 'o11', 'created_at': '1717345372.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 13, 'name': 'o12', 'created_at': '1717345373.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 14, 'name': 'o13', 'created_at': '1717345374.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 15, 'name': 'o14', 'created_at': '1717345375.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 16, 'name': 'o15', 'created_at': '1717345376.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 17, 'name': 'o16', 'created_at': '1717345377.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 18, 'name': 'o17', 'created_at': '1717345378.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 19, 'name': 'o18', 'created_at': '1717345379.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 20, 'name': 'o19', 'created_at': '1717345380.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '41b5c4e2-2172-463b-bf23-a895055ea417-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': 'o20', 'created_at': '1717345381.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 22, 'name': 'o21', 'created_at': '1717345382.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 23, 'name': 'o22', 'created_at': '1717345383.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 24, 'name': 'o23', 'created_at': '1717345384.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 25, 'name': 'o24', 'created_at': '1717345385.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 26, 'name': 'o25', 'created_at': '1717345386.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 27, 'name': 'o26', 'created_at': '1717345387.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 28, 'name': 'o27', 'created_at': '1717345388.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 29, 'name': 'o28', 'created_at': '1717345389.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 30, 'name': 'o29', 'created_at': '1717345390.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '41b5c4e2-2172-463b-bf23-a895055ea417-sda') test DEBUG: /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 30 (0.0368493s) test DEBUG: Synchronization for /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db has fallen more than 30 rows behind; moving on and will try again next pass. REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '492c46354211255d442c968cf5982434', '41b5c4e2-2172-463b-bf23-a895055ea417-sda', '1717345360.45756', '1717345360.00000', '0', '', '1717345360.00000', 51, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('41b5c4e2-2172-463b-bf23-a895055ea417-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14164 seconds (7.06000/s) test INFO: Removed 0 dbs test INFO: 1 successes, 1 failures test INFO: diff:1 diff_capped:1 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '492c46354211255d442c968cf5982434', '41b5c4e2-2172-463b-bf23-a895055ea417-sda', '1717345360.45756', '1717345360.00000', '0', '', '1717345360.00000', 51, 0) test DEBUG: /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row 30 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 31, 'name': 'o30', 'created_at': '1717345391.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 32, 'name': 'o31', 'created_at': '1717345392.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 33, 'name': 'o32', 'created_at': '1717345393.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 34, 'name': 'o33', 'created_at': '1717345394.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 35, 'name': 'o34', 'created_at': '1717345395.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 36, 'name': 'o35', 'created_at': '1717345396.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 37, 'name': 'o36', 'created_at': '1717345397.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 38, 'name': 'o37', 'created_at': '1717345398.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 39, 'name': 'o38', 'created_at': '1717345399.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 40, 'name': 'o39', 'created_at': '1717345400.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '41b5c4e2-2172-463b-bf23-a895055ea417-sda') test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part ... ok test.unit.container.test_replicator.TestReplicatorSync.test_local_ids test.unit.container.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue ... ok test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' - rejecting request test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range test.unit.common.middleware.test_helpers.TestFakeSwift.test_range ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon test.unit.common.middleware.test_tempauth.TestAuth.test_anon ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_options_call test.unit.common.middleware.test_tempauth.TestAuth.test_options_call ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm ... ok test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac test.unit.common.test_digest.TestDigestUtils.test_get_hmac ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type ... ok test.unit.common.test_splice.TestSplice.test_available test.unit.common.test_splice.TestSplice.test_available ... ok test.unit.common.test_splice.TestSplice.test_errno test.unit.common.test_splice.TestSplice.test_errno ... ok test.unit.common.test_splice.TestSplice.test_fileno test.unit.common.test_splice.TestSplice.test_fileno ... ok test.unit.common.test_splice.TestSplice.test_flags test.unit.common.test_splice.TestSplice.test_flags ... ok test.unit.common.test_splice.TestSplice.test_flags_list test.unit.common.test_splice.TestSplice.test_flags_list ... ok test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe ... ok test.unit.common.test_splice.TestSplice.test_unavailable test.unit.common.test_splice.TestSplice.test_unavailable ... ok test.unit.common.test_splice.TestSplice.test_unavailable_in_libc test.unit.common.test_splice.TestSplice.test_unavailable_in_libc ... ok test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges ... ok Sun Jun 2 04:22:35 2024 /tmp/tmp72zva20l/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile3542003 Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile3542003 Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile123 Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile3542003 Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile3542003 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile3542003 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:22:35 2024 /tmp/tmpzd8jn39r/unittest.profile3542003 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx043ae735b1fd44b8b9d4d-00665c9c5e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx6c7242e4d34d44c19a426-00665c9c5e) test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/PREFIX_uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9900>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com:8080', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage1.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage2.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage3.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f46c59f9600>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test INFO: - - 02/Jun/2024/16/22/31 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.577642202 1717345351.578055382 0 test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0017 - - 1717345351.577577829 1717345351.579263449 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.582063198 1717345351.582512617 0 test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0016 - - 1717345351.581977129 1717345351.583549976 0 test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.586477757 1717345351.586915016 0 test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0015 - - 1717345351.586388588 1717345351.587902308 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717345351.591333151 1717345351.591813326 0 test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0021 - - 1717345351.590658188 1717345351.592806339 0 test INFO: - - 02/Jun/2024/16/22/31 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.595742226 1717345351.596105576 0 test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0016 - - 1717345351.595579624 1717345351.597144365 0 test INFO: - - 02/Jun/2024/16/22/31 HEAD /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.600776672 1717345351.601139545 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345351.601773024 1717345351.602285385 0 test INFO: - - 02/Jun/2024/16/22/31 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345351.603230238 1717345351.603538275 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0044 - - 1717345351.600407124 1717345351.604766130 - test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717345351.607828379 1717345351.608293772 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345351.608947515 1717345351.609482527 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717345351.610495567 1717345351.610985518 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0045 - - 1717345351.607518911 1717345351.612029076 - test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.615858793 1717345351.616228104 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345351.616854429 1717345351.617330790 0 test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_init test.unit.common.test_internal_client.TestCompressingfileReader.test_init ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_read test.unit.common.test_internal_client.TestCompressingfileReader.test_read ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek test.unit.common.test_internal_client.TestCompressingfileReader.test_seek ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same ... ok test.unit.common.test_swob.TestTransferEncoding.test_is_chunked test.unit.common.test_swob.TestTransferEncoding.test_is_chunked ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_reserved test.unit.common.test_utils.TestFallocateReserve.test_all_reserved ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_disabled test.unit.common.test_utils.TestFallocateReserve.test_disabled ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space test.unit.common.test_utils.TestFallocateReserve.test_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve ... ok test.unit.container.test_sync.TestContainerSync.test_FileLikeIter test.unit.container.test_sync.TestContainerSync.test_FileLikeIter ... ok test.unit.container.test_sync.TestContainerSync.test_container_first_loop test.unit.container.test_sync.TestContainerSync.test_container_first_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_report test.unit.container.test_sync.TestContainerSync.test_container_report ... ok test.unit.container.test_sync.TestContainerSync.test_container_second_loop test.unit.container.test_sync.TestContainerSync.test_container_second_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_stop_at test.unit.container.test_sync.TestContainerSync.test_container_stop_at ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms ... ok cont2 DEBUG: Could not load '/tmp/tmpao6dmh8h/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpao6dmh8h/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0169 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0220 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0133 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0118 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd3412d2d79bc49058ea68-00665c9c4f" "proxy-server 3542011" 0.0017 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:41001/sda1/0/a/c" "txd3412d2d79bc49058ea68-00665c9c4f" "container-server 3542011" 0.0019 "-" 3542011 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:48709/sdb1/0/a/c" "txd3412d2d79bc49058ea68-00665c9c4f" "container-server 3542011" 0.0015 "-" 3542011 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txd3412d2d79bc49058ea68-00665c9c4f" "proxy-server 3542011" 0.0451 "-" 3542011 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txd3412d2d79bc49058ea68-00665c9c4f" "proxy-server 3542011" 0.0234 "-" 3542011 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a/c HTTP/1.0 201 - - t - - - txd3412d2d79bc49058ea68-00665c9c4f - 0.0567 - - 1717345359.682922840 1717345359.739639997 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txb76d2016b12e476a9fb33-00665c9c4f" "proxy-server 3542011" 0.0021 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:48709/sdb1/2/a1/c1" "txb76d2016b12e476a9fb33-00665c9c4f" "container-server 3542011" 0.0042 "-" 3542011 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:41001/sda1/2/a1/c1" "txb76d2016b12e476a9fb33-00665c9c4f" "container-server 3542011" 0.0021 "-" 3542011 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txb76d2016b12e476a9fb33-00665c9c4f" "proxy-server 3542011" 0.0246 "-" 3542011 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txb76d2016b12e476a9fb33-00665c9c4f" "proxy-server 3542011" 0.0410 "-" 3542011 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txb76d2016b12e476a9fb33-00665c9c4f - 0.0533 - - 1717345359.741792202 1717345359.795078039 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf19b893112324f6cabd0e-00665c9c4f" "proxy-server 3542011" 0.0097 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:41001/sda1/0/a/c1" "txf19b893112324f6cabd0e-00665c9c4f" "container-server 3542011" 0.0019 "-" 3542011 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:48709/sdb1/0/a/c1" "txf19b893112324f6cabd0e-00665c9c4f" "container-server 3542011" 0.0114 "-" 3542011 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txf19b893112324f6cabd0e-00665c9c4f" "proxy-server 3542011" 0.0384 "-" 3542011 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txf19b893112324f6cabd0e-00665c9c4f" "proxy-server 3542011" 0.0543 "-" 3542011 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txf19b893112324f6cabd0e-00665c9c4f - 0.0745 - - 1717345359.797056437 1717345359.871604204 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa33d7062be594dc392067-00665c9c4f" "proxy-server 3542011" 0.0101 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:48709/sdb1/2/a/c2" "txa33d7062be594dc392067-00665c9c4f" "container-server 3542011" 0.0136 "-" 3542011 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:41001/sda1/2/a/c2" "txa33d7062be594dc392067-00665c9c4f" "container-server 3542011" 0.0063 "-" 3542011 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa33d7062be594dc392067-00665c9c4f" "proxy-server 3542011" 0.0429 "-" 3542011 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:39 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa33d7062be594dc392067-00665c9c4f" "proxy-server 3542011" 0.0588 "-" 3542011 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/39 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txa33d7062be594dc392067-00665c9c4f - 0.0792 - - 1717345359.873694897 1717345359.952851295 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container ... ok test INFO: - - 02/Jun/2024/16/22/33 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345353.304673195 1717345353.304741383 - test INFO: - - 02/Jun/2024/16/22/34 GET / HTTP/1.0 200 - - b05bf940... - 8 - - - 0.0001 - - 1717345354.638145447 1717345354.638214111 - test INFO: - - 02/Jun/2024/16/22/34 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0000 - - 1717345354.639280319 1717345354.639330149 - test INFO: - - 02/Jun/2024/16/22/34 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717345354.640591860 1717345354.640674353 - test INFO: - - 02/Jun/2024/16/22/34 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345354.642086744 1717345354.642142057 - test INFO: - - 02/Jun/2024/16/22/34 GET / HTTP/1.0 200 - - ... - 8 - - - 0.0001 - - 1717345354.643335819 1717345354.643389940 - test INFO: - - 02/Jun/2024/16/22/35 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717345355.178332567 1717345355.178404093 - test INFO: - - 02/Jun/2024/16/22/35 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717345355.179508448 1717345355.179560661 - test INFO: - - 02/Jun/2024/16/22/35 GET / HTTP/1.0 200 - - - - 8 - - - 0.0000 - one%2Cand%20two 1717345355.180748224 1717345355.180797100 - test INFO: - - 02/Jun/2024/16/22/35 GET / HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717345355.181935072 1717345355.181984663 - test INFO: - - 02/Jun/2024/16/22/35 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345355.183354378 1717345355.183408022 - test INFO: - - 02/Jun/2024/16/22/35 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one 1717345355.184613705 1717345355.184664011 - test INFO: template which can be edited in config: HTTP/1.0 / GET {SMD5}c65475e457fea0951fbb9ec9596b2177 - 1.0000 26/Apr/1970/17/46/40 10000001.000000000 0.5 example.com test INFO: HTTP/1.0 /bucket/path/to/key GET AUTH_test bucket path/to/key test INFO: - - 02/Jun/2024/16/22/35 GET /%3Fx%3D3 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345355.189152718 1717345355.189202785 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 HEAD /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 POST /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 DELETE /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 OPTIONS /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/42 GET - HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET / HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /bad HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /baddy/mc_badderson HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1 HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1/ HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_gen_headers test.unit.common.test_direct_client.TestDirectClient.test_gen_headers ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry test.unit.common.test_direct_client.TestDirectClient.test_retry ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params ... ok test.unit.common.test_swob.TestMatch.test_match test.unit.common.test_swob.TestMatch.test_match ... ok test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_noquote test.unit.common.test_swob.TestMatch.test_match_noquote ... ok test.unit.common.test_swob.TestMatch.test_match_star test.unit.common.test_swob.TestMatch.test_match_star ... ok test.unit.common.test_utils.TestPairs.test_pairs test.unit.common.test_utils.TestPairs.test_pairs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name ... ok test ERROR: Failed to unlink '/tmp/tmpkh8oer7n/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpkh8oer7n/containers/part/suffix/hash/container_1717345376.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpkh8oer7n/containers/part/suffix/hash/container_1717345376.00000.db': ['/tmp/tmpkh8oer7n/containers/part/suffix/hash/container.db', '/tmp/tmpkh8oer7n/containers/part/suffix/hash/container_1717345377.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpkh8oer7n/containers/part/suffix/hash/container_1717345377.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716678000 test DEBUG: looking for objects in 1716678000 test DEBUG: checking container 1716674400 test DEBUG: looking for objects in 1716674400 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'something', 'last_modified': '1970-01-01T00:00:00.000000', 'hash': '0.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test ERROR: Error listing containers in account .misplaced_objects (Unexpected response: 503 Service Unavailable) test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: ppi_skip: Container policy (0) in the middle of a part power increase (PPI) test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'ppi_skip': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'bogus', 'last_modified': '1970-01-01T01:00:18.841870', 'hash': '3618.841870', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': '00000bogus', 'last_modified': '1970-01-01T01:00:00.000000', 'hash': '3600.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'invalid_record': 1, 'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1717340400 test DEBUG: looking for objects in 1717340400 test DEBUG: checking container 1717336800 test DEBUG: looking for objects in 1717336800 test DEBUG: checking container 1717333200 test DEBUG: looking for objects in 1717333200 test DEBUG: checking container 1717329600 test DEBUG: looking for objects in 1717329600 test DEBUG: checking container 1717326000 test DEBUG: looking for objects in 1717326000 test DEBUG: checking container 1717322400 test DEBUG: looking for objects in 1717322400 test DEBUG: checking container 1717318800 test DEBUG: looking for objects in 1717318800 test DEBUG: checking container 1717315200 test DEBUG: looking for objects in 1717315200 test DEBUG: checking container 1717311600 test DEBUG: looking for objects in 1717311600 test DEBUG: checking container 1716739200 test DEBUG: looking for objects in 1716739200 test DEBUG: checking container 1716735600 test DEBUG: looking for objects in 1716735600 test DEBUG: checking container 1716732000 test DEBUG: looking for objects in 1716732000 test DEBUG: checking container 1716728400 test DEBUG: looking for objects in 1716728400 test DEBUG: checking container 1716724800 test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse ... ok test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process ... ok test.unit.container.test_reconciler.TestReconciler.test_no_concurrency test.unit.container.test_reconciler.TestReconciler.test_no_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_object_delete test.unit.container.test_reconciler.TestReconciler.test_object_delete ... ok test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move test.unit.container.test_reconciler.TestReconciler.test_object_move ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking ... ok test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0013 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0002 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0010 "-" 3541997 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0267 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0015 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0123 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0107 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0021 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0219 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0133 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0116 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0129 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0042 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0135 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c_zero" 201 - "-" "-" "-" 0.0025 "-" 3541997 0 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0120 "-" 3541997 1 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0122 "-" 3541997 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0101 "-" 3541997 3 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0114 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0132 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0108 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0102 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0100 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0063 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0163 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0189 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0050 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0173 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3541997 3 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0111 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0147 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0129 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0143 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0029 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0086 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0119 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0106 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0099 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0058 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0150 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0024 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0097 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0120 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0107 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0113 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0161 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0086 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0118 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0222 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0023 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0035 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0028 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3541997 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield ... ok test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose test.unit.cli.test_ringcomposer.TestCommands.test_compose ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show test.unit.cli.test_ringcomposer.TestCommands.test_show ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware test DEBUG: tenant mismatch: BLAN_a != tenant_id test DEBUG: allow user with role(s) swiftoperator,admin as account admin test DEBUG: allow user with role(s) swiftoperator,admin as account admin test DEBUG: allowing proxy 127.0.0.1 for container-sync test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: Names allowed in acls. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL: allowme authorizing test DEBUG: Names allowed in acls. test DEBUG: user *:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:* allowed in ACL authorizing. test DEBUG: User tenant_name:user_name is not allowed to delete its own account test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Using identity: {'user': ('user_id', 'user_name'), 'tenant': ('test', 'tenant_name'), 'roles': ['reselleradmin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: User test has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. Sun Jun 2 04:22:42 2024 /tmp/log/swift/profile/default.profile456 Sun Jun 2 04:22:42 2024 /tmp/log/swift/profile/default.profile123 Sun Jun 2 04:22:42 2024 /tmp/log/swift/profile/default.profile3541995 45 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 9 0.002 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 9 0.001 0.000 0.001 0.000 :0(exec) 9 0.000 0.000 0.000 0.000 :0(getcwd) 9 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 9 0.000 0.000 0.000 0.000 :0(setprofile) test_digest_utils WARNING: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 test ERROR: Problem finding shard upper in '/tmp/tmpe3oseu4f/test_container.db': : LockTimeout (Nones) test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:full_control OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission ... ok test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0017 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmpynzl6v89/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0291 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0051 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0156 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0060 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0107 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0012 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0138 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0147 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0003 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0183 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0010 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0013 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0185 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0076 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0039 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0081 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0037 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0043 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0041 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmpso1fzlzy/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0076 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0012 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0013 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0166 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker ... ok test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0123 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0122 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0139 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0105 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0108 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0123 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0130 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0116 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0132 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0121 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0130 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0145 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0155 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0157 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0178 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0198 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0109 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0149 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0005 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0278 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0189 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0156 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0131 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0045 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0087 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0101 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0140 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0078 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0139 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0013 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0088 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0138 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0094 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0077 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0058 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0093 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0140 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0086 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0120 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0059 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0208 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.5405 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.1415 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3542001 - test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestAccountController.test_PUT_auto_create test.unit.account.test_server.TestAccountController.test_PUT_auto_create ... ok test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_not_found test.unit.account.test_server.TestAccountController.test_PUT_not_found ... ok test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestAccountController.test__call__raise_timeout test.unit.account.test_server.TestAccountController.test__call__raise_timeout ... ok test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD ... ok test.unit.account.test_server.TestAccountController.test_correct_allowed_method test.unit.account.test_server.TestAccountController.test_correct_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 41, 'name': 'o40', 'created_at': '1717345401.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 42, 'name': 'o41', 'created_at': '1717345402.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 43, 'name': 'o42', 'created_at': '1717345403.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 44, 'name': 'o43', 'created_at': '1717345404.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 45, 'name': 'o44', 'created_at': '1717345405.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 46, 'name': 'o45', 'created_at': '1717345406.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 47, 'name': 'o46', 'created_at': '1717345407.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 48, 'name': 'o47', 'created_at': '1717345408.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 49, 'name': 'o48', 'created_at': '1717345409.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 50, 'name': 'o49', 'created_at': '1717345410.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '41b5c4e2-2172-463b-bf23-a895055ea417-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 51, 'name': 'o101', 'created_at': '1717345511.00000', 'size': 0, 'content_type': 'content-type-new', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '41b5c4e2-2172-463b-bf23-a895055ea417-sda') test DEBUG: /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 51 (0.0311301s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '492c46354211255d442c968cf5982434', '41b5c4e2-2172-463b-bf23-a895055ea417-sda', '1717345360.45756', '1717345360.00000', '0', '', '1717345360.00000', 51, 0) test DEBUG: /tmp/tmphhwd4qqy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05524 seconds (18.10290/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:0 ts_repl:0 test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdb test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdb test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00051 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00044 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00046 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpmufda9iq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '48d2716d-880e-40a8-a51c-b9498a10b9f3-sda', '1717345360.75750', '1717344000.00000', '0', '', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmpmufda9iq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '48d2716d-880e-40a8-a51c-b9498a10b9f3-sda') test DEBUG: /tmp/tmpmufda9iq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0111165s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '48d2716d-880e-40a8-a51c-b9498a10b9f3-sda', '1717345360.75750', '1717344000.00000', '0', '', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('48d2716d-880e-40a8-a51c-b9498a10b9f3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpmufda9iq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10630 seconds (9.40736/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpmufda9iq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'c419a27d-ea54-473a-beee-9561b61d541b-sda', '1717345360.90671', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('c419a27d-ea54-473a-beee-9561b61d541b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'c419a27d-ea54-473a-beee-9561b61d541b-sda', '1717345360.90671', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('c419a27d-ea54-473a-beee-9561b61d541b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxqp1v3pt/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '67a74127-0fc0-4c00-bf60-bdfb17a661c0-sda', '1717345360.98902', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717345360.99538"]}', '1717344000.00000', 0, 1) test DEBUG: /tmp/tmpxqp1v3pt/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid ... ok Unlinked retiring db '/tmp/tmp3yc9bv1e/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container ... ok Unlinked retiring db '/tmp/tmpa73l7xix/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container ... ok Unlinked retiring db '/tmp/tmpqcdis1er/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container ... ok Unlinked retiring db '/tmp/tmpsjiw9lmr/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container ... ok test INFO: - - 02/Jun/2024/16/22/31 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.618300676 1717345351.618696451 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0173 - - 1717345351.615520239 1717345351.632794380 - test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.636197329 1717345351.636602402 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345351.637238026 1717345351.637707710 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717345351.638535976 1717345351.639052391 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0043 - - 1717345351.635914087 1717345351.640197992 - test INFO: - - 02/Jun/2024/16/22/31 DELETE /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345351.643184185 1717345351.643538713 0 test INFO: - - 02/Jun/2024/16/22/31 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0006 SSC - 1717345351.644310951 1717345351.644868135 0 test INFO: - - 02/Jun/2024/16/22/31 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345351.645827770 1717345351.646142960 0 test INFO: - - 02/Jun/2024/16/22/31 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0043 - - 1717345351.642894745 1717345351.647204161 - test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717345353.0486739, time window end: 1717345353.0486739, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 5, sync_point2: None, total_rows: 1 test-container-sync ERROR: ERROR Syncing : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 388, in container_sync if not self.container_sync_row( ^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: TestContainerSync.test_container_report..fake_container_sync_row() missing 1 required positional argument: 'realm_key' test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717345353.0517025, time window end: 1717345353.051714, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync ERROR: ERROR Syncing {'ROWID': 1, 'name': 'o'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 534, in container_sync_row row['created_at']) ~~~^^^^^^^^^^^^^^ KeyError: 'created_at' test-container-sync INFO: Container sync report: a/c, time window start: 1717345353.0518975, time window end: 1717345353.052292, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync INFO: Container sync report: a/c, time window start: 1717345353.0524645, time window end: 1717345353.0525372, puts: 0, posts: 0, deletes: 1, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms ... ok test.unit.container.test_sync.TestContainerSync.test_run_forever test.unit.container.test_sync.TestContainerSync.test_run_forever ... ok test.unit.container.test_sync.TestContainerSync.test_run_once test.unit.container.test_sync.TestContainerSync.test_run_once ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_GET test.unit.obj.test_server.TestZeroCopy.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_GET_big test.unit.obj.test_server.TestZeroCopy.test_GET_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_quarantine test.unit.obj.test_server.TestZeroCopy.test_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestObjectUpdater.test_conf_params test.unit.obj.test_updater.TestObjectUpdater.test_conf_params ... ok test.unit.obj.test_updater.TestObjectUpdater.test_creation test.unit.obj.test_updater.TestObjectUpdater.test_creation ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order ... ok test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: tx326b4b24ced14b2bb211a-00665c9c5b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: txfe32ea7efd2744909f86c-00665c9c5b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfe32ea7efd2744909f86c-00665c9c5b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfe32ea7efd2744909f86c-00665c9c5b) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: txfe32ea7efd2744909f86c-00665c9c5b) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx5b26f9f8b8334bcca6f7b-00665c9c5b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5b26f9f8b8334bcca6f7b-00665c9c5b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5b26f9f8b8334bcca6f7b-00665c9c5b) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx5b26f9f8b8334bcca6f7b-00665c9c5b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: Setting object count to 3120 (txn: txf9c92e7094ab40a79c5e7-00665c9c5d) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: Setting object count to 3120 (txn: tx9118ec2065b9467ea65c2-00665c9c5d) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: Setting object count to 3120 (txn: txe60c897bbcce466c85a65-00665c9c5d) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: Setting object count to 3120 (txn: tx3693f4089fdf4440a12f0-00665c9c5e) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking ... ok test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0002 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0203 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0095 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0096 "-" 3541997 - test INFO: - - - [02/Jun/2024:16:22:34 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0093 "-" 3541997 - Finished. Marked 0 objects for deletion. test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'X-Amz-Acl': 'public-read', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Grant AllUsers READ permission on the bucket /bucket test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory ... ok test DEBUG: looking for objects in 1716724800 test DEBUG: checking container 1716721200 test DEBUG: looking for objects in 1716721200 test DEBUG: checking container 1716717600 test DEBUG: looking for objects in 1716717600 test DEBUG: checking container 1716714000 test DEBUG: looking for objects in 1716714000 test DEBUG: checking container 1716710400 test DEBUG: looking for objects in 1716710400 test DEBUG: checking container 1716706800 test DEBUG: looking for objects in 1716706800 test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345377.289940) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345377.289940) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345377.289940) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345377.289940) from storage policy 1 test DEBUG: delete_attempt: '/AUTH_bob/c/o1' (1717345377.289940) in policy_index 1 will be deleted from policy_index 0 (1717345377.28994_0000000000000002) test INFO: delete_success: '/AUTH_bob/c/o1' (1717345377.289940) deleted from policy_index 0 (1717345377.28994_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (1717345377.289940) from policy_index 1 (1717345377.28994_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (1717345377.289940) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (1717345377.289940) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1717345377.289940) from the queue (1717345377.28994_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'delete_attempt': 1, 'delete_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3679.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3680.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383930) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 should be in policy_index 0 test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523d180>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523f580>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523ee00>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f542523ee00>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpaotnsj6p/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpaotnsj6p/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmptbvekdr0/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test INFO: Since Wed Dec 31 12:00:00 1969: container audits: 0 passed audit, 1 failed audit test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test ERROR: ERROR Could not get container info /tmp/tmptbvekdr0/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test DEBUG: Audit passed for test DEBUG: Audit passed for test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpsmatzw_i/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test.unit.common.test_db_auditor.TestAuditor.test_run_forever test.unit.common.test_db_auditor.TestAuditor.test_run_forever ... ok test.unit.common.test_db_auditor.TestAuditor.test_run_once test.unit.common.test_db_auditor.TestAuditor.test_run_once ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:write OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats ... ok test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txe15f68b4a0134427942ce-00665c9c52) test INFO: Client disconnected on read of '/a/c' (txn: txe15f68b4a0134427942ce-00665c9c52) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: tx378a98a3d18247b8963a4-00665c9c52) test INFO: Client disconnected on read of '/a/c' (txn: tx378a98a3d18247b8963a4-00665c9c52) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: txed948811b0d8407b976d8-00665c9c54) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: txed948811b0d8407b976d8-00665c9c54) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: txed948811b0d8407b976d8-00665c9c54) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: txed948811b0d8407b976d8-00665c9c54) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object ... ok test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0012 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0156 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0035 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0011 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0158 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0235 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0162 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0012 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0013 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0004 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0158 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0167 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0157 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0211 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0036 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0013 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0010 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0147 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0192 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0013 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0139 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0213 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0172 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0004 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file ... ok REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '67a74127-0fc0-4c00-bf60-bdfb17a661c0-sda') test DEBUG: /tmp/tmpxqp1v3pt/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0132966s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '67a74127-0fc0-4c00-bf60-bdfb17a661c0-sda', '1717345360.98902', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717345360.99538"]}', '1717344000.00000', 0, 1) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('67a74127-0fc0-4c00-bf60-bdfb17a661c0-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpxqp1v3pt/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13571 seconds (7.36886/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpxqp1v3pt/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '6d1c0b30-f679-433c-946e-81cd5907cf53-sda', '1717345361.14176', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('6d1c0b30-f679-433c-946e-81cd5907cf53-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '6d1c0b30-f679-433c-946e-81cd5907cf53-sda', '1717345361.14176', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('6d1c0b30-f679-433c-946e-81cd5907cf53-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpf298fd5o/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'facc03c4-5093-40fd-ba59-7352ad396838-sda', '1717345361.28750', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717345361.29435"]}', '1717344000.00000', 0, 1) test DEBUG: /tmp/tmpf298fd5o/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'facc03c4-5093-40fd-ba59-7352ad396838-sda') test DEBUG: /tmp/tmpf298fd5o/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.00828815s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'facc03c4-5093-40fd-ba59-7352ad396838-sda', '1717345361.28750', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717345361.29435"]}', '1717344000.00000', 0, 1) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('facc03c4-5093-40fd-ba59-7352ad396838-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpf298fd5o/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09977 seconds (10.02352/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpf298fd5o/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '23ed7749-7eed-425b-9e16-25a242f9734c-sda', '1717345361.41007', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('23ed7749-7eed-425b-9e16-25a242f9734c-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '23ed7749-7eed-425b-9e16-25a242f9734c-sda', '1717345361.41007', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('23ed7749-7eed-425b-9e16-25a242f9734c-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2i3upcyj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '6d78ada4c88803012f939458c7f84b74', '32f29114-b851-43c8-bd0e-eb1f646f7368-sda', '1717345361.48603', '1717345361.00000', '0', '', '1717345361.00000', 1, 1) test DEBUG: /tmp/tmp2i3upcyj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-unu', 'created_at': '1717345363.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o-nulo', 'created_at': '1717345364.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '32f29114-b851-43c8-bd0e-eb1f646f7368-sda') test DEBUG: /tmp/tmp2i3upcyj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.01105s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '6d78ada4c88803012f939458c7f84b74', '32f29114-b851-43c8-bd0e-eb1f646f7368-sda', '1717345361.48603', '1717345361.00000', '0', '', '1717345361.00000', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('32f29114-b851-43c8-bd0e-eb1f646f7368-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp2i3upcyj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09358 seconds (10.68628/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp2i3upcyj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '28ebfabc681d9362431d1a932eb1e712', '03a998fd-6467-4f5d-bcf8-1f20f67d1224-sda', '1717345361.59464', '1717344000', '0', '', '1717344000', 1, 0) test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize ... ok test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container ... ok test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0144 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0080 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.2270 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0014 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0178 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0142 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0080 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0013 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.0284 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0143 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0013 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0223 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0090 "-" 3542001 - test DEBUG: - - - [02/Jun/2024:16:22:36 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3542001 - test DEBUG: - - - [02/Jun/2024:16:22:36 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3542001 - test DEBUG: - - - [02/Jun/2024:16:22:36 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0004 "-" 3542001 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:22:36 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0071 "-" 3542001 - test DEBUG: - - - [02/Jun/2024:16:22:36 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0004 "-" 3542001 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0013 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0013 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0173 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0001 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0002 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0144 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0014 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0138 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0149 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0125 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0264 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0038 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0162 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0141 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0094 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0031 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0027 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0177 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3542001 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0122 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0201 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0073 "-" 3542001 - test.unit.account.test_server.TestAccountController.test_empty_policy_stats test.unit.account.test_server.TestAccountController.test_empty_policy_stats ... ok test.unit.account.test_server.TestAccountController.test_init test.unit.account.test_server.TestAccountController.test_init ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestAccountController.test_list_allowed_methods test.unit.account.test_server.TestAccountController.test_list_allowed_methods ... ok test.unit.account.test_server.TestAccountController.test_log_line_format test.unit.account.test_server.TestAccountController.test_log_line_format ... ok test.unit.account.test_server.TestAccountController.test_markers_with_reserved test.unit.account.test_server.TestAccountController.test_markers_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path ... ok test.unit.account.test_server.TestAccountController.test_not_allowed_method test.unit.account.test_server.TestAccountController.test_not_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_null_markers test.unit.account.test_server.TestAccountController.test_null_markers ... ok test.unit.account.test_server.TestAccountController.test_params_format test.unit.account.test_server.TestAccountController.test_params_format ... ok test.unit.account.test_server.TestAccountController.test_params_utf8 test.unit.account.test_server.TestAccountController.test_params_utf8 ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_non_default test.unit.account.test_server.TestAccountController.test_policy_stats_non_default ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_prefix_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestAccountController.test_serv_reserv test.unit.account.test_server.TestAccountController.test_serv_reserv ... ok test.unit.account.test_server.TestAccountController.test_through_call test.unit.account.test_server.TestAccountController.test_through_call ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path test.unit.account.test_server.TestAccountController.test_through_call_invalid_path ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 ... ok test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: Setting object count to 3120 (txn: tx148dafb7bf8d4d2ebd335-00665c9c5e) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: txc990a00414cf43cebede2-00665c9c5e) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txaf08dba74384473997f71-00665c9c5e) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx91e9e1b3f5804d938b741-00665c9c5e) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txc0a16452959d420fa579a-00665c9c5e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc0a16452959d420fa579a-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc0a16452959d420fa579a-00665c9c5e) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txc0a16452959d420fa579a-00665c9c5e) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx9f0daeb6f4b04fb788281-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9f0daeb6f4b04fb788281-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx9f0daeb6f4b04fb788281-00665c9c5e) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx9f0daeb6f4b04fb788281-00665c9c5e) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx8567e28187ee4a6b8bcf6-00665c9c5e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx8567e28187ee4a6b8bcf6-00665c9c5e) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx8567e28187ee4a6b8bcf6-00665c9c5e) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx8567e28187ee4a6b8bcf6-00665c9c5e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory ... ok test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383930) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (36123.383930) moved from policy_index 1 to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 (0000036123.38393_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (36123.383930) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/36000/1:/AUTH_bob/c/o1' (36123.383930) from the queue (0000036123.38393_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.123460) in policy_index 1 will be moved to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.123460) moved from policy_index 1 to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_failed: '/AUTH_bob/c/o1' (3600.123456) was not cleaned up in storage_policy 1 (Unexpected response: 503 Service Unavailable) test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: unavailable_destination: '/AUTH_bob/c/o1' (3600.123456) unable to determine the destination timestamp, if any test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_destination': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716678000 test DEBUG: looking for objects in 1716678000 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1716680097.364248) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1716680097.364248) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1716680097.364248) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1716680097.364248) from storage policy 1 test CRITICAL: lost_source: '/AUTH_jeb/c/o1' (1716680097.36425) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_jeb/c/o1' (1716680097.364248) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/1716678000/1:/AUTH_jeb/c/o1' (1716680097.364248) from the queue (1716680097.36425_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1717345377.372524) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1717345377.372524) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1717345377.372524) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1717345377.372524) from storage policy 1 test WARNING: unavailable_source: '/AUTH_jeb/c/o1' (1717345377.37252) in policy_index 1 responded 404 (0000000000.00000) test INFO: retry: '/AUTH_jeb/c/o1' (1717345377.372524) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim ... ok test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test ERROR: ERROR: Unable to access foo/bar: permission_denied test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp6vb1jkub/devices/sda1 (pid: 3541999) test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmp6vb1jkub/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345362.00000 test INFO: Object update sweep completed on /tmp/tmp6vb1jkub/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpglapmybm/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'Content-Md5': 'Je0by0I7C3IA9IX8X/ccjg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '25ed1bcb423b0b7200f485fc5ff71c8e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument ... ok test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe ... ok test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit "once" mode test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpe4gt5qiq/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpe4gt5qiq/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit "once" mode completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match ... ok test.unit.common.test_utils.TestCooperativeIterator.test_close test.unit.common.test_utils.TestCooperativeIterator.test_close ... ok test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version ... ok test.unit.common.test_utils.TestCooperativeIterator.test_init test.unit.common.test_utils.TestCooperativeIterator.test_init ... ok test.unit.common.test_utils.TestCooperativeIterator.test_iter test.unit.common.test_utils.TestCooperativeIterator.test_iter ... ok test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps ... ok test.unit.common.test_utils.TestCooperativeIterator.test_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_sleeps ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_eq test.unit.common.test_utils.TestNamespaceBoundList.test_eq ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_parse test.unit.common.test_utils.TestNamespaceBoundList.test_parse ... ok test.unit.obj.test_diskfile.TestDiskFile.test_close_error test.unit.obj.test_diskfile.TestDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit test.unit.obj.test_diskfile.TestDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_delete test.unit.obj.test_diskfile.TestDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects ... ok test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states ... ok test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: public None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: authenticated None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users ... ok test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging ... ok test.unit.proxy.test_mem_server.TestController.test_account_info_200 test.unit.proxy.test_mem_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_404 test.unit.proxy.test_mem_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_container_count test.unit.proxy.test_mem_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_account test.unit.proxy.test_mem_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock ... ok test.unit.proxy.test_mem_server.TestController.test_container_info_200 test.unit.proxy.test_mem_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_404 test.unit.proxy.test_mem_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_make_requests test.unit.proxy.test_mem_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_transfer_headers test.unit.proxy.test_mem_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path ... ok fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY jojo', (txn: tx4f362d569fb44b0984d5e-00665c9c4b) proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXYjojo a b c d e', (txn: tx38c33de09abf437295dfb-00665c9c4b) proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY a b c d e', (txn: tx0d8c960757c44f1687497-00665c9c4b) proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'GET /someurl HTTP/1.0\r\n', (txn: test-bad-req-trans-id) proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:22:35] "GET /someurl HTTP/1.1" 200 160 0.000088 proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:22:35] "GET /otherurl HTTP/1.1" 200 179 0.000058 proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:22:35] "GET /someurl HTTP/1.0" 200 186 0.000049 proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:22:35] "GET /someurl HTTP/1.0" 200 179 0.000050 proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:22:35] "GET /someurl HTTP/1.0" 200 180 0.000050 proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:22:35] "GET /someurl HTTP/1.0" 200 180 0.000051 proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True proxy INFO: (3542031) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3542031) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:22:35] "GET /someurl HTTP/1.0" 200 180 0.000056 proxy INFO: wsgi exiting proxy INFO: (3542031) wsgi exited, is_accepting=True WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify memory limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? test ERROR: Problem finding shard upper in '/tmp/tmpx42mmolg/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpx42mmolg/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpw90mi2au/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpw90mi2au/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle ... ok test.unit.obj.test_server.TestObjectController.test_DELETE test.unit.obj.test_server.TestObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET test.unit.obj.test_server.TestObjectController.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_but_expired test.unit.obj.test_server.TestObjectController.test_GET_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match test.unit.obj.test_server.TestObjectController.test_GET_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_none_match test.unit.obj.test_server.TestObjectController.test_GET_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine test.unit.obj.test_server.TestObjectController.test_GET_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD test.unit.obj.test_server.TestObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_OPTIONS test.unit.obj.test_server.TestObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_but_expired test.unit.obj.test_server.TestObjectController.test_POST_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_connection test.unit.obj.test_server.TestObjectController.test_POST_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_invalid_path test.unit.obj.test_server.TestObjectController.test_POST_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_not_exist test.unit.obj.test_server.TestObjectController.test_POST_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_system_metadata test.unit.obj.test_server.TestObjectController.test_POST_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_update_meta test.unit.obj.test_server.TestObjectController.test_POST_update_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_commits_data test.unit.obj.test_server.TestObjectController.test_PUT_commits_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_common test.unit.obj.test_server.TestObjectController.test_PUT_common ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_connection test.unit.obj.test_server.TestObjectController.test_PUT_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_etag test.unit.obj.test_server.TestObjectController.test_PUT_no_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite test.unit.obj.test_server.TestObjectController.test_PUT_overwrite ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_slow test.unit.obj.test_server.TestObjectController.test_PUT_slow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_works test.unit.obj.test_server.TestObjectController.test_REPLICATE_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test__call__returns_500 test.unit.obj.test_server.TestObjectController.test__call__returns_500 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_allowed_headers test.unit.obj.test_server.TestObjectController.test_allowed_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_http_connect test.unit.obj.test_server.TestObjectController.test_async_update_http_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_bad_sinces test.unit.obj.test_server.TestObjectController.test_bad_sinces ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_method test.unit.obj.test_server.TestObjectController.test_call_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_request test.unit.obj.test_server.TestObjectController.test_call_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_name_collision test.unit.obj.test_server.TestObjectController.test_call_name_collision ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_not_found test.unit.obj.test_server.TestObjectController.test_call_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_put test.unit.obj.test_server.TestObjectController.test_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_async test.unit.obj.test_server.TestObjectController.test_container_update_async ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_bad_args test.unit.obj.test_server.TestObjectController.test_container_update_bad_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_success test.unit.obj.test_server.TestObjectController.test_container_update_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_content_encoding test.unit.obj.test_server.TestObjectController.test_content_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_correct_allowed_method test.unit.obj.test_server.TestObjectController.test_correct_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_cap test.unit.obj.test_server.TestObjectController.test_delete_at_cap ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_negative test.unit.obj.test_server.TestObjectController.test_delete_at_negative ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_backend_replication test.unit.obj.test_server.TestObjectController.test_delete_backend_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_device_unavailable test.unit.obj.test_server.TestObjectController.test_device_unavailable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_dynamic_datadir test.unit.obj.test_server.TestObjectController.test_dynamic_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_init test.unit.obj.test_server.TestObjectController.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_list_allowed_methods test.unit.obj.test_server.TestObjectController.test_list_allowed_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_log_line_format test.unit.obj.test_server.TestObjectController.test_log_line_format ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_object_name_length test.unit.obj.test_server.TestObjectController.test_max_object_name_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_upload_time test.unit.obj.test_server.TestObjectController.test_max_upload_time ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_allowed_method test.unit.obj.test_server.TestObjectController.test_not_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_update_with_offset test.unit.obj.test_server.TestObjectController.test_object_update_with_offset ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_serv_reserv test.unit.obj.test_server.TestObjectController.test_serv_reserv ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_short_body test.unit.obj.test_server.TestObjectController.test_short_body ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects ... ok test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states ... ok Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Unlinked retiring db '/tmp/tmpcnjw8z89/containers/part/suffix/hash/container.db' Container 'a/c' cannot be set to sharded state while in sharded state Unlinked retiring db '/tmp/tmpcnjw8z89/containers/part/suffix/hash/container_1717345419.00000.db' Unlinked retiring db '/tmp/tmpcnjw8z89/containers/part/suffix/hash/container_1717345430.00000.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file ... ok test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0011 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0011 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0012 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0169 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0061 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:34 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0037 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.5550 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.1210 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0017 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0119 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.2195 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0018 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:35 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0017 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0348 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0174 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0136 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0019 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0164 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0174 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmpjxwjyf70/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0031 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0147 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0160 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmpf19y7czo/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0032 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0176 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0249 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0184 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0025 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs ... ok test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: Setting object count to 1068 (txn: tx6a99b9129f6a4a12b19ba-00665c9c60) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Setting object count to 1068 (txn: tx91dbc442e2b3432baefc2-00665c9c60) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txcef36d6252c140eb96e84-00665c9c63) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txcef36d6252c140eb96e84-00665c9c63) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txcef36d6252c140eb96e84-00665c9c63) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txcef36d6252c140eb96e84-00665c9c63) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txcef36d6252c140eb96e84-00665c9c63) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txcef36d6252c140eb96e84-00665c9c63) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txcef36d6252c140eb96e84-00665c9c63) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: txcef36d6252c140eb96e84-00665c9c63) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txd5054311d28b47a78c822-00665c9c65) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txd5054311d28b47a78c822-00665c9c65) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5054311d28b47a78c822-00665c9c65) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txd5054311d28b47a78c822-00665c9c65) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd5054311d28b47a78c822-00665c9c65) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txd5054311d28b47a78c822-00665c9c65) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txd5054311d28b47a78c822-00665c9c65) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: txd5054311d28b47a78c822-00665c9c65) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txab4459cd60e247409cd08-00665c9c68) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock ... ok REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('03a998fd-6467-4f5d-bcf8-1f20f67d1224-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '28ebfabc681d9362431d1a932eb1e712', '03a998fd-6467-4f5d-bcf8-1f20f67d1224-sda', '1717345361.59464', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('03a998fd-6467-4f5d-bcf8-1f20f67d1224-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2i3upcyj/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'da5681fd6018282f9312ba2b3b9ca266', '07aa9b66-7124-4f20-b0d3-1717ca685edb-sdb', '1717345361.48603', '1717345362.00000', '0', '', '1717345362.00000', 1, 1) test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row -1 REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-unu', 'created_at': '1717345365.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o-nulo', 'created_at': '1717345366.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '07aa9b66-7124-4f20-b0d3-1717ca685edb-sdb') test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.0112894s) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': '32f29114-b851-43c8-bd0e-eb1f646f7368-sda', 'sync_point': 2}],) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'da5681fd6018282f9312ba2b3b9ca266', '07aa9b66-7124-4f20-b0d3-1717ca685edb-sdb', '1717345361.48603', '1717345362.00000', '0', '', '1717345362.00000', 1, 1) test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, starting at row -1 REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-unu', 'created_at': '1717345365.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o-nulo', 'created_at': '1717345366.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '07aa9b66-7124-4f20-b0d3-1717ca685edb-sdb') test DEBUG: Replicating db /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '28ebfabc681d9362431d1a932eb1e712', 'f7664165-0b8a-46c1-90f3-cdb7a8acb04f-', '1717345361.59464', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.2:1002/sdc, nothing to do REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '28ebfabc681d9362431d1a932eb1e712', 'f7664165-0b8a-46c1-90f3-cdb7a8acb04f-', '1717345361.59464', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.0:1000/sda, nothing to do test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0293534s) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': '32f29114-b851-43c8-bd0e-eb1f646f7368-sda', 'sync_point': 2}],) test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.09767 seconds (20.47803/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:2 diff_capped:0 empty:0 hashmatch:0 no_change:2 remote_merge:0 rsync:0 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, 'edc0117d8ad835f9eef7c917b0854402', 'f7664165-0b8a-46c1-90f3-cdb7a8acb04f-', '1717345361.59464', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row 1 REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '0:/a/c/o-nulo', 'created_at': '1717345366.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717345366.00000', 'deleted': 0, 'storage_policy_index': 0}], 'f7664165-0b8a-46c1-90f3-cdb7a8acb04f-') test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.017175s) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': '03a998fd-6467-4f5d-bcf8-1f20f67d1224-sda', 'sync_point': 1}],) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, 'edc0117d8ad835f9eef7c917b0854402', 'f7664165-0b8a-46c1-90f3-cdb7a8acb04f-', '1717345361.59464', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, starting at row 1 REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '0:/a/c/o-nulo', 'created_at': '1717345366.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717345366.00000', 'deleted': 0, 'storage_policy_index': 0}], 'f7664165-0b8a-46c1-90f3-cdb7a8acb04f-') test DEBUG: /tmp/tmp2i3upcyj/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0133078s) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': '03a998fd-6467-4f5d-bcf8-1f20f67d1224-sda', 'sync_point': 1}],) test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '06df67c5a14134b2a97f05d23a72285d', 'b851f61c-cfe0-48a6-8835-6d727d89bd54-', '1717345361.89847', '1717345361.89417', '0', '', '1717345361.89417', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b851f61c-cfe0-48a6-8835-6d727d89bd54-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717345361.90656.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '06df67c5a14134b2a97f05d23a72285d', 'b851f61c-cfe0-48a6-8835-6d727d89bd54-', '1717345361.89847', '1717345361.89417', '0', '', '1717345361.89417', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345361.90642', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345361.90642', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345361.90642', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345361.90685', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345361.90685', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345361.90656', 'epoch': '1717345361.90656', 'reported': 0, 'tombstones': -1}], 'b851f61c-cfe0-48a6-8835-6d727d89bd54-') test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test ERROR: Failed to unlink '/tmp/tmpt1kwzfwi/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpt1kwzfwi/containers/part/suffix/hash/container_1717345404.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpt1kwzfwi/containers/part/suffix/hash/container_1717345404.00000.db': ['/tmp/tmpt1kwzfwi/containers/part/suffix/hash/container.db', '/tmp/tmpt1kwzfwi/containers/part/suffix/hash/container_1717345405.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpt1kwzfwi/containers/part/suffix/hash/container_1717345405.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: Unlinked retiring db '/tmp/tmp0_9l90p4/containers/part/suffix/hash/2dd3e24a-c339-4c71-8885-317e0301c348.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required ... ok Unlinked retiring db '/tmp/tmppx47zor0/containers/part/suffix/hash/container.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta ... ok test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0002 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0002 "-" 3542001 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0159 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0017 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0109 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0031 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0029 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0098 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0023 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0099 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0150 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0016 "-" 3542001 0 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0108 "-" 3542001 1 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0095 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0164 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0025 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0119 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0105 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0132 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0025 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0028 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0168 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0028 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0004 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0002 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0002 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0001 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0193 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0184 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 1 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:37 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0033 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0178 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0100 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0145 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0073 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0105 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0125 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0111 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0117 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0032 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0146 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0018 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0111 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0094 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0099 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0105 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0023 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0132 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0066 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0079 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0094 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0099 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0021 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0026 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0024 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0001 "-" 3542001 - test.unit.account.test_server.TestAccountController.test_utf8_metadata test.unit.account.test_server.TestAccountController.test_utf8_metadata ... ok test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects ... ok Unlinked retiring db '/tmp/tmpf31z6xa3/containers/part/suffix/hash/container-32bfce6e-aa83-438b-88df-068fe24533c9.db' Unlinked retiring db '/tmp/tmpf31z6xa3/containers/part/suffix/hash/container-70efefcf-2f10-4b67-b7c0-3fd8662e22c1.db' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging ... ok test.unit.container.test_sharder.TestCleavingContext.test_cursor test.unit.container.test_sharder.TestCleavingContext.test_cursor ... ok test.unit.container.test_sharder.TestCleavingContext.test_delete test.unit.container.test_sharder.TestCleavingContext.test_delete ... ok test.unit.container.test_sharder.TestCleavingContext.test_done test.unit.container.test_sharder.TestCleavingContext.test_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_init test.unit.container.test_sharder.TestCleavingContext.test_init ... ok test.unit.container.test_sharder.TestCleavingContext.test_iter test.unit.container.test_sharder.TestCleavingContext.test_iter ... ok test.unit.container.test_sharder.TestCleavingContext.test_load test.unit.container.test_sharder.TestCleavingContext.test_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_all test.unit.container.test_sharder.TestCleavingContext.test_load_all ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_range_done test.unit.container.test_sharder.TestCleavingContext.test_range_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_reset test.unit.container.test_sharder.TestCleavingContext.test_reset ... ok test.unit.container.test_sharder.TestCleavingContext.test_start test.unit.container.test_sharder.TestCleavingContext.test_start ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states ... ok test.unit.container.test_sharder.TestCleavingContext.test_store test.unit.container.test_sharder.TestCleavingContext.test_store ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style ... ok test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_logging test.unit.obj.test_auditor.TestAuditor.test_object_run_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_sleeper test.unit.obj.test_auditor.TestAuditor.test_sleeper ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init test.unit.obj.test_diskfile.TestDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_base.TestGetterSource.test_close test.unit.proxy.controllers.test_base.TestGetterSource.test_close ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_init test.unit.proxy.controllers.test_base.TestGetterSource.test_init ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_sort test.unit.proxy.controllers.test_base.TestGetterSource.test_sort ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test ERROR: unhandled_error: unable to upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 635, in ensure_object_in_right_location self.swift.upload_object( File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 844, in upload_object self.handle_request('PUT', path, headers, acceptable_statuses, fobj, File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 264, in handle_request resp = self.make_request(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'unhandled_error': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test WARNING: copy_failed: upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 returned Unexpected response: 503 Service Unavailable test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1716739199.000000) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1716739199.000000) from storage policy 1 test CRITICAL: lost_source: '/AUTH_bob/c/o1' (1716739199.00000) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1716739199.000000) from the queue (1716739199.00000_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.234570) in policy_index 1 will be moved to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.234570) moved from policy_index 1 to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3600.123456) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3600.123456) from the queue (0000003600.12346_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345377.423239) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345377.423239) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345377.423239) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345377.423239) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717345377.42324) in policy_index 1 responded 200 (1717345376.42324) test INFO: retry: '/AUTH_bob/c/o1' (1717345377.423239) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 1 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 should be in policy_index 1 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 0 test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces ... ok test.unit.container.test_reconciler.TestReconciler.test_processes_config test.unit.container.test_reconciler.TestReconciler.test_processes_config ... ok test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone ... ok test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy ... ok test ERROR: Problem finding shard upper in '/tmp/tmpe3oseu4f/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpisp9vdsu/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpisp9vdsu/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs ... ok test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpglapmybm/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345362.00000 test INFO: Object update sweep completed on /tmp/tmpglapmybm/devices/sda1 in 11.15s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 11.15s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp45ltoz4q/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmp45ltoz4q/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_new test DEBUG: Update sent for /.shards_a/c_shard_new/o /tmp/tmp45ltoz4q/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 test INFO: Object update sweep completed on /tmp/tmp45ltoz4q/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1b_itwzo/devices/sda1 (pid: 3541999) test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmp1b_itwzo/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 test INFO: Object update sweep completed on /tmp/tmp1b_itwzo/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1b_itwzo/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmp1b_itwzo/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_1 test DEBUG: Update sent for /.shards_a/c_shard_1/o /tmp/tmp1b_itwzo/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 test INFO: Object update sweep completed on /tmp/tmp1b_itwzo/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1nq_a62p/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_older/o /tmp/tmp1nq_a62p/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_new test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_new/o /tmp/tmp1nq_a62p/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_newer test INFO: Object update sweep completed on /tmp/tmp1nq_a62p/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1nq_a62p/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /.shards_a/c_shard_newer/o /tmp/tmp1nq_a62p/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 test INFO: Object update sweep completed on /tmp/tmp1nq_a62p/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpry1zkhcc/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_2 test INFO: Object update sweep completed on /tmp/tmpry1zkhcc/devices/sda1 in 0.03s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.04s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpry1zkhcc/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_2/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to None test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_3 test INFO: Object update sweep completed on /tmp/tmpry1zkhcc/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: nojunk None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Fri, 01 Apr 2014 12:00:00 GMT', 'X-Amz-Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000'} test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000'} test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000'} test DEBUG: checking permission: bucket None POST {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000'} test DEBUG: bucket None POST test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error /tmp/tmp1yeq41kv/tmppjudjp53, build version 4, id 399de0d614fe48919bb66e06db1adb12 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmp1yeq41kv/tmppjudjp53.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 Reassigned 192 (300.00%) partitions. Balance is now 5.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance test.unit.cli.test_ringbuilder.TestCommands.test_rebalance ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property ... ok test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345356.521789789 1717345356.521864176 - test INFO: - - 02/Jun/2024/16/22/36 GET /foo HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345356.522555113 1717345356.522608042 - test INFO: - - 02/Jun/2024/16/22/36 GET /foo/bar HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717345356.523039341 1717345356.523087502 - test INFO: - - 02/Jun/2024/16/22/36 GET /v1 HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717345356.523490667 1717345356.523536921 - test INFO: - - 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - 17 - - - 0.0001 SOS - 1717345356.527399540 1717345356.527501583 - test INFO: - - 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - - - - - 0.0000 - - 1717345356.529187441 1717345356.529232979 - test INFO: - - 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - 3 - - - 0.0001 - - 1717345356.530666351 1717345356.530756235 - test INFO: - - 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - 12 - - - 0.0000 - - 1717345356.531918287 1717345356.531959534 - test INFO: - - 02/Jun/2024/16/22/36 PUT /v1/a/c/o HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717345356.543628693 1717345356.543668032 1 test INFO: - - 02/Jun/2024/16/22/36 GET /v1/a/c HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345356.545276403 1717345356.545392752 1 test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345356.547174931 1717345356.547235250 - test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/22/36 GET / HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717345356.548206329 1717345356.548256159 - test INFO: - - 02/Jun/2024/16/22/36 GET /v1/something HTTP/1.0 200 - - - - 8 - - - 0.0002 - - 1717345356.549684286 1717345356.549837589 - test INFO: - - 02/Jun/2024/16/22/36 GET / HTTP/1.0 499 - - - - 4 - - - 0.0001 - - 1717345356.551860571 1717345356.551918030 - test INFO: - - 02/Jun/2024/16/22/36 POST /v1/a/c HTTP/1.0 200 - - - 11 8 - - Host:%20localhost:80 0.0000 - - 1717345356.553074360 1717345356.553106785 - test INFO: - - 02/Jun/2024/16/22/36 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717345356.554675341 1717345356.554709196 0 test INFO: - - 02/Jun/2024/16/22/36 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717345356.555910349 1717345356.555942774 -1 test INFO: - - 02/Jun/2024/16/22/36 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717345356.557224274 1717345356.557254314 - direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! test ERROR: Problem finding shard upper in '/tmp/tmphjv0_izz/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmphjv0_izz/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpe9lp0v5y/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpe9lp0v5y/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges ... ok test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET ... ok test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range ... ok test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init test.unit.obj.test_diskfile.TestECDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok test-replicator INFO: [worker 1/2 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 1/2 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/2 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 2/2 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/3 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 2/3 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 3/3 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 3/3 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3542005] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3542005] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=8804] Running object replicator in script mode. test-replicator DEBUG: [worker 1/3 pid=8804] debug message test-replicator INFO: [worker 1/3 pid=8804] info message test-replicator WARNING: [worker 1/3 pid=8804] warning message test-replicator ERROR: [worker 1/3 pid=8804] error message test-replicator INFO: [worker 1/3 pid=8804] Object replication complete (once). (0.00 minutes) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpcbbk4xy_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpcbbk4xy_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpcbbk4xy_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpcbbk4xy_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:13 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0117 "-" 3542005 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:13 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0165 "-" 3542005 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0429 "-" 3542005 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0138 "-" 3542005 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx0271e03db79c45be85f2d-00665c9c72" "proxy-server 3542005" 0.0022 "-" 3542005 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:60043/sda1/0/a/c" "tx0271e03db79c45be85f2d-00665c9c72" "container-server 3542005" 0.0023 "-" 3542005 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:60967/sdb1/0/a/c" "tx0271e03db79c45be85f2d-00665c9c72" "container-server 3542005" 0.0019 "-" 3542005 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx0271e03db79c45be85f2d-00665c9c72" "proxy-server 3542005" 0.0404 "-" 3542005 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx0271e03db79c45be85f2d-00665c9c72" "proxy-server 3542005" 0.0286 "-" 3542005 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx0271e03db79c45be85f2d-00665c9c72 - 0.0551 - - 1717345394.046330214 1717345394.101386547 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txc7cdd7a324244fce9c809-00665c9c72" "proxy-server 3542005" 0.0021 "-" 3542005 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:60967/sdb1/2/a1/c1" "txc7cdd7a324244fce9c809-00665c9c72" "container-server 3542005" 0.0017 "-" 3542005 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:60043/sda1/2/a1/c1" "txc7cdd7a324244fce9c809-00665c9c72" "container-server 3542005" 0.0016 "-" 3542005 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txc7cdd7a324244fce9c809-00665c9c72" "proxy-server 3542005" 0.0890 "-" 3542005 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txc7cdd7a324244fce9c809-00665c9c72" "proxy-server 3542005" 0.0539 "-" 3542005 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txc7cdd7a324244fce9c809-00665c9c72 - 0.1024 - - 1717345394.103679419 1717345394.206118584 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txc7d324ab9d6342a2b7663-00665c9c72" "proxy-server 3542005" 0.0085 "-" 3542005 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:60043/sda1/0/a/c1" "txc7d324ab9d6342a2b7663-00665c9c72" "container-server 3542005" 0.0106 "-" 3542005 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:60967/sdb1/0/a/c1" "txc7d324ab9d6342a2b7663-00665c9c72" "container-server 3542005" 0.0017 "-" 3542005 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txc7d324ab9d6342a2b7663-00665c9c72" "proxy-server 3542005" 0.0439 "-" 3542005 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txc7d324ab9d6342a2b7663-00665c9c72" "proxy-server 3542005" 0.0315 "-" 3542005 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txc7d324ab9d6342a2b7663-00665c9c72 - 0.0640 - - 1717345394.208356380 1717345394.272344112 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx523eae58f68a4de98f6ee-00665c9c72" "proxy-server 3542005" 0.0104 "-" 3542005 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:60043/sda1/2/a/c2" "tx523eae58f68a4de98f6ee-00665c9c72" "container-server 3542005" 0.0105 "-" 3542005 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:60967/sdb1/2/a/c2" "tx523eae58f68a4de98f6ee-00665c9c72" "container-server 3542005" 0.0017 "-" 3542005 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx523eae58f68a4de98f6ee-00665c9c72" "proxy-server 3542005" 0.0315 "-" 3542005 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx523eae58f68a4de98f6ee-00665c9c72" "proxy-server 3542005" 0.0479 "-" 3542005 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx523eae58f68a4de98f6ee-00665c9c72 - 0.0684 - - 1717345394.274590492 1717345394.343030691 2 test ERROR: Failed to unlink '/tmp/tmpw0qjx5sw/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpw0qjx5sw/containers/part/suffix/hash/container_1717345397.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpw0qjx5sw/containers/part/suffix/hash/container_1717345397.00000.db': ['/tmp/tmpw0qjx5sw/containers/part/suffix/hash/container.db', '/tmp/tmpw0qjx5sw/containers/part/suffix/hash/container_1717345398.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpw0qjx5sw/containers/part/suffix/hash/container_1717345398.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Skipping the update of recently changed /tmp/tmp4n2tps45/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmp4n2tps45/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmp4n2tps45/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmp4n2tps45/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping: /tmp/tmpy2tkw0k1/garbage is not a directory test DEBUG: Skipping: /tmp/tmpy2tkw0k1/garbage is not mounted test DEBUG: Skipping: /tmp/tmpjtzknx2g/sdq is not mounted test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test WARNING: Handoff requested (9) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpu654yv5g/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpu654yv5g/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpu654yv5g/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpu654yv5g/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0168 "-" 3542031 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0243 "-" 3542031 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0314 "-" 3542031 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0249 "-" 3542031 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx4341772e30864a539d4be-00665c9c7b" "proxy-server 3542031" 0.0023 "-" 3542031 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:54093/sda1/1/a/c" "tx4341772e30864a539d4be-00665c9c7b" "container-server 3542031" 0.0015 "-" 3542031 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:52365/sdb1/1/a/c" "tx4341772e30864a539d4be-00665c9c7b" "container-server 3542031" 0.0018 "-" 3542031 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx4341772e30864a539d4be-00665c9c7b" "proxy-server 3542031" 0.0823 "-" 3542031 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx4341772e30864a539d4be-00665c9c7b" "proxy-server 3542031" 0.0363 "-" 3542031 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/23 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx4341772e30864a539d4be-00665c9c7b - 0.0956 - - 1717345403.109157562 1717345403.204800367 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx71cb33d07a524dc69d9b5-00665c9c7b" "proxy-server 3542031" 0.0021 "-" 3542031 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:54093/sda1/1/a1/c1" "tx71cb33d07a524dc69d9b5-00665c9c7b" "container-server 3542031" 0.0015 "-" 3542031 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:52365/sdb1/1/a1/c1" "tx71cb33d07a524dc69d9b5-00665c9c7b" "container-server 3542031" 0.0013 "-" 3542031 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx71cb33d07a524dc69d9b5-00665c9c7b" "proxy-server 3542031" 0.0432 "-" 3542031 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx71cb33d07a524dc69d9b5-00665c9c7b" "proxy-server 3542031" 0.0205 "-" 3542031 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/23 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx71cb33d07a524dc69d9b5-00665c9c7b - 0.0555 - - 1717345403.207030773 1717345403.262556553 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx5a3965c4fa6941b3bb4d6-00665c9c7b" "proxy-server 3542031" 0.0116 "-" 3542031 - setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpylz253_4/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpylz253_4/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpylz253_4/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpylz253_4/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0152 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0108 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0116 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0104 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6e8df8e0f2e24a32bcb2b-00665c9c5a" "proxy-server 3542011" 0.0024 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:50259/sdb1/0/a/c" "tx6e8df8e0f2e24a32bcb2b-00665c9c5a" "container-server 3542011" 0.0022 "-" 3542011 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:35345/sda1/0/a/c" "tx6e8df8e0f2e24a32bcb2b-00665c9c5a" "container-server 3542011" 0.0020 "-" 3542011 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6e8df8e0f2e24a32bcb2b-00665c9c5a" "proxy-server 3542011" 0.0240 "-" 3542011 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6e8df8e0f2e24a32bcb2b-00665c9c5a" "proxy-server 3542011" 0.0480 "-" 3542011 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/50 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx6e8df8e0f2e24a32bcb2b-00665c9c5a - 0.0639 - - 1717345370.687907696 1717345370.751769066 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txfc98178188aa4849b2e77-00665c9c5a" "proxy-server 3542011" 0.0022 "-" 3542011 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:50259/sdb1/2/a1/c1" "txfc98178188aa4849b2e77-00665c9c5a" "container-server 3542011" 0.0020 "-" 3542011 0 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec test.unit.proxy.test_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Setting object count to 1058 (txn: txab4459cd60e247409cd08-00665c9c68) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Setting object count to 1024 (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Setting object count to 1024 (txn: tx8b648054a76044e3b48a3-00665c9c6b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Setting object count to 1072 (txn: tx306304cd74ba4ae3be80e-00665c9c6e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:52365/sdb1/2/a/c1" "tx5a3965c4fa6941b3bb4d6-00665c9c7b" "container-server 3542031" 0.0158 "-" 3542031 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:54093/sda1/2/a/c1" "tx5a3965c4fa6941b3bb4d6-00665c9c7b" "container-server 3542031" 0.0020 "-" 3542031 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx5a3965c4fa6941b3bb4d6-00665c9c7b" "proxy-server 3542031" 0.0590 "-" 3542031 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx5a3965c4fa6941b3bb4d6-00665c9c7b" "proxy-server 3542031" 0.0414 "-" 3542031 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/23 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx5a3965c4fa6941b3bb4d6-00665c9c7b - 0.0815 - - 1717345403.264420271 1717345403.345917463 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa951c72587994187a466e-00665c9c7b" "proxy-server 3542031" 0.0105 "-" 3542031 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:54093/sda1/1/a/c2" "txa951c72587994187a466e-00665c9c7b" "container-server 3542031" 0.0015 "-" 3542031 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:52365/sdb1/1/a/c2" "txa951c72587994187a466e-00665c9c7b" "container-server 3542031" 0.0105 "-" 3542031 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa951c72587994187a466e-00665c9c7b" "proxy-server 3542031" 0.0470 "-" 3542031 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:23 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa951c72587994187a466e-00665c9c7b" "proxy-server 3542031" 0.0325 "-" 3542031 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/23 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txa951c72587994187a466e-00665c9c7b - 0.0702 - - 1717345403.348693609 1717345403.418918371 2 test DEBUG: /tmp/tmpr00tmsff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345361.90656.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpr00tmsff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345361.90656.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '3dd797205641f85e4dda3ed2af5ffef6', 'c4a3b4e0-030c-4aaf-923c-2af753a737f4-', '1717345362.05044', '1717345362.04309', '0', '', '1717345362.04309', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.13862', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345362.13862', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345362.13862', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.14662', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.14662', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.05779', 'epoch': '1717345362.05779', 'reported': 0, 'tombstones': -1}], 'c4a3b4e0-030c-4aaf-923c-2af753a737f4-') test DEBUG: /tmp/tmp8ecir52v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.05779.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp8ecir52v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.05779.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '3dd797205641f85e4dda3ed2af5ffef6', 'c4a3b4e0-030c-4aaf-923c-2af753a737f4-', '1717345362.05044', '1717345362.12266', '0', '', '1717345362.04309', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.13862', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345362.13862', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345362.13862', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.14662', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.14662', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.05779', 'epoch': '1717345362.05779', 'reported': 0, 'tombstones': -1}], 'c4a3b4e0-030c-4aaf-923c-2af753a737f4-') test DEBUG: /tmp/tmp8ecir52v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.05779.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp8ecir52v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.05779.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '3dd797205641f85e4dda3ed2af5ffef6', '1f15289e-7008-4915-956f-1af6dbdc17c8-', '1717345362.31021', '1717345362.30337', '0', '', '1717345362.30337', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.41037', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345362.41037', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345362.41037', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.42210', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.42210', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.31744', 'epoch': '1717345362.31744', 'reported': 0, 'tombstones': -1}], '1f15289e-7008-4915-956f-1af6dbdc17c8-') test DEBUG: /tmp/tmp9ilvz_5h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.31744.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp9ilvz_5h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.31744.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '3dd797205641f85e4dda3ed2af5ffef6', '1f15289e-7008-4915-956f-1af6dbdc17c8-', '1717345362.31021', '1717345362.39780', '0', '', '1717345362.30337', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.41037', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345362.41037', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345362.41037', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.42210', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.42210', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.31744', 'epoch': '1717345362.31744', 'reported': 0, 'tombstones': -1}], '1f15289e-7008-4915-956f-1af6dbdc17c8-') test DEBUG: /tmp/tmp9ilvz_5h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.31744.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp9ilvz_5h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.31744.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'fcec58c5d82f0a62955fde911d9a114f', '2a7db1ff-1956-43ff-9c22-a356f10fcb4a-', '1717345362.58259', '1717345362.57709', '0', '', '1717345362.57709', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.68197', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.68197', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345362.68197', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.69023', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.69023', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.59120', 'epoch': '1717345362.59120', 'reported': 0, 'tombstones': -1}], '2a7db1ff-1956-43ff-9c22-a356f10fcb4a-') test DEBUG: /tmp/tmpap1ehy5z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.59120.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpap1ehy5z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.59120.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'fcec58c5d82f0a62955fde911d9a114f', '2a7db1ff-1956-43ff-9c22-a356f10fcb4a-', '1717345362.58259', '1717345362.66647', '0', '', '1717345362.57709', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:50 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:35345/sda1/2/a1/c1" "txfc98178188aa4849b2e77-00665c9c5a" "container-server 3542011" 0.0019 "-" 3542011 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfc98178188aa4849b2e77-00665c9c5a" "proxy-server 3542011" 4.4849 "-" 3542011 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfc98178188aa4849b2e77-00665c9c5a" "proxy-server 3542011" 4.5013 "-" 3542011 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/55 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txfc98178188aa4849b2e77-00665c9c5a - 4.5145 - - 1717345370.754275084 1717345375.268742561 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa81d0ce974bc4a2592b2f-00665c9c5f" "proxy-server 3542011" 0.0093 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:35345/sda1/0/a/c1" "txa81d0ce974bc4a2592b2f-00665c9c5f" "container-server 3542011" 0.0015 "-" 3542011 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:50259/sdb1/0/a/c1" "txa81d0ce974bc4a2592b2f-00665c9c5f" "container-server 3542011" 0.0074 "-" 3542011 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa81d0ce974bc4a2592b2f-00665c9c5f" "proxy-server 3542011" 0.0464 "-" 3542011 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa81d0ce974bc4a2592b2f-00665c9c5f" "proxy-server 3542011" 0.0303 "-" 3542011 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/55 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txa81d0ce974bc4a2592b2f-00665c9c5f - 0.0667 - - 1717345375.270939112 1717345375.337643147 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx349fac55278f468d901e1-00665c9c5f" "proxy-server 3542011" 0.0094 "-" 3542011 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:35345/sda1/2/a/c2" "tx349fac55278f468d901e1-00665c9c5f" "container-server 3542011" 0.0015 "-" 3542011 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:50259/sdb1/2/a/c2" "tx349fac55278f468d901e1-00665c9c5f" "container-server 3542011" 0.0091 "-" 3542011 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx349fac55278f468d901e1-00665c9c5f" "proxy-server 3542011" 0.0479 "-" 3542011 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:22:55 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx349fac55278f468d901e1-00665c9c5f" "proxy-server 3542011" 0.0329 "-" 3542011 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/22/55 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx349fac55278f468d901e1-00665c9c5f - 0.0679 - - 1717345375.339597940 1717345375.407461882 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0177 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0186 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0023 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0166 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0164 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0041 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0225 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0011 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0151 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0030 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0015 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0009 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0010 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0005 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0004 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0004 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0004 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0006 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0019 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0009 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0011 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0005 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0004 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0004 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0012 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0005 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0004 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0007 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0004 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0005 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0013 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0162 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0034 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0033 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0348 "-" 3542007 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0007 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0004 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0010 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0174 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0159 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0146 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0082 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0033 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted ... ok test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 will be moved to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 0 to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 0 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 0 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/0:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c ت/o1 ت' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c ت/o1 ت' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c ت/o1 ت' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345377.471450) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345377.471450) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345377.471450) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345377.471450) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717345377.47145) in policy_index 1 responded 503 (0000000000.00000) test INFO: retry: '/AUTH_bob/c/o1' (1717345377.471450) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345377.479956) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345377.479956) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345377.479956) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345377.479956) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717345377.47996) in policy_index 1 responded 404 (1717345377.47996_0000000000000002) test INFO: retry: '/AUTH_bob/c/o1' (1717345377.479956) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: unavailable_container: '/AUTH_bob/c/o1' (3618.841870) unable to determine the destination policy_index test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_container': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile ... ok test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible ... ok test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object ... ok proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp2e_l0_7o/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp2e_l0_7o/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp2e_l0_7o/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp2e_l0_7o/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0117 "-" 3542029 - HTTP PERF: 0.01624 seconds to PUT 127.0.0.1:35179 /sda1/0/a) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0120 "-" 3542029 - HTTP PERF: 0.01447 seconds to PUT 127.0.0.1:47517 /sdb1/0/a) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0108 "-" 3542029 - HTTP PERF: 0.01426 seconds to PUT 127.0.0.1:35179 /sda1/2/a1) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0120 "-" 3542029 - HTTP PERF: 0.01459 seconds to PUT 127.0.0.1:47517 /sdb1/2/a1) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx45bc421d5b054b6193133-00665c9c7f" "proxy-server 3542029" 0.0024 "-" 3542029 - HTTP PERF: 0.00475 seconds to HEAD 127.0.0.1:35179 /sda1/0/a?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:41387/sda1/3/a/c" "tx45bc421d5b054b6193133-00665c9c7f" "container-server 3542029" 0.0018 "-" 3542029 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:45113/sdb1/3/a/c" "tx45bc421d5b054b6193133-00665c9c7f" "container-server 3542029" 0.0035 "-" 3542029 0 HTTP PERF: 0.00955 seconds to PUT 127.0.0.1:35179 /sda1/0/a/c) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx45bc421d5b054b6193133-00665c9c7f" "proxy-server 3542029" 0.0252 "-" 3542029 0 HTTP PERF: 0.02706 seconds to PUT 127.0.0.1:47517 /sdb1/0/a/c) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx45bc421d5b054b6193133-00665c9c7f" "proxy-server 3542029" 0.0412 "-" 3542029 0 HTTP PERF: 0.04339 seconds to PUT 127.0.0.1:45113 /sdb1/3/a/c) HTTP PERF: 0.04459 seconds to PUT 127.0.0.1:41387 /sda1/3/a/c) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/27 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx45bc421d5b054b6193133-00665c9c7f - 0.0544 - - 1717345407.730054379 1717345407.784417868 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx69d9da7f6b254ebe8a4ea-00665c9c7f" "proxy-server 3542029" 0.0020 "-" 3542029 - HTTP PERF: 0.00418 seconds to HEAD 127.0.0.1:47517 /sdb1/2/a1?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:45113/sdb1/0/a1/c1" "tx69d9da7f6b254ebe8a4ea-00665c9c7f" "container-server 3542029" 0.0017 "-" 3542029 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:41387/sda1/0/a1/c1" "tx69d9da7f6b254ebe8a4ea-00665c9c7f" "container-server 3542029" 0.0020 "-" 3542029 0 HTTP PERF: 0.00780 seconds to PUT 127.0.0.1:35179 /sda1/2/a1/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx69d9da7f6b254ebe8a4ea-00665c9c7f" "proxy-server 3542029" 0.0234 "-" 3542029 0 HTTP PERF: 0.02541 seconds to PUT 127.0.0.1:47517 /sdb1/2/a1/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx69d9da7f6b254ebe8a4ea-00665c9c7f" "proxy-server 3542029" 0.0381 "-" 3542029 0 HTTP PERF: 0.04130 seconds to PUT 127.0.0.1:41387 /sda1/0/a1/c1) HTTP PERF: 0.04097 seconds to PUT 127.0.0.1:45113 /sdb1/0/a1/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/27 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx69d9da7f6b254ebe8a4ea-00665c9c7f - 0.0511 - - 1717345407.786489964 1717345407.837563038 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx64a56adbcfea4a83b67d9-00665c9c7f" "proxy-server 3542029" 0.0207 "-" 3542029 - HTTP PERF: 0.02301 seconds to HEAD 127.0.0.1:35179 /sda1/0/a?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:41387/sda1/3/a/c1" "tx64a56adbcfea4a83b67d9-00665c9c7f" "container-server 3542029" 0.0107 "-" 3542029 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:45113/sdb1/3/a/c1" "tx64a56adbcfea4a83b67d9-00665c9c7f" "container-server 3542029" 0.0020 "-" 3542029 1 HTTP PERF: 0.01719 seconds to PUT 127.0.0.1:47517 /sdb1/0/a/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx64a56adbcfea4a83b67d9-00665c9c7f" "proxy-server 3542029" 0.0327 "-" 3542029 1 HTTP PERF: 0.03453 seconds to PUT 127.0.0.1:35179 /sda1/0/a/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx64a56adbcfea4a83b67d9-00665c9c7f" "proxy-server 3542029" 0.0474 "-" 3542029 1 HTTP PERF: 0.04968 seconds to PUT 127.0.0.1:41387 /sda1/3/a/c1) HTTP PERF: 0.05062 seconds to PUT 127.0.0.1:45113 /sdb1/3/a/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/27 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx64a56adbcfea4a83b67d9-00665c9c7f - 0.0782 - - 1717345407.839742184 1717345407.917987108 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx32d3502d73794266b4717-00665c9c7f" "proxy-server 3542029" 0.0095 "-" 3542029 - HTTP PERF: 0.01192 seconds to HEAD 127.0.0.1:35179 /sda1/0/a?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:41387/sda1/0/a/c2" "tx32d3502d73794266b4717-00665c9c7f" "container-server 3542029" 0.0078 "-" 3542029 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:45113/sdb1/0/a/c2" "tx32d3502d73794266b4717-00665c9c7f" "container-server 3542029" 0.0017 "-" 3542029 2 HTTP PERF: 0.01353 seconds to PUT 127.0.0.1:35179 /sda1/0/a/c2) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx32d3502d73794266b4717-00665c9c7f" "proxy-server 3542029" 0.0303 "-" 3542029 2 HTTP PERF: 0.03210 seconds to PUT 127.0.0.1:47517 /sdb1/0/a/c2) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:27 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx32d3502d73794266b4717-00665c9c7f" "proxy-server 3542029" 0.0451 "-" 3542029 2 HTTP PERF: 0.04731 seconds to PUT 127.0.0.1:41387 /sda1/0/a/c2) HTTP PERF: 0.04825 seconds to PUT 127.0.0.1:45113 /sdb1/0/a/c2) test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/27 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx32d3502d73794266b4717-00665c9c7f - 0.0649 - - 1717345407.920404434 1717345407.985274792 2 test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0175 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0091 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0112 "-" 3542001 - test INFO: - - - [02/Jun/2024:16:22:38 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0137 "-" 3542001 - test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission ... ok test ERROR: Failed to unlink '/tmp/tmplw9hdkpz/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmplw9hdkpz/containers/part/suffix/hash/container_1717345408.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmplw9hdkpz/containers/part/suffix/hash/container_1717345408.00000.db': ['/tmp/tmplw9hdkpz/containers/part/suffix/hash/container.db', '/tmp/tmplw9hdkpz/containers/part/suffix/hash/container_1717345409.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmplw9hdkpz/containers/part/suffix/hash/container_1717345409.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpnryexd_3/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpnryexd_3/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpnryexd_3/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpnryexd_3/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0109 "-" 3541995 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0107 "-" 3541995 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0099 "-" 3541995 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0132 "-" 3541995 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txca9a0c6627034143b0f89-00665c9c81" "proxy-server 3541995" 0.0027 "-" 3541995 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:46565/sda1/0/a/c" "txca9a0c6627034143b0f89-00665c9c81" "container-server 3541995" 0.0014 "-" 3541995 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:48759/sdb1/0/a/c" "txca9a0c6627034143b0f89-00665c9c81" "container-server 3541995" 0.0013 "-" 3541995 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txca9a0c6627034143b0f89-00665c9c81" "proxy-server 3541995" 0.0285 "-" 3541995 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txca9a0c6627034143b0f89-00665c9c81" "proxy-server 3541995" 0.0179 "-" 3541995 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/29 PUT /v1/a/c HTTP/1.0 201 - - t - - - txca9a0c6627034143b0f89-00665c9c81 - 0.0405 - - 1717345409.887047529 1717345409.927591324 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txa88084dcdc2b427db2de0-00665c9c81" "proxy-server 3541995" 0.0016 "-" 3541995 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:48759/sdb1/2/a1/c1" "txa88084dcdc2b427db2de0-00665c9c81" "container-server 3541995" 0.0028 "-" 3541995 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:46565/sda1/2/a1/c1" "txa88084dcdc2b427db2de0-00665c9c81" "container-server 3541995" 0.0014 "-" 3541995 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txa88084dcdc2b427db2de0-00665c9c81" "proxy-server 3541995" 0.0180 "-" 3541995 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txa88084dcdc2b427db2de0-00665c9c81" "proxy-server 3541995" 0.0309 "-" 3541995 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/29 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txa88084dcdc2b427db2de0-00665c9c81 - 0.0405 - - 1717345409.929381132 1717345409.969854116 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:29 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd06abbea27bf40ffb40ac-00665c9c81" "proxy-server 3541995" 0.0062 "-" 3541995 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:46565/sda1/0/a/c1" "txd06abbea27bf40ffb40ac-00665c9c81" "container-server 3541995" 0.0014 "-" 3541995 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:48759/sdb1/0/a/c1" "txd06abbea27bf40ffb40ac-00665c9c81" "container-server 3541995" 0.0066 "-" 3541995 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd06abbea27bf40ffb40ac-00665c9c81" "proxy-server 3541995" 0.0214 "-" 3541995 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd06abbea27bf40ffb40ac-00665c9c81" "proxy-server 3541995" 0.0346 "-" 3541995 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/30 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txd06abbea27bf40ffb40ac-00665c9c81 - 0.0492 - - 1717345409.971681595 1717345410.020873547 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx08e6446f029c437e8f5a3-00665c9c82" "proxy-server 3541995" 0.0070 "-" 3541995 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:48759/sdb1/2/a/c2" "tx08e6446f029c437e8f5a3-00665c9c82" "container-server 3541995" 0.0099 "-" 3541995 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:46565/sda1/2/a/c2" "tx08e6446f029c437e8f5a3-00665c9c82" "container-server 3541995" 0.0014 "-" 3541995 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx08e6446f029c437e8f5a3-00665c9c82" "proxy-server 3541995" 0.0268 "-" 3541995 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:30 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx08e6446f029c437e8f5a3-00665c9c82" "proxy-server 3541995" 0.0395 "-" 3541995 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/30 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx08e6446f029c437e8f5a3-00665c9c82 - 0.0550 - - 1717345410.022635937 1717345410.077673435 2 test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Not a path query: /, skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Cached creds valid test DEBUG: Connecting with tenant: tenant_id test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Cached keystone credentials test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_creation test.unit.common.test_db_replicator.TestDBReplicator.test_creation ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response ... ok test ERROR: Failed to unlink '/tmp/tmpsfkng5gc/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpsfkng5gc/containers/part/suffix/hash/container_1717345405.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpsfkng5gc/containers/part/suffix/hash/container_1717345405.00000.db': ['/tmp/tmpsfkng5gc/containers/part/suffix/hash/container.db', '/tmp/tmpsfkng5gc/containers/part/suffix/hash/container_1717345406.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpsfkng5gc/containers/part/suffix/hash/container_1717345406.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Base DELETE returning 503 for (204, 204, 404) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ef1a7da5b8a4ad78019f-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ef1a7da5b8a4ad78019f-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ef1a7da5b8a4ad78019f-00665c9c7f) test ERROR: Container HEAD returning 503 for [] (txn: tx2ef1a7da5b8a4ad78019f-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info test.unit.proxy.controllers.test_base.TestFuncs.test_get_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit ... ok swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. Reassigned 768 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) Reassigned 192 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (0:24:00 remaining) No partitions could be reassigned. There is no need to do so at this time Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 1000.0 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) ------------------------------------------------------------------------------- An error has occurred during ring validation. Common causes of failure are rings that are empty or do not have enough devices to accommodate the replica count. Original exception message: There are no devices in this ring, or all devices have been deleted ------------------------------------------------------------------------------- Partition power increase in progress. You need to finish the increase first before rebalancing. Reassigned 0 (0.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 20 (31.25%) partitions. Balance is now 24.48. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 24.48 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 33.33 ------------------------------------------------------------------------------- NOTE: Dispersion of 33.333333 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 16.67 ------------------------------------------------------------------------------- NOTE: Dispersion of 16.666667 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. test.unit.cli.test_ringbuilder.TestCommands.test_remove_device test.unit.cli.test_ringbuilder.TestCommands.test_remove_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching ... ok test INFO: Object update sweep starting on /tmp/tmpry1zkhcc/devices/sda1 (pid: 3541999) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_3/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 to None test INFO: Object update sweep completed on /tmp/tmpry1zkhcc/devices/sda1 in 11.30s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 11.30s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpry1zkhcc/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpry1zkhcc/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345372.00000 test INFO: Object update sweep completed on /tmp/tmpry1zkhcc/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbh4cwf5z/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /.shards_a/shard_c/o /tmp/tmpbh4cwf5z/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345385.00000 test INFO: Object update sweep completed on /tmp/tmpbh4cwf5z/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbh4cwf5z/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpbh4cwf5z/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345386.00000 test INFO: Object update sweep completed on /tmp/tmpbh4cwf5z/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbh4cwf5z/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpbh4cwf5z/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345387.00000 test INFO: Object update sweep completed on /tmp/tmpbh4cwf5z/devices/sda1 in 0.00s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbh4cwf5z/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpbh4cwf5z/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345388.00000 test INFO: Object update sweep completed on /tmp/tmpbh4cwf5z/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbh4cwf5z/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpbh4cwf5z/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345389.00000 test INFO: Object update sweep completed on /tmp/tmpbh4cwf5z/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine ... ok test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2 new rows) test-replicator ERROR: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (negative max_row_delta: -1) test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Failed to delete db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py ERROR reading HTTP response from {'replication_ip': '127.0.0.1', 'replication_port': 80, 'device': 'sdb1'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 170, in replicate self.request('REPLICATE', self.path, body, File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 353, in other_req raise Exception('blah') Exception: blah test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted ... ok test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Setting object count to 1072 (txn: tx2f9a1df6fbd34f559d2fd-00665c9c71) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Setting object count to 1072 (txn: tx1cbca7aa26ea4b4185416-00665c9c74) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test INFO: - - - [02/Jun/2024:16:22:36 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0036 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0037 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:37 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0036 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0013 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0011 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0114 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0121 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:41 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0022 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0065 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:42 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0098 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0208 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0132 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0046 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0026 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmpodta1hwi/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0065 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0042 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0180 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0038 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0192 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0163 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0206 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test ERROR: Unhandled Exception trying to reconcile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 852, in run_once self.reconcile() File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 831, in reconcile for container in self._iter_containers(): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 748, in _iter_containers one_page = list(itertools.islice( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 331, in _iter_items resp = self.make_request( ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: Reconciler Stats: {} sharder-test DEBUG: Unlinked retiring db '/tmp/tmpeifugv_e/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Unexpected response while deleting object account container obj: 404 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 412 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 500 test-expirer DEBUG: b'' test-expirer ERROR: Exception while deleting object account container obj : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer ERROR: Exception while deleting object .expiring_objects 1717258978 1717258978-a0/c0/o0 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717258978 1717258978-a1/c1/o1 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717258978 1717258978-a2/c2/o2 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717258978 1717258978-a3/c3/o3 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717258978 1717258978-a4/c4/o4 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345377 1717345377-a5/c5/o5 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init test.unit.obj.test_expirer.TestObjectExpirer.test_init ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_report test.unit.obj.test_expirer.TestObjectExpirer.test_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_default test.unit.obj.test_server.TestConfigOptionHandling.test_default ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.68197', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.68197', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345362.68197', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.69023', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.69023', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.59120', 'epoch': '1717345362.59120', 'reported': 0, 'tombstones': -1}], '2a7db1ff-1956-43ff-9c22-a356f10fcb4a-') test DEBUG: /tmp/tmpap1ehy5z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.59120.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpap1ehy5z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.59120.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'fcec58c5d82f0a62955fde911d9a114f', 'b0578b2b-74cf-4d81-8b6e-15eda3903016-', '1717345362.84226', '1717345362.83734', '0', '', '1717345362.83734', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.92832', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.92832', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345362.92832', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.93650', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.93650', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.85019', 'epoch': '1717345362.85019', 'reported': 0, 'tombstones': -1}], 'b0578b2b-74cf-4d81-8b6e-15eda3903016-') test DEBUG: /tmp/tmp43pct3id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.85019.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp43pct3id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.85019.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'fcec58c5d82f0a62955fde911d9a114f', 'b0578b2b-74cf-4d81-8b6e-15eda3903016-', '1717345362.84226', '1717345362.91524', '0', '', '1717345362.83734', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345362.92832', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.92832', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345362.92832', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345362.93650', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345362.93650', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345362.85019', 'epoch': '1717345362.85019', 'reported': 0, 'tombstones': -1}], 'b0578b2b-74cf-4d81-8b6e-15eda3903016-') test DEBUG: /tmp/tmp43pct3id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.85019.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp43pct3id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345362.85019.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '88a0d8945546ab4431c8e6f928ddec0b', '78112bdb-3bae-4268-9675-abf591f10b6a-', '1717345363.08178', '1717345363.07582', '0', '', '1717345363.07582', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345363.09062', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345363.09062', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345363.09062', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345363.09102', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345363.09102', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345363.09080', 'epoch': '1717345363.09080', 'reported': 0, 'tombstones': -1}], '78112bdb-3bae-4268-9675-abf591f10b6a-') test DEBUG: /tmp/tmp91b40fy8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.09080.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp91b40fy8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.09080.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '88a0d8945546ab4431c8e6f928ddec0b', '78112bdb-3bae-4268-9675-abf591f10b6a-', '1717345363.08178', '1717345363.16198', '0', '', '1717345363.07582', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345363.09062', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345363.09062', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345363.09062', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345363.09102', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345363.09102', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345363.09080', 'epoch': '1717345363.09080', 'reported': 0, 'tombstones': -1}], '78112bdb-3bae-4268-9675-abf591f10b6a-') test DEBUG: /tmp/tmp91b40fy8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.09080.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp91b40fy8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.09080.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '88a0d8945546ab4431c8e6f928ddec0b', 'b5263950-d4b8-49cb-84fb-45c4365deb6f-', '1717345363.26458', '1717345363.25895', '0', '', '1717345363.25895', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345363.27066', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345363.27066', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345363.27066', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345363.27099', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345363.27099', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345363.27082', 'epoch': '1717345363.27082', 'reported': 0, 'tombstones': -1}], 'b5263950-d4b8-49cb-84fb-45c4365deb6f-') test DEBUG: /tmp/tmpmveb3gei/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.27082.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpmveb3gei/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.27082.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '88a0d8945546ab4431c8e6f928ddec0b', 'b5263950-d4b8-49cb-84fb-45c4365deb6f-', '1717345363.26458', '1717345363.33730', '0', '', '1717345363.25895', 99, 0) test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values ... ok test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/0 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 1, 'weight': 10.0, 'zone': 1, 'ip': '1.1.1.1', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.1', 'replication_port': 6200, 'region': 1}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 675, in _replicate_object success = self._repl_to_node(node, broker, partition, info, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 481, in _repl_to_node return self._handle_sync_response(node, response, info, broker, http, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 486, in _handle_sync_response if response.status == HTTP_NOT_FOUND: # completely missing, rsync ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 713, in status raise self._status Exception: ugh test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 2, 'weight': 10.0, 'zone': 2, 'ip': '1.1.1.2', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.2', 'replication_port': 6200, 'region': 2}: Timeout (Nones) test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (1/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator ERROR: ERROR Remote drive not mounted {'id': 3, 'weight': 10.0, 'zone': 3, 'ip': '1.1.1.3', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.3', 'replication_port': 6200, 'region': 1} test-replicator ERROR: ERROR Remote drive not mounted {'id': 4, 'weight': 10.0, 'zone': 4, 'ip': '1.1.1.4', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.4', 'replication_port': 6200, 'region': 2} test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.5:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (3/5 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator ERROR: Unable to decode remote metadata 'metadata' test-replicator ERROR: Quarantining DB test-replicator ERROR: ERROR Failed to get my own IPs? test-replicator ERROR: Can't find itself 127.0.0.1 with port 1000 in ring file, not replicating test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00041 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00089 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: Boom! test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00068 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 1 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: /srv/node/sdb is not mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_stats test.unit.common.test_db_replicator.TestDBReplicator.test_stats ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync test.unit.common.test_db_replicator.TestDBReplicator.test_usync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted ... ok d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" with 3.14159265359 weight got id 4 d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Partition power increase in progress. You need to finish the increase first before removing devices. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 test.test.com 6200 r.test.com 7000 sda3 3.14 2 33.67 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data No matching devices found swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info test.unit.cli.test_ringbuilder.TestCommands.test_set_info ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbh4cwf5z/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpbh4cwf5z/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345390.00000 test INFO: Object update sweep completed on /tmp/tmpbh4cwf5z/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpj07bh3py/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpj07bh3py/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345389.00000 test INFO: Object update sweep completed on /tmp/tmpj07bh3py/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpj07bh3py/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmpj07bh3py/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.00000 test INFO: Object update sweep completed on /tmp/tmpj07bh3py/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1dvccyb5/devices/sda1 (pid: 3541999) test ERROR: ERROR Pickle problem, quarantining /tmp/tmp1dvccyb5/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345388.00000: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/updater.py", line 431, in _load_update return pickle.load(open(update_path, 'rb')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _pickle.UnpicklingError: invalid load key, '\xff'. test INFO: Object update sweep completed on /tmp/tmp1dvccyb5/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp73bytb6u/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c1/o04 /tmp/tmp73bytb6u/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345388.54772 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmp73bytb6u/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717345388.54929 test INFO: Object update sweep completed on /tmp/tmp73bytb6u/devices/sda1 in 0.21s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.21s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpdofmm5eq/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c1/o01 /tmp/tmpdofmm5eq/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345388.76392 test DEBUG: Update sent for /a/c1/o03 /tmp/tmpdofmm5eq/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345388.76436 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpdofmm5eq/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345388.76414 test INFO: Object update sweep completed on /tmp/tmpdofmm5eq/devices/sda1 in 0.10s seconds:, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.10s, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpg7rk51a0/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c1/o00 /tmp/tmpg7rk51a0/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717345388.87345 test DEBUG: Update sent for /a/c1/o01 /tmp/tmpg7rk51a0/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345388.87372 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpg7rk51a0/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345388.87398 test DEBUG: Update sent for /a/c1/o04 /tmp/tmpg7rk51a0/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345388.87443 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals ... ok test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Setting object count to 1072 (txn: txfa8b277a8ff1494cb7f3d-00665c9c77) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx0db7a9c7bc434cf1a4936-00665c9c7a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx0db7a9c7bc434cf1a4936-00665c9c7a) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0db7a9c7bc434cf1a4936-00665c9c7a) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx0db7a9c7bc434cf1a4936-00665c9c7a) test ERROR: Aborting listing from shards due to bad response: [404] (txn: tx0db7a9c7bc434cf1a4936-00665c9c7a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: txce1b12456cce45d388316-00665c9c7d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test-expirer ERROR: Exception while deleting object .expiring_objects 1717345377 1717345377-a6/c6/o6 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345377 1717345377-a7/c7/o7 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345377 1717345377-a8/c8/o8♡ TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345377 1717345377-a9/c9/o9ø TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 3s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer ERROR: Unexcepted error handling task '1717258985-invalid0': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid0 test-expirer ERROR: Unexcepted error handling task '1717258985-invalid1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test-expirer INFO: Pass so far 0s; 0 objects expired test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0000', 'task_object': '0000-invalid0', 'delete_timestamp': 0000000000.00000_0000000000000000, 'target_path': 'invalid0', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0001', 'task_object': '0001-invalid1', 'delete_timestamp': 0000000001.00000_0000000000000000, 'target_path': 'invalid1', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0002', 'task_object': '0002-invalid2', 'delete_timestamp': 0000000002.00000_0000000000000000, 'target_path': 'invalid2', 'is_async_delete': False} test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 380, in run_forever self.run_once(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/test/unit/obj/test_expirer.py", line 907, in raise_exceptions raise Exception('exception %d' % raises[0]) Exception: exception 1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 330, in run_once self.swift.get_account_info(task_account) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'get_account_info' test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpitr8xd1m/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpitr8xd1m/tmp_test_proxy_server_chunked/container-sync-realms.conf' test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345363.27066', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345363.27066', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345363.27066', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345363.27099', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345363.27099', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345363.27082', 'epoch': '1717345363.27082', 'reported': 0, 'tombstones': -1}], 'b5263950-d4b8-49cb-84fb-45c4365deb6f-') test DEBUG: /tmp/tmpmveb3gei/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.27082.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpmveb3gei/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345363.27082.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '3ef4019f6008b9573b5a18f3371030ce', '355b9737-3ca9-43ad-9c79-1add6f86b8c9-', '1717345377.80855', '1717345377.80398', '0', '', '1717345377.80398', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('355b9737-3ca9-43ad-9c79-1add6f86b8c9-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717345377.84507.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '3ef4019f6008b9573b5a18f3371030ce', '355b9737-3ca9-43ad-9c79-1add6f86b8c9-', '1717345377.80855', '1717345377.80398', '0', '', '1717345377.80398', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345377.81856', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345377.81856', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345377.81856', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345377.84531', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345377.84531', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345377.84507', 'epoch': '1717345377.84507', 'reported': 0, 'tombstones': -1}], '355b9737-3ca9-43ad-9c79-1add6f86b8c9-') test DEBUG: /tmp/tmp7la37jo3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345377.84507.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp7la37jo3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345377.84507.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '3ef4019f6008b9573b5a18f3371030ce', 'f9dc744c-eb2a-4f96-9634-5bd1cabad7ab-', '1717345378.00680', '1717345377.99950', '0', '', '1717345377.99950', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f9dc744c-eb2a-4f96-9634-5bd1cabad7ab-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.03939.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '3ef4019f6008b9573b5a18f3371030ce', 'f9dc744c-eb2a-4f96-9634-5bd1cabad7ab-', '1717345378.00680', '1717345377.99950', '0', '', '1717345377.99950', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345378.02163', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345378.02163', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345378.02163', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345378.03968', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345378.03968', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345378.03939', 'epoch': '1717345378.03939', 'reported': 0, 'tombstones': -1}], 'f9dc744c-eb2a-4f96-9634-5bd1cabad7ab-') test DEBUG: /tmp/tmpryh5rbey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.03939.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpryh5rbey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.03939.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '0fadfc003944ede42ca15b07a7a99f27', '9ffb796b-0f30-4366-bfb9-312590ca7c96-', '1717345378.17368', '1717345378.16909', '0', '', '1717345378.16909', 3, 0) test WARNING: /tmp/tmp345xic09/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.19130.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmp345xic09/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.19130.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '0fadfc003944ede42ca15b07a7a99f27', '9ffb796b-0f30-4366-bfb9-312590ca7c96-', '1717345378.17368', '1717345378.25852', '0', '', '1717345378.16909', 3, 0) test WARNING: /tmp/tmp345xic09/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.19130.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmp345xic09/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.19130.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '0fadfc003944ede42ca15b07a7a99f27', 'a5debc5c-ecd2-4ca1-88e6-2952d20f45dd-', '1717345378.33538', '1717345378.33084', '0', '', '1717345378.33084', 3, 0) test WARNING: /tmp/tmpsdiasli7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.35117.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmpsdiasli7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.35117.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '0fadfc003944ede42ca15b07a7a99f27', 'a5debc5c-ecd2-4ca1-88e6-2952d20f45dd-', '1717345378.33538', '1717345378.41346', '0', '', '1717345378.33084', 3, 0) test WARNING: /tmp/tmpsdiasli7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.35117.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmpsdiasli7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.35117.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '2b0f841446b34e389129d8920f937fb6', 'c2c1efb7-9226-4f8b-b81a-81ab0b53a9b3-', '1717345378.48000', '1717345378.47496', '0', '', '1717345378.47496', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded ... ok test INFO: - - - [02/Jun/2024:16:22:42 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0194 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:55 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0138 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:22:55 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0029 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0104 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0095 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0049 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0147 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0118 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0168 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0077 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0194 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0132 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0040 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0037 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0036 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0039 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0257 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0005 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0242 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0114 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics ... ok DEBUG: New builder; performing initial balance New builder; performing initial balance DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 768 new parts and 0 removed parts from replica-count change 768 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 178 (Last start was 0) Gather start is 178 (Last start was 0) DEBUG: Gathered 768 parts Gathered 768 parts DEBUG: Placed 122/0 onto dev r1z1-10.1.1.1/sdb Placed 122/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 122/1 onto dev r1z1-10.1.1.1/sdc Placed 122/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 122/2 onto dev r1z1-10.1.1.1/sda Placed 122/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 10/0 onto dev r1z1-10.1.1.1/sdd Placed 10/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 10/1 onto dev r1z1-10.1.1.1/sdb Placed 10/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 10/2 onto dev r1z1-10.1.1.1/sdc Placed 10/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 125/0 onto dev r1z1-10.1.1.1/sda Placed 125/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 125/1 onto dev r1z1-10.1.1.1/sdd Placed 125/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 125/2 onto dev r1z1-10.1.1.1/sdb Placed 125/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 17/0 onto dev r1z1-10.1.1.1/sdc Placed 17/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 17/1 onto dev r1z1-10.1.1.1/sda Placed 17/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 17/2 onto dev r1z1-10.1.1.1/sdd Placed 17/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 195/0 onto dev r1z1-10.1.1.1/sdb Placed 195/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 195/1 onto dev r1z1-10.1.1.1/sdc Placed 195/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 195/2 onto dev r1z1-10.1.1.1/sda Placed 195/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 117/0 onto dev r1z1-10.1.1.1/sdd Placed 117/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 117/1 onto dev r1z1-10.1.1.1/sdb Placed 117/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 117/2 onto dev r1z1-10.1.1.1/sdc Placed 117/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 30/0 onto dev r1z1-10.1.1.1/sda Placed 30/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 30/1 onto dev r1z1-10.1.1.1/sdd Placed 30/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 30/2 onto dev r1z1-10.1.1.1/sdb Placed 30/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 250/0 onto dev r1z1-10.1.1.1/sdc Placed 250/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 250/1 onto dev r1z1-10.1.1.1/sda Placed 250/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 250/2 onto dev r1z1-10.1.1.1/sdd Placed 250/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 41/0 onto dev r1z1-10.1.1.1/sdb Placed 41/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 41/1 onto dev r1z1-10.1.1.1/sdc Placed 41/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 41/2 onto dev r1z1-10.1.1.1/sda Placed 41/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 227/0 onto dev r1z1-10.1.1.1/sdd Placed 227/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 227/1 onto dev r1z1-10.1.1.1/sdb Placed 227/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 227/2 onto dev r1z1-10.1.1.1/sdc Placed 227/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 22/0 onto dev r1z1-10.1.1.1/sda Placed 22/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 22/1 onto dev r1z1-10.1.1.1/sdd Placed 22/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 22/2 onto dev r1z1-10.1.1.1/sdb Placed 22/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 100/0 onto dev r1z1-10.1.1.1/sdc Placed 100/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 100/1 onto dev r1z1-10.1.1.1/sda Placed 100/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 100/2 onto dev r1z1-10.1.1.1/sdd Placed 100/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 159/0 onto dev r1z1-10.1.1.1/sdb Placed 159/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 159/1 onto dev r1z1-10.1.1.1/sdc Placed 159/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 159/2 onto dev r1z1-10.1.1.1/sda Placed 159/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 225/0 onto dev r1z1-10.1.1.1/sdd Placed 225/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 225/1 onto dev r1z1-10.1.1.1/sdb Placed 225/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 225/2 onto dev r1z1-10.1.1.1/sdc Placed 225/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 254/0 onto dev r1z1-10.1.1.1/sda Placed 254/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 254/1 onto dev r1z1-10.1.1.1/sdd Placed 254/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 254/2 onto dev r1z1-10.1.1.1/sdb Placed 254/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 206/0 onto dev r1z1-10.1.1.1/sdc Placed 206/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 206/1 onto dev r1z1-10.1.1.1/sda Placed 206/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 206/2 onto dev r1z1-10.1.1.1/sdd Placed 206/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 49/0 onto dev r1z1-10.1.1.1/sdb Placed 49/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 49/1 onto dev r1z1-10.1.1.1/sdc Placed 49/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 49/2 onto dev r1z1-10.1.1.1/sda Placed 49/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 194/0 onto dev r1z1-10.1.1.1/sdd Placed 194/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 194/1 onto dev r1z1-10.1.1.1/sdb Placed 194/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 194/2 onto dev r1z1-10.1.1.1/sdc Placed 194/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 198/0 onto dev r1z1-10.1.1.1/sda Placed 198/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 198/1 onto dev r1z1-10.1.1.1/sdd Placed 198/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 198/2 onto dev r1z1-10.1.1.1/sdb Placed 198/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 175/0 onto dev r1z1-10.1.1.1/sdc Placed 175/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 175/1 onto dev r1z1-10.1.1.1/sda Placed 175/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 175/2 onto dev r1z1-10.1.1.1/sdd Placed 175/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 199/0 onto dev r1z1-10.1.1.1/sdb Placed 199/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 199/1 onto dev r1z1-10.1.1.1/sdc Placed 199/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 199/2 onto dev r1z1-10.1.1.1/sda Placed 199/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 207/0 onto dev r1z1-10.1.1.1/sdd Placed 207/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 207/1 onto dev r1z1-10.1.1.1/sdb Placed 207/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 207/2 onto dev r1z1-10.1.1.1/sdc Placed 207/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 67/0 onto dev r1z1-10.1.1.1/sda Placed 67/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 67/1 onto dev r1z1-10.1.1.1/sdd Placed 67/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 67/2 onto dev r1z1-10.1.1.1/sdb Placed 67/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 21/0 onto dev r1z1-10.1.1.1/sdc Placed 21/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 21/1 onto dev r1z1-10.1.1.1/sda Placed 21/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 21/2 onto dev r1z1-10.1.1.1/sdd Placed 21/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 71/0 onto dev r1z1-10.1.1.1/sdb Placed 71/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 71/1 onto dev r1z1-10.1.1.1/sdc Placed 71/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 71/2 onto dev r1z1-10.1.1.1/sda Placed 71/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 96/0 onto dev r1z1-10.1.1.1/sdd Placed 96/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 96/1 onto dev r1z1-10.1.1.1/sdb Placed 96/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 96/2 onto dev r1z1-10.1.1.1/sdc Placed 96/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 70/0 onto dev r1z1-10.1.1.1/sda Placed 70/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 70/1 onto dev r1z1-10.1.1.1/sdd Placed 70/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 70/2 onto dev r1z1-10.1.1.1/sdb Placed 70/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 46/0 onto dev r1z1-10.1.1.1/sdc Placed 46/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 46/1 onto dev r1z1-10.1.1.1/sda Placed 46/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 46/2 onto dev r1z1-10.1.1.1/sdd Placed 46/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 35/0 onto dev r1z1-10.1.1.1/sdb Placed 35/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 35/1 onto dev r1z1-10.1.1.1/sdc Placed 35/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 35/2 onto dev r1z1-10.1.1.1/sda Placed 35/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 73/0 onto dev r1z1-10.1.1.1/sdd Placed 73/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 73/1 onto dev r1z1-10.1.1.1/sdb Placed 73/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 73/2 onto dev r1z1-10.1.1.1/sdc Placed 73/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 77/0 onto dev r1z1-10.1.1.1/sda Placed 77/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 77/1 onto dev r1z1-10.1.1.1/sdd Placed 77/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 77/2 onto dev r1z1-10.1.1.1/sdb Placed 77/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 164/0 onto dev r1z1-10.1.1.1/sdc Placed 164/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 164/1 onto dev r1z1-10.1.1.1/sda Placed 164/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 164/2 onto dev r1z1-10.1.1.1/sdd Placed 164/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 181/0 onto dev r1z1-10.1.1.1/sdb Placed 181/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 181/1 onto dev r1z1-10.1.1.1/sdc Placed 181/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 181/2 onto dev r1z1-10.1.1.1/sda Placed 181/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 56/0 onto dev r1z1-10.1.1.1/sdd Placed 56/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 56/1 onto dev r1z1-10.1.1.1/sdb Placed 56/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 56/2 onto dev r1z1-10.1.1.1/sdc Placed 56/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 251/0 onto dev r1z1-10.1.1.1/sda Placed 251/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 251/1 onto dev r1z1-10.1.1.1/sdd Placed 251/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 251/2 onto dev r1z1-10.1.1.1/sdb Placed 251/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 149/0 onto dev r1z1-10.1.1.1/sdc Placed 149/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 149/1 onto dev r1z1-10.1.1.1/sda Placed 149/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 149/2 onto dev r1z1-10.1.1.1/sdd Placed 149/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 120/0 onto dev r1z1-10.1.1.1/sdb Placed 120/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 120/1 onto dev r1z1-10.1.1.1/sdc Placed 120/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 120/2 onto dev r1z1-10.1.1.1/sda Placed 120/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 219/0 onto dev r1z1-10.1.1.1/sdd Placed 219/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 219/1 onto dev r1z1-10.1.1.1/sdb Placed 219/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 219/2 onto dev r1z1-10.1.1.1/sdc Placed 219/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 20/0 onto dev r1z1-10.1.1.1/sda Placed 20/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 20/1 onto dev r1z1-10.1.1.1/sdd Placed 20/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 20/2 onto dev r1z1-10.1.1.1/sdb Placed 20/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 176/0 onto dev r1z1-10.1.1.1/sdc Placed 176/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 176/1 onto dev r1z1-10.1.1.1/sda Placed 176/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 176/2 onto dev r1z1-10.1.1.1/sdd Placed 176/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 63/0 onto dev r1z1-10.1.1.1/sdb Placed 63/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 63/1 onto dev r1z1-10.1.1.1/sdc Placed 63/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 63/2 onto dev r1z1-10.1.1.1/sda Placed 63/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 5/0 onto dev r1z1-10.1.1.1/sdd Placed 5/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 5/1 onto dev r1z1-10.1.1.1/sdb Placed 5/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 5/2 onto dev r1z1-10.1.1.1/sdc Placed 5/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 234/0 onto dev r1z1-10.1.1.1/sda Placed 234/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 234/1 onto dev r1z1-10.1.1.1/sdd Placed 234/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 234/2 onto dev r1z1-10.1.1.1/sdb Placed 234/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 144/0 onto dev r1z1-10.1.1.1/sdc Placed 144/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 144/1 onto dev r1z1-10.1.1.1/sda Placed 144/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 144/2 onto dev r1z1-10.1.1.1/sdd Placed 144/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 170/0 onto dev r1z1-10.1.1.1/sdb Placed 170/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 170/1 onto dev r1z1-10.1.1.1/sdc Placed 170/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 170/2 onto dev r1z1-10.1.1.1/sda Placed 170/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 192/0 onto dev r1z1-10.1.1.1/sdd Placed 192/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 192/1 onto dev r1z1-10.1.1.1/sdb Placed 192/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 192/2 onto dev r1z1-10.1.1.1/sdc Placed 192/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 157/0 onto dev r1z1-10.1.1.1/sda Placed 157/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 157/1 onto dev r1z1-10.1.1.1/sdd Placed 157/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 157/2 onto dev r1z1-10.1.1.1/sdb Placed 157/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 214/0 onto dev r1z1-10.1.1.1/sdc Placed 214/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 214/1 onto dev r1z1-10.1.1.1/sda Placed 214/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 214/2 onto dev r1z1-10.1.1.1/sdd Placed 214/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 90/0 onto dev r1z1-10.1.1.1/sdb Placed 90/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 90/1 onto dev r1z1-10.1.1.1/sdc Placed 90/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 90/2 onto dev r1z1-10.1.1.1/sda Placed 90/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 218/0 onto dev r1z1-10.1.1.1/sdd Placed 218/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 218/1 onto dev r1z1-10.1.1.1/sdb Placed 218/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 218/2 onto dev r1z1-10.1.1.1/sdc Placed 218/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 87/0 onto dev r1z1-10.1.1.1/sda Placed 87/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 87/1 onto dev r1z1-10.1.1.1/sdd Placed 87/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 87/2 onto dev r1z1-10.1.1.1/sdb Placed 87/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 80/0 onto dev r1z1-10.1.1.1/sdc Placed 80/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 80/1 onto dev r1z1-10.1.1.1/sda Placed 80/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 80/2 onto dev r1z1-10.1.1.1/sdd Placed 80/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 42/0 onto dev r1z1-10.1.1.1/sdb Placed 42/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 42/1 onto dev r1z1-10.1.1.1/sdc Placed 42/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 42/2 onto dev r1z1-10.1.1.1/sda Placed 42/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 47/0 onto dev r1z1-10.1.1.1/sdd Placed 47/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 47/1 onto dev r1z1-10.1.1.1/sdb Placed 47/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 47/2 onto dev r1z1-10.1.1.1/sdc Placed 47/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 4/0 onto dev r1z1-10.1.1.1/sda Placed 4/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 4/1 onto dev r1z1-10.1.1.1/sdd Placed 4/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 4/2 onto dev r1z1-10.1.1.1/sdb Placed 4/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 60/0 onto dev r1z1-10.1.1.1/sdc Placed 60/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 60/1 onto dev r1z1-10.1.1.1/sda Placed 60/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 60/2 onto dev r1z1-10.1.1.1/sdd Placed 60/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 82/0 onto dev r1z1-10.1.1.1/sdb Placed 82/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 82/1 onto dev r1z1-10.1.1.1/sdc Placed 82/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 82/2 onto dev r1z1-10.1.1.1/sda Placed 82/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 187/0 onto dev r1z1-10.1.1.1/sdd Placed 187/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 187/1 onto dev r1z1-10.1.1.1/sdb Placed 187/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 187/2 onto dev r1z1-10.1.1.1/sdc Placed 187/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 253/0 onto dev r1z1-10.1.1.1/sda Placed 253/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 253/1 onto dev r1z1-10.1.1.1/sdd Placed 253/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 253/2 onto dev r1z1-10.1.1.1/sdb Placed 253/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 236/0 onto dev r1z1-10.1.1.1/sdc Placed 236/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 236/1 onto dev r1z1-10.1.1.1/sda Placed 236/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 236/2 onto dev r1z1-10.1.1.1/sdd Placed 236/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 54/0 onto dev r1z1-10.1.1.1/sdb Placed 54/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 54/1 onto dev r1z1-10.1.1.1/sdc Placed 54/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 54/2 onto dev r1z1-10.1.1.1/sda Placed 54/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 183/0 onto dev r1z1-10.1.1.1/sdd Placed 183/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 183/1 onto dev r1z1-10.1.1.1/sdb Placed 183/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 183/2 onto dev r1z1-10.1.1.1/sdc Placed 183/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 84/0 onto dev r1z1-10.1.1.1/sda Placed 84/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 84/1 onto dev r1z1-10.1.1.1/sdd Placed 84/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 84/2 onto dev r1z1-10.1.1.1/sdb Placed 84/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 1/0 onto dev r1z1-10.1.1.1/sdc Placed 1/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 1/1 onto dev r1z1-10.1.1.1/sda Placed 1/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 1/2 onto dev r1z1-10.1.1.1/sdd Placed 1/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 93/0 onto dev r1z1-10.1.1.1/sdb Placed 93/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 93/1 onto dev r1z1-10.1.1.1/sdc Placed 93/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 93/2 onto dev r1z1-10.1.1.1/sda Placed 93/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 147/0 onto dev r1z1-10.1.1.1/sdd Placed 147/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 147/1 onto dev r1z1-10.1.1.1/sdb Placed 147/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 147/2 onto dev r1z1-10.1.1.1/sdc Placed 147/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 131/0 onto dev r1z1-10.1.1.1/sda Placed 131/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 131/1 onto dev r1z1-10.1.1.1/sdd Placed 131/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 131/2 onto dev r1z1-10.1.1.1/sdb Placed 131/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 31/0 onto dev r1z1-10.1.1.1/sdc Placed 31/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 31/1 onto dev r1z1-10.1.1.1/sda Placed 31/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 31/2 onto dev r1z1-10.1.1.1/sdd Placed 31/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 150/0 onto dev r1z1-10.1.1.1/sdb Placed 150/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 150/1 onto dev r1z1-10.1.1.1/sdc Placed 150/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 150/2 onto dev r1z1-10.1.1.1/sda Placed 150/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 162/0 onto dev r1z1-10.1.1.1/sdd Placed 162/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 162/1 onto dev r1z1-10.1.1.1/sdb Placed 162/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 162/2 onto dev r1z1-10.1.1.1/sdc Placed 162/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 182/0 onto dev r1z1-10.1.1.1/sda Placed 182/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 182/1 onto dev r1z1-10.1.1.1/sdd Placed 182/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 182/2 onto dev r1z1-10.1.1.1/sdb Placed 182/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 221/0 onto dev r1z1-10.1.1.1/sdc Placed 221/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 221/1 onto dev r1z1-10.1.1.1/sda Placed 221/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 221/2 onto dev r1z1-10.1.1.1/sdd Placed 221/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 210/0 onto dev r1z1-10.1.1.1/sdb Placed 210/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 210/1 onto dev r1z1-10.1.1.1/sdc Placed 210/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 210/2 onto dev r1z1-10.1.1.1/sda Placed 210/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 145/0 onto dev r1z1-10.1.1.1/sdd Placed 145/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 145/1 onto dev r1z1-10.1.1.1/sdb Placed 145/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 145/2 onto dev r1z1-10.1.1.1/sdc Placed 145/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 215/0 onto dev r1z1-10.1.1.1/sda Placed 215/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 215/1 onto dev r1z1-10.1.1.1/sdd Placed 215/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 215/2 onto dev r1z1-10.1.1.1/sdb Placed 215/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 29/0 onto dev r1z1-10.1.1.1/sdc Placed 29/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 29/1 onto dev r1z1-10.1.1.1/sda Placed 29/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 29/2 onto dev r1z1-10.1.1.1/sdd Placed 29/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 180/0 onto dev r1z1-10.1.1.1/sdb Placed 180/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 180/1 onto dev r1z1-10.1.1.1/sdc Placed 180/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 180/2 onto dev r1z1-10.1.1.1/sda Placed 180/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 228/0 onto dev r1z1-10.1.1.1/sdd Placed 228/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 228/1 onto dev r1z1-10.1.1.1/sdb Placed 228/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 228/2 onto dev r1z1-10.1.1.1/sdc Placed 228/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 106/0 onto dev r1z1-10.1.1.1/sda Placed 106/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 106/1 onto dev r1z1-10.1.1.1/sdd Placed 106/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 106/2 onto dev r1z1-10.1.1.1/sdb Placed 106/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 255/0 onto dev r1z1-10.1.1.1/sdc Placed 255/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 255/1 onto dev r1z1-10.1.1.1/sda Placed 255/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 255/2 onto dev r1z1-10.1.1.1/sdd Placed 255/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 129/0 onto dev r1z1-10.1.1.1/sdb Placed 129/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 129/1 onto dev r1z1-10.1.1.1/sdc Placed 129/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 129/2 onto dev r1z1-10.1.1.1/sda Placed 129/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 252/0 onto dev r1z1-10.1.1.1/sdd Placed 252/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 252/1 onto dev r1z1-10.1.1.1/sdb Placed 252/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 252/2 onto dev r1z1-10.1.1.1/sdc Placed 252/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 40/0 onto dev r1z1-10.1.1.1/sda Placed 40/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 40/1 onto dev r1z1-10.1.1.1/sdd Placed 40/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 40/2 onto dev r1z1-10.1.1.1/sdb Placed 40/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 200/0 onto dev r1z1-10.1.1.1/sdc Placed 200/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 200/1 onto dev r1z1-10.1.1.1/sda Placed 200/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 200/2 onto dev r1z1-10.1.1.1/sdd Placed 200/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 240/0 onto dev r1z1-10.1.1.1/sdb Placed 240/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 240/1 onto dev r1z1-10.1.1.1/sdc Placed 240/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 240/2 onto dev r1z1-10.1.1.1/sda Placed 240/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 48/0 onto dev r1z1-10.1.1.1/sdd Placed 48/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 48/1 onto dev r1z1-10.1.1.1/sdb Placed 48/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 48/2 onto dev r1z1-10.1.1.1/sdc Placed 48/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 196/0 onto dev r1z1-10.1.1.1/sda Placed 196/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 196/1 onto dev r1z1-10.1.1.1/sdd Placed 196/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 196/2 onto dev r1z1-10.1.1.1/sdb Placed 196/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 202/0 onto dev r1z1-10.1.1.1/sdc Placed 202/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 202/1 onto dev r1z1-10.1.1.1/sda Placed 202/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 202/2 onto dev r1z1-10.1.1.1/sdd Placed 202/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 12/0 onto dev r1z1-10.1.1.1/sdb Placed 12/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 12/1 onto dev r1z1-10.1.1.1/sdc Placed 12/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 12/2 onto dev r1z1-10.1.1.1/sda Placed 12/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 6/0 onto dev r1z1-10.1.1.1/sdd Placed 6/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 6/1 onto dev r1z1-10.1.1.1/sdb Placed 6/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 6/2 onto dev r1z1-10.1.1.1/sdc Placed 6/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 15/0 onto dev r1z1-10.1.1.1/sda Placed 15/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 15/1 onto dev r1z1-10.1.1.1/sdd Placed 15/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 15/2 onto dev r1z1-10.1.1.1/sdb Placed 15/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 184/0 onto dev r1z1-10.1.1.1/sdc Placed 184/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 184/1 onto dev r1z1-10.1.1.1/sda Placed 184/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 184/2 onto dev r1z1-10.1.1.1/sdd Placed 184/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 26/0 onto dev r1z1-10.1.1.1/sdb Placed 26/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 26/1 onto dev r1z1-10.1.1.1/sdc Placed 26/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 26/2 onto dev r1z1-10.1.1.1/sda Placed 26/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 75/0 onto dev r1z1-10.1.1.1/sdd Placed 75/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 75/1 onto dev r1z1-10.1.1.1/sdb Placed 75/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 75/2 onto dev r1z1-10.1.1.1/sdc Placed 75/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 53/0 onto dev r1z1-10.1.1.1/sda Placed 53/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 53/1 onto dev r1z1-10.1.1.1/sdd Placed 53/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 53/2 onto dev r1z1-10.1.1.1/sdb Placed 53/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 64/0 onto dev r1z1-10.1.1.1/sdc Placed 64/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 64/1 onto dev r1z1-10.1.1.1/sda Placed 64/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 64/2 onto dev r1z1-10.1.1.1/sdd Placed 64/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 222/0 onto dev r1z1-10.1.1.1/sdb Placed 222/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 222/1 onto dev r1z1-10.1.1.1/sdc Placed 222/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 222/2 onto dev r1z1-10.1.1.1/sda Placed 222/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 172/0 onto dev r1z1-10.1.1.1/sdd Placed 172/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 172/1 onto dev r1z1-10.1.1.1/sdb Placed 172/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 172/2 onto dev r1z1-10.1.1.1/sdc Placed 172/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 173/0 onto dev r1z1-10.1.1.1/sda Placed 173/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 173/1 onto dev r1z1-10.1.1.1/sdd Placed 173/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 173/2 onto dev r1z1-10.1.1.1/sdb Placed 173/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 166/0 onto dev r1z1-10.1.1.1/sdc Placed 166/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 166/1 onto dev r1z1-10.1.1.1/sda Placed 166/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 166/2 onto dev r1z1-10.1.1.1/sdd Placed 166/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 61/0 onto dev r1z1-10.1.1.1/sdb Placed 61/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 61/1 onto dev r1z1-10.1.1.1/sdc Placed 61/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 61/2 onto dev r1z1-10.1.1.1/sda Placed 61/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 3/0 onto dev r1z1-10.1.1.1/sdd Placed 3/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 3/1 onto dev r1z1-10.1.1.1/sdb Placed 3/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 3/2 onto dev r1z1-10.1.1.1/sdc Placed 3/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 112/0 onto dev r1z1-10.1.1.1/sda Placed 112/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 112/1 onto dev r1z1-10.1.1.1/sdd Placed 112/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 112/2 onto dev r1z1-10.1.1.1/sdb Placed 112/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 213/0 onto dev r1z1-10.1.1.1/sdc Placed 213/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 213/1 onto dev r1z1-10.1.1.1/sda Placed 213/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 213/2 onto dev r1z1-10.1.1.1/sdd Placed 213/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 247/0 onto dev r1z1-10.1.1.1/sdb Placed 247/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 247/1 onto dev r1z1-10.1.1.1/sdc Placed 247/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 247/2 onto dev r1z1-10.1.1.1/sda Placed 247/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 224/0 onto dev r1z1-10.1.1.1/sdd Placed 224/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 224/1 onto dev r1z1-10.1.1.1/sdb Placed 224/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 224/2 onto dev r1z1-10.1.1.1/sdc Placed 224/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 92/0 onto dev r1z1-10.1.1.1/sda Placed 92/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 92/1 onto dev r1z1-10.1.1.1/sdd Placed 92/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 92/2 onto dev r1z1-10.1.1.1/sdb Placed 92/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 246/0 onto dev r1z1-10.1.1.1/sdc Placed 246/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 246/1 onto dev r1z1-10.1.1.1/sda Placed 246/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 246/2 onto dev r1z1-10.1.1.1/sdd Placed 246/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 230/0 onto dev r1z1-10.1.1.1/sdb Placed 230/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 230/1 onto dev r1z1-10.1.1.1/sdc Placed 230/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 230/2 onto dev r1z1-10.1.1.1/sda Placed 230/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 27/0 onto dev r1z1-10.1.1.1/sdd Placed 27/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 27/1 onto dev r1z1-10.1.1.1/sdb Placed 27/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 27/2 onto dev r1z1-10.1.1.1/sdc Placed 27/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 13/0 onto dev r1z1-10.1.1.1/sda Placed 13/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 13/1 onto dev r1z1-10.1.1.1/sdd Placed 13/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 13/2 onto dev r1z1-10.1.1.1/sdb Placed 13/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 104/0 onto dev r1z1-10.1.1.1/sdc Placed 104/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 104/1 onto dev r1z1-10.1.1.1/sda Placed 104/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 104/2 onto dev r1z1-10.1.1.1/sdd Placed 104/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 167/0 onto dev r1z1-10.1.1.1/sdb Placed 167/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 167/1 onto dev r1z1-10.1.1.1/sdc Placed 167/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 167/2 onto dev r1z1-10.1.1.1/sda Placed 167/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 141/0 onto dev r1z1-10.1.1.1/sdd Placed 141/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 141/1 onto dev r1z1-10.1.1.1/sdb Placed 141/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 141/2 onto dev r1z1-10.1.1.1/sdc Placed 141/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 57/0 onto dev r1z1-10.1.1.1/sda Placed 57/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 57/1 onto dev r1z1-10.1.1.1/sdd Placed 57/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 57/2 onto dev r1z1-10.1.1.1/sdb Placed 57/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 127/0 onto dev r1z1-10.1.1.1/sdc Placed 127/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 127/1 onto dev r1z1-10.1.1.1/sda Placed 127/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 127/2 onto dev r1z1-10.1.1.1/sdd Placed 127/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 249/0 onto dev r1z1-10.1.1.1/sdb Placed 249/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 249/1 onto dev r1z1-10.1.1.1/sdc Placed 249/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 249/2 onto dev r1z1-10.1.1.1/sda Placed 249/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 25/0 onto dev r1z1-10.1.1.1/sdd Placed 25/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 25/1 onto dev r1z1-10.1.1.1/sdb Placed 25/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 25/2 onto dev r1z1-10.1.1.1/sdc Placed 25/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 118/0 onto dev r1z1-10.1.1.1/sda Placed 118/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 118/1 onto dev r1z1-10.1.1.1/sdd Placed 118/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 118/2 onto dev r1z1-10.1.1.1/sdb Placed 118/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 109/0 onto dev r1z1-10.1.1.1/sdc Placed 109/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 109/1 onto dev r1z1-10.1.1.1/sda Placed 109/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 109/2 onto dev r1z1-10.1.1.1/sdd Placed 109/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 123/0 onto dev r1z1-10.1.1.1/sdb Placed 123/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 123/1 onto dev r1z1-10.1.1.1/sdc Placed 123/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 123/2 onto dev r1z1-10.1.1.1/sda Placed 123/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 223/0 onto dev r1z1-10.1.1.1/sdd Placed 223/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 223/1 onto dev r1z1-10.1.1.1/sdb Placed 223/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 223/2 onto dev r1z1-10.1.1.1/sdc Placed 223/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 33/0 onto dev r1z1-10.1.1.1/sda Placed 33/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 33/1 onto dev r1z1-10.1.1.1/sdd Placed 33/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 33/2 onto dev r1z1-10.1.1.1/sdb Placed 33/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 133/0 onto dev r1z1-10.1.1.1/sdc Placed 133/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 133/1 onto dev r1z1-10.1.1.1/sda Placed 133/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 133/2 onto dev r1z1-10.1.1.1/sdd Placed 133/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 101/0 onto dev r1z1-10.1.1.1/sdb Placed 101/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 101/1 onto dev r1z1-10.1.1.1/sdc Placed 101/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 101/2 onto dev r1z1-10.1.1.1/sda Placed 101/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 216/0 onto dev r1z1-10.1.1.1/sdd Placed 216/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 216/1 onto dev r1z1-10.1.1.1/sdb Placed 216/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 216/2 onto dev r1z1-10.1.1.1/sdc Placed 216/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 55/0 onto dev r1z1-10.1.1.1/sda Placed 55/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 55/1 onto dev r1z1-10.1.1.1/sdd Placed 55/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 55/2 onto dev r1z1-10.1.1.1/sdb Placed 55/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 66/0 onto dev r1z1-10.1.1.1/sdc Placed 66/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 66/1 onto dev r1z1-10.1.1.1/sda Placed 66/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 66/2 onto dev r1z1-10.1.1.1/sdd Placed 66/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 163/0 onto dev r1z1-10.1.1.1/sdb Placed 163/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 163/1 onto dev r1z1-10.1.1.1/sdc Placed 163/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 163/2 onto dev r1z1-10.1.1.1/sda Placed 163/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 51/0 onto dev r1z1-10.1.1.1/sdd Placed 51/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 51/1 onto dev r1z1-10.1.1.1/sdb Placed 51/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 51/2 onto dev r1z1-10.1.1.1/sdc Placed 51/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 243/0 onto dev r1z1-10.1.1.1/sda Placed 243/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 243/1 onto dev r1z1-10.1.1.1/sdd Placed 243/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 243/2 onto dev r1z1-10.1.1.1/sdb Placed 243/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 72/0 onto dev r1z1-10.1.1.1/sdc Placed 72/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 72/1 onto dev r1z1-10.1.1.1/sda Placed 72/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 72/2 onto dev r1z1-10.1.1.1/sdd Placed 72/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 113/0 onto dev r1z1-10.1.1.1/sdb Placed 113/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 113/1 onto dev r1z1-10.1.1.1/sdc Placed 113/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 113/2 onto dev r1z1-10.1.1.1/sda Placed 113/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 203/0 onto dev r1z1-10.1.1.1/sdd Placed 203/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 203/1 onto dev r1z1-10.1.1.1/sdb Placed 203/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 203/2 onto dev r1z1-10.1.1.1/sdc Placed 203/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 229/0 onto dev r1z1-10.1.1.1/sda Placed 229/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 229/1 onto dev r1z1-10.1.1.1/sdd Placed 229/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 229/2 onto dev r1z1-10.1.1.1/sdb Placed 229/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 235/0 onto dev r1z1-10.1.1.1/sdc Placed 235/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 235/1 onto dev r1z1-10.1.1.1/sda Placed 235/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 235/2 onto dev r1z1-10.1.1.1/sdd Placed 235/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 186/0 onto dev r1z1-10.1.1.1/sdb Placed 186/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 186/1 onto dev r1z1-10.1.1.1/sdc Placed 186/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 186/2 onto dev r1z1-10.1.1.1/sda Placed 186/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 233/0 onto dev r1z1-10.1.1.1/sdd Placed 233/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 233/1 onto dev r1z1-10.1.1.1/sdb Placed 233/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 233/2 onto dev r1z1-10.1.1.1/sdc Placed 233/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 69/0 onto dev r1z1-10.1.1.1/sda Placed 69/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 69/1 onto dev r1z1-10.1.1.1/sdd Placed 69/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 69/2 onto dev r1z1-10.1.1.1/sdb Placed 69/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 78/0 onto dev r1z1-10.1.1.1/sdc Placed 78/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 78/1 onto dev r1z1-10.1.1.1/sda Placed 78/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 78/2 onto dev r1z1-10.1.1.1/sdd Placed 78/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 152/0 onto dev r1z1-10.1.1.1/sdb Placed 152/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 152/1 onto dev r1z1-10.1.1.1/sdc Placed 152/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 152/2 onto dev r1z1-10.1.1.1/sda Placed 152/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 111/0 onto dev r1z1-10.1.1.1/sdd Placed 111/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 111/1 onto dev r1z1-10.1.1.1/sdb Placed 111/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 111/2 onto dev r1z1-10.1.1.1/sdc Placed 111/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 177/0 onto dev r1z1-10.1.1.1/sda Placed 177/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 177/1 onto dev r1z1-10.1.1.1/sdd Placed 177/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 177/2 onto dev r1z1-10.1.1.1/sdb Placed 177/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 185/0 onto dev r1z1-10.1.1.1/sdc Placed 185/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 185/1 onto dev r1z1-10.1.1.1/sda Placed 185/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 185/2 onto dev r1z1-10.1.1.1/sdd Placed 185/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 8/0 onto dev r1z1-10.1.1.1/sdb Placed 8/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 8/1 onto dev r1z1-10.1.1.1/sdc Placed 8/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 8/2 onto dev r1z1-10.1.1.1/sda Placed 8/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 146/0 onto dev r1z1-10.1.1.1/sdd Placed 146/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 146/1 onto dev r1z1-10.1.1.1/sdb Placed 146/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 146/2 onto dev r1z1-10.1.1.1/sdc Placed 146/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 76/0 onto dev r1z1-10.1.1.1/sda Placed 76/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 76/1 onto dev r1z1-10.1.1.1/sdd Placed 76/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 76/2 onto dev r1z1-10.1.1.1/sdb Placed 76/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 94/0 onto dev r1z1-10.1.1.1/sdc Placed 94/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 94/1 onto dev r1z1-10.1.1.1/sda Placed 94/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 94/2 onto dev r1z1-10.1.1.1/sdd Placed 94/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 248/0 onto dev r1z1-10.1.1.1/sdb Placed 248/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 248/1 onto dev r1z1-10.1.1.1/sdc Placed 248/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 248/2 onto dev r1z1-10.1.1.1/sda Placed 248/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 138/0 onto dev r1z1-10.1.1.1/sdd Placed 138/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 138/1 onto dev r1z1-10.1.1.1/sdb Placed 138/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 138/2 onto dev r1z1-10.1.1.1/sdc Placed 138/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 110/0 onto dev r1z1-10.1.1.1/sda Placed 110/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 110/1 onto dev r1z1-10.1.1.1/sdd Placed 110/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 110/2 onto dev r1z1-10.1.1.1/sdb Placed 110/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 211/0 onto dev r1z1-10.1.1.1/sdc Placed 211/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 211/1 onto dev r1z1-10.1.1.1/sda Placed 211/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 211/2 onto dev r1z1-10.1.1.1/sdd Placed 211/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 165/0 onto dev r1z1-10.1.1.1/sdb Placed 165/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 165/1 onto dev r1z1-10.1.1.1/sdc Placed 165/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 165/2 onto dev r1z1-10.1.1.1/sda Placed 165/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 204/0 onto dev r1z1-10.1.1.1/sdd Placed 204/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 204/1 onto dev r1z1-10.1.1.1/sdb Placed 204/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 204/2 onto dev r1z1-10.1.1.1/sdc Placed 204/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 140/0 onto dev r1z1-10.1.1.1/sda Placed 140/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 140/1 onto dev r1z1-10.1.1.1/sdd Placed 140/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 140/2 onto dev r1z1-10.1.1.1/sdb Placed 140/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 97/0 onto dev r1z1-10.1.1.1/sdc Placed 97/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 97/1 onto dev r1z1-10.1.1.1/sda Placed 97/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 97/2 onto dev r1z1-10.1.1.1/sdd Placed 97/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 45/0 onto dev r1z1-10.1.1.1/sdb Placed 45/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 45/1 onto dev r1z1-10.1.1.1/sdc Placed 45/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 45/2 onto dev r1z1-10.1.1.1/sda Placed 45/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 74/0 onto dev r1z1-10.1.1.1/sdd Placed 74/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 74/1 onto dev r1z1-10.1.1.1/sdb Placed 74/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 74/2 onto dev r1z1-10.1.1.1/sdc Placed 74/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 201/0 onto dev r1z1-10.1.1.1/sda Placed 201/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 201/1 onto dev r1z1-10.1.1.1/sdd Placed 201/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 201/2 onto dev r1z1-10.1.1.1/sdb Placed 201/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 68/0 onto dev r1z1-10.1.1.1/sdc Placed 68/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 68/1 onto dev r1z1-10.1.1.1/sda Placed 68/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 68/2 onto dev r1z1-10.1.1.1/sdd Placed 68/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 114/0 onto dev r1z1-10.1.1.1/sdb Placed 114/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 114/1 onto dev r1z1-10.1.1.1/sdc Placed 114/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 114/2 onto dev r1z1-10.1.1.1/sda Placed 114/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 168/0 onto dev r1z1-10.1.1.1/sdd Placed 168/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 168/1 onto dev r1z1-10.1.1.1/sdb Placed 168/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 168/2 onto dev r1z1-10.1.1.1/sdc Placed 168/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 2/0 onto dev r1z1-10.1.1.1/sda Placed 2/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 2/1 onto dev r1z1-10.1.1.1/sdd Placed 2/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 2/2 onto dev r1z1-10.1.1.1/sdb Placed 2/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 116/0 onto dev r1z1-10.1.1.1/sdc Placed 116/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 116/1 onto dev r1z1-10.1.1.1/sda Placed 116/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 116/2 onto dev r1z1-10.1.1.1/sdd Placed 116/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 81/0 onto dev r1z1-10.1.1.1/sdb Placed 81/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 81/1 onto dev r1z1-10.1.1.1/sdc Placed 81/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 81/2 onto dev r1z1-10.1.1.1/sda Placed 81/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 59/0 onto dev r1z1-10.1.1.1/sdd Placed 59/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 59/1 onto dev r1z1-10.1.1.1/sdb Placed 59/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 59/2 onto dev r1z1-10.1.1.1/sdc Placed 59/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 91/0 onto dev r1z1-10.1.1.1/sda Placed 91/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 91/1 onto dev r1z1-10.1.1.1/sdd Placed 91/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 91/2 onto dev r1z1-10.1.1.1/sdb Placed 91/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 24/0 onto dev r1z1-10.1.1.1/sdc Placed 24/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 24/1 onto dev r1z1-10.1.1.1/sda Placed 24/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 24/2 onto dev r1z1-10.1.1.1/sdd Placed 24/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 232/0 onto dev r1z1-10.1.1.1/sdb Placed 232/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 232/1 onto dev r1z1-10.1.1.1/sdc Placed 232/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 232/2 onto dev r1z1-10.1.1.1/sda Placed 232/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 190/0 onto dev r1z1-10.1.1.1/sdd Placed 190/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 190/1 onto dev r1z1-10.1.1.1/sdb Placed 190/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 190/2 onto dev r1z1-10.1.1.1/sdc Placed 190/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 34/0 onto dev r1z1-10.1.1.1/sda Placed 34/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 34/1 onto dev r1z1-10.1.1.1/sdd Placed 34/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 34/2 onto dev r1z1-10.1.1.1/sdb Placed 34/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 208/0 onto dev r1z1-10.1.1.1/sdc Placed 208/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 208/1 onto dev r1z1-10.1.1.1/sda Placed 208/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 208/2 onto dev r1z1-10.1.1.1/sdd Placed 208/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 244/0 onto dev r1z1-10.1.1.1/sdb Placed 244/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 244/1 onto dev r1z1-10.1.1.1/sdc Placed 244/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 244/2 onto dev r1z1-10.1.1.1/sda Placed 244/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 103/0 onto dev r1z1-10.1.1.1/sdd Placed 103/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 103/1 onto dev r1z1-10.1.1.1/sdb Placed 103/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 103/2 onto dev r1z1-10.1.1.1/sdc Placed 103/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 212/0 onto dev r1z1-10.1.1.1/sda Placed 212/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 212/1 onto dev r1z1-10.1.1.1/sdd Placed 212/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 212/2 onto dev r1z1-10.1.1.1/sdb Placed 212/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 50/0 onto dev r1z1-10.1.1.1/sdc Placed 50/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 50/1 onto dev r1z1-10.1.1.1/sda Placed 50/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 50/2 onto dev r1z1-10.1.1.1/sdd Placed 50/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 178/0 onto dev r1z1-10.1.1.1/sdb Placed 178/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 178/1 onto dev r1z1-10.1.1.1/sdc Placed 178/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 178/2 onto dev r1z1-10.1.1.1/sda Placed 178/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 197/0 onto dev r1z1-10.1.1.1/sdd Placed 197/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 197/1 onto dev r1z1-10.1.1.1/sdb Placed 197/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 197/2 onto dev r1z1-10.1.1.1/sdc Placed 197/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 65/0 onto dev r1z1-10.1.1.1/sda Placed 65/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 65/1 onto dev r1z1-10.1.1.1/sdd Placed 65/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 65/2 onto dev r1z1-10.1.1.1/sdb Placed 65/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 119/0 onto dev r1z1-10.1.1.1/sdc Placed 119/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 119/1 onto dev r1z1-10.1.1.1/sda Placed 119/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 119/2 onto dev r1z1-10.1.1.1/sdd Placed 119/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 0/0 onto dev r1z1-10.1.1.1/sdb Placed 0/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 0/1 onto dev r1z1-10.1.1.1/sdc Placed 0/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 0/2 onto dev r1z1-10.1.1.1/sda Placed 0/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 238/0 onto dev r1z1-10.1.1.1/sdd Placed 238/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 238/1 onto dev r1z1-10.1.1.1/sdb Placed 238/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 238/2 onto dev r1z1-10.1.1.1/sdc Placed 238/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 62/0 onto dev r1z1-10.1.1.1/sda Placed 62/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 62/1 onto dev r1z1-10.1.1.1/sdd Placed 62/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 62/2 onto dev r1z1-10.1.1.1/sdb Placed 62/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 83/0 onto dev r1z1-10.1.1.1/sdc Placed 83/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 83/1 onto dev r1z1-10.1.1.1/sda Placed 83/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 83/2 onto dev r1z1-10.1.1.1/sdd Placed 83/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 38/0 onto dev r1z1-10.1.1.1/sdb Placed 38/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 38/1 onto dev r1z1-10.1.1.1/sdc Placed 38/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 38/2 onto dev r1z1-10.1.1.1/sda Placed 38/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 132/0 onto dev r1z1-10.1.1.1/sdd Placed 132/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 132/1 onto dev r1z1-10.1.1.1/sdb Placed 132/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 132/2 onto dev r1z1-10.1.1.1/sdc Placed 132/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 148/0 onto dev r1z1-10.1.1.1/sda Placed 148/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 148/1 onto dev r1z1-10.1.1.1/sdd Placed 148/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 148/2 onto dev r1z1-10.1.1.1/sdb Placed 148/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 14/0 onto dev r1z1-10.1.1.1/sdc Placed 14/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 14/1 onto dev r1z1-10.1.1.1/sda Placed 14/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 14/2 onto dev r1z1-10.1.1.1/sdd Placed 14/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 39/0 onto dev r1z1-10.1.1.1/sdb Placed 39/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 39/1 onto dev r1z1-10.1.1.1/sdc Placed 39/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 39/2 onto dev r1z1-10.1.1.1/sda Placed 39/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 18/0 onto dev r1z1-10.1.1.1/sdd Placed 18/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 18/1 onto dev r1z1-10.1.1.1/sdb Placed 18/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 18/2 onto dev r1z1-10.1.1.1/sdc Placed 18/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 44/0 onto dev r1z1-10.1.1.1/sda Placed 44/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 44/1 onto dev r1z1-10.1.1.1/sdd Placed 44/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 44/2 onto dev r1z1-10.1.1.1/sdb Placed 44/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 142/0 onto dev r1z1-10.1.1.1/sdc Placed 142/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 142/1 onto dev r1z1-10.1.1.1/sda Placed 142/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 142/2 onto dev r1z1-10.1.1.1/sdd Placed 142/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 130/0 onto dev r1z1-10.1.1.1/sdb Placed 130/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 130/1 onto dev r1z1-10.1.1.1/sdc Placed 130/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 130/2 onto dev r1z1-10.1.1.1/sda Placed 130/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 160/0 onto dev r1z1-10.1.1.1/sdd Placed 160/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 160/1 onto dev r1z1-10.1.1.1/sdb Placed 160/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 160/2 onto dev r1z1-10.1.1.1/sdc Placed 160/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 217/0 onto dev r1z1-10.1.1.1/sda Placed 217/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 217/1 onto dev r1z1-10.1.1.1/sdd Placed 217/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 217/2 onto dev r1z1-10.1.1.1/sdb Placed 217/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 99/0 onto dev r1z1-10.1.1.1/sdc Placed 99/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 99/1 onto dev r1z1-10.1.1.1/sda Placed 99/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 99/2 onto dev r1z1-10.1.1.1/sdd Placed 99/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 161/0 onto dev r1z1-10.1.1.1/sdb Placed 161/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 161/1 onto dev r1z1-10.1.1.1/sdc Placed 161/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 161/2 onto dev r1z1-10.1.1.1/sda Placed 161/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 115/0 onto dev r1z1-10.1.1.1/sdd Placed 115/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 115/1 onto dev r1z1-10.1.1.1/sdb Placed 115/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 115/2 onto dev r1z1-10.1.1.1/sdc Placed 115/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 32/0 onto dev r1z1-10.1.1.1/sda Placed 32/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 32/1 onto dev r1z1-10.1.1.1/sdd Placed 32/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 32/2 onto dev r1z1-10.1.1.1/sdb Placed 32/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 9/0 onto dev r1z1-10.1.1.1/sdc Placed 9/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 9/1 onto dev r1z1-10.1.1.1/sda Placed 9/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 9/2 onto dev r1z1-10.1.1.1/sdd Placed 9/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 143/0 onto dev r1z1-10.1.1.1/sdb Placed 143/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 143/1 onto dev r1z1-10.1.1.1/sdc Placed 143/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 143/2 onto dev r1z1-10.1.1.1/sda Placed 143/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 136/0 onto dev r1z1-10.1.1.1/sdd Placed 136/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 136/1 onto dev r1z1-10.1.1.1/sdb Placed 136/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 136/2 onto dev r1z1-10.1.1.1/sdc Placed 136/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 102/0 onto dev r1z1-10.1.1.1/sda Placed 102/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 102/1 onto dev r1z1-10.1.1.1/sdd Placed 102/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 102/2 onto dev r1z1-10.1.1.1/sdb Placed 102/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 23/0 onto dev r1z1-10.1.1.1/sdc Placed 23/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 23/1 onto dev r1z1-10.1.1.1/sda Placed 23/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 23/2 onto dev r1z1-10.1.1.1/sdd Placed 23/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 154/0 onto dev r1z1-10.1.1.1/sdb Placed 154/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 154/1 onto dev r1z1-10.1.1.1/sdc Placed 154/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 154/2 onto dev r1z1-10.1.1.1/sda Placed 154/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 242/0 onto dev r1z1-10.1.1.1/sdd Placed 242/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 242/1 onto dev r1z1-10.1.1.1/sdb Placed 242/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 242/2 onto dev r1z1-10.1.1.1/sdc Placed 242/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 191/0 onto dev r1z1-10.1.1.1/sda Placed 191/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 191/1 onto dev r1z1-10.1.1.1/sdd Placed 191/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 191/2 onto dev r1z1-10.1.1.1/sdb Placed 191/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 107/0 onto dev r1z1-10.1.1.1/sdc Placed 107/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 107/1 onto dev r1z1-10.1.1.1/sda Placed 107/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 107/2 onto dev r1z1-10.1.1.1/sdd Placed 107/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 134/0 onto dev r1z1-10.1.1.1/sdb Placed 134/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 134/1 onto dev r1z1-10.1.1.1/sdc Placed 134/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 134/2 onto dev r1z1-10.1.1.1/sda Placed 134/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 169/0 onto dev r1z1-10.1.1.1/sdd Placed 169/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 169/1 onto dev r1z1-10.1.1.1/sdb Placed 169/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 169/2 onto dev r1z1-10.1.1.1/sdc Placed 169/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 237/0 onto dev r1z1-10.1.1.1/sda Placed 237/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 237/1 onto dev r1z1-10.1.1.1/sdd Placed 237/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 237/2 onto dev r1z1-10.1.1.1/sdb Placed 237/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 16/0 onto dev r1z1-10.1.1.1/sdc Placed 16/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 16/1 onto dev r1z1-10.1.1.1/sda Placed 16/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 16/2 onto dev r1z1-10.1.1.1/sdd Placed 16/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 19/0 onto dev r1z1-10.1.1.1/sdb Placed 19/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 19/1 onto dev r1z1-10.1.1.1/sdc Placed 19/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 19/2 onto dev r1z1-10.1.1.1/sda Placed 19/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 135/0 onto dev r1z1-10.1.1.1/sdd Placed 135/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 135/1 onto dev r1z1-10.1.1.1/sdb Placed 135/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 135/2 onto dev r1z1-10.1.1.1/sdc Placed 135/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 193/0 onto dev r1z1-10.1.1.1/sda Placed 193/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 193/1 onto dev r1z1-10.1.1.1/sdd Placed 193/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 193/2 onto dev r1z1-10.1.1.1/sdb Placed 193/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 121/0 onto dev r1z1-10.1.1.1/sdc Placed 121/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 121/1 onto dev r1z1-10.1.1.1/sda Placed 121/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 121/2 onto dev r1z1-10.1.1.1/sdd Placed 121/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 52/0 onto dev r1z1-10.1.1.1/sdb Placed 52/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 52/1 onto dev r1z1-10.1.1.1/sdc Placed 52/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 52/2 onto dev r1z1-10.1.1.1/sda Placed 52/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 95/0 onto dev r1z1-10.1.1.1/sdd Placed 95/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 95/1 onto dev r1z1-10.1.1.1/sdb Placed 95/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 95/2 onto dev r1z1-10.1.1.1/sdc Placed 95/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 171/0 onto dev r1z1-10.1.1.1/sda Placed 171/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 171/1 onto dev r1z1-10.1.1.1/sdd Placed 171/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 171/2 onto dev r1z1-10.1.1.1/sdb Placed 171/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 7/0 onto dev r1z1-10.1.1.1/sdc Placed 7/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 7/1 onto dev r1z1-10.1.1.1/sda Placed 7/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 7/2 onto dev r1z1-10.1.1.1/sdd Placed 7/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 79/0 onto dev r1z1-10.1.1.1/sdb Placed 79/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 79/1 onto dev r1z1-10.1.1.1/sdc Placed 79/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 79/2 onto dev r1z1-10.1.1.1/sda Placed 79/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 139/0 onto dev r1z1-10.1.1.1/sdd Placed 139/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 139/1 onto dev r1z1-10.1.1.1/sdb Placed 139/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 139/2 onto dev r1z1-10.1.1.1/sdc Placed 139/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 156/0 onto dev r1z1-10.1.1.1/sda Placed 156/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 156/1 onto dev r1z1-10.1.1.1/sdd Placed 156/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 156/2 onto dev r1z1-10.1.1.1/sdb Placed 156/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 128/0 onto dev r1z1-10.1.1.1/sdc Placed 128/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 128/1 onto dev r1z1-10.1.1.1/sda Placed 128/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 128/2 onto dev r1z1-10.1.1.1/sdd Placed 128/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 239/0 onto dev r1z1-10.1.1.1/sdb Placed 239/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 239/1 onto dev r1z1-10.1.1.1/sdc Placed 239/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 239/2 onto dev r1z1-10.1.1.1/sda Placed 239/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 124/0 onto dev r1z1-10.1.1.1/sdd Placed 124/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 124/1 onto dev r1z1-10.1.1.1/sdb Placed 124/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 124/2 onto dev r1z1-10.1.1.1/sdc Placed 124/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 241/0 onto dev r1z1-10.1.1.1/sda Placed 241/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 241/1 onto dev r1z1-10.1.1.1/sdd Placed 241/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 241/2 onto dev r1z1-10.1.1.1/sdb Placed 241/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 126/0 onto dev r1z1-10.1.1.1/sdc Placed 126/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 126/1 onto dev r1z1-10.1.1.1/sda Placed 126/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 126/2 onto dev r1z1-10.1.1.1/sdd Placed 126/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 108/0 onto dev r1z1-10.1.1.1/sdb Placed 108/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 108/1 onto dev r1z1-10.1.1.1/sdc Placed 108/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 108/2 onto dev r1z1-10.1.1.1/sda Placed 108/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 153/0 onto dev r1z1-10.1.1.1/sdd Placed 153/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 153/1 onto dev r1z1-10.1.1.1/sdb Placed 153/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 153/2 onto dev r1z1-10.1.1.1/sdc Placed 153/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 151/0 onto dev r1z1-10.1.1.1/sda Placed 151/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 151/1 onto dev r1z1-10.1.1.1/sdd Placed 151/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 151/2 onto dev r1z1-10.1.1.1/sdb Placed 151/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 137/0 onto dev r1z1-10.1.1.1/sdc Placed 137/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 137/1 onto dev r1z1-10.1.1.1/sda Placed 137/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 137/2 onto dev r1z1-10.1.1.1/sdd Placed 137/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 188/0 onto dev r1z1-10.1.1.1/sdb Placed 188/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 188/1 onto dev r1z1-10.1.1.1/sdc Placed 188/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 188/2 onto dev r1z1-10.1.1.1/sda Placed 188/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 88/0 onto dev r1z1-10.1.1.1/sdd Placed 88/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 88/1 onto dev r1z1-10.1.1.1/sdb Placed 88/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 88/2 onto dev r1z1-10.1.1.1/sdc Placed 88/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 231/0 onto dev r1z1-10.1.1.1/sda Placed 231/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 231/1 onto dev r1z1-10.1.1.1/sdd Placed 231/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 231/2 onto dev r1z1-10.1.1.1/sdb Placed 231/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 37/0 onto dev r1z1-10.1.1.1/sdc Placed 37/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 37/1 onto dev r1z1-10.1.1.1/sda Placed 37/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 37/2 onto dev r1z1-10.1.1.1/sdd Placed 37/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 11/0 onto dev r1z1-10.1.1.1/sdb Placed 11/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 11/1 onto dev r1z1-10.1.1.1/sdc Placed 11/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 11/2 onto dev r1z1-10.1.1.1/sda Placed 11/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 189/0 onto dev r1z1-10.1.1.1/sdd Placed 189/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 189/1 onto dev r1z1-10.1.1.1/sdb Placed 189/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 189/2 onto dev r1z1-10.1.1.1/sdc Placed 189/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 28/0 onto dev r1z1-10.1.1.1/sda Placed 28/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 28/1 onto dev r1z1-10.1.1.1/sdd Placed 28/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 28/2 onto dev r1z1-10.1.1.1/sdb Placed 28/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 36/0 onto dev r1z1-10.1.1.1/sdc Placed 36/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 36/1 onto dev r1z1-10.1.1.1/sda Placed 36/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 36/2 onto dev r1z1-10.1.1.1/sdd Placed 36/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 220/0 onto dev r1z1-10.1.1.1/sdb Placed 220/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 220/1 onto dev r1z1-10.1.1.1/sdc Placed 220/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 220/2 onto dev r1z1-10.1.1.1/sda Placed 220/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 43/0 onto dev r1z1-10.1.1.1/sdd Placed 43/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 43/1 onto dev r1z1-10.1.1.1/sdb Placed 43/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 43/2 onto dev r1z1-10.1.1.1/sdc Placed 43/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 86/0 onto dev r1z1-10.1.1.1/sda Placed 86/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 86/1 onto dev r1z1-10.1.1.1/sdd Placed 86/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 86/2 onto dev r1z1-10.1.1.1/sdb Placed 86/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 158/0 onto dev r1z1-10.1.1.1/sdc Placed 158/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 158/1 onto dev r1z1-10.1.1.1/sda Placed 158/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 158/2 onto dev r1z1-10.1.1.1/sdd Placed 158/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 245/0 onto dev r1z1-10.1.1.1/sdb Placed 245/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 245/1 onto dev r1z1-10.1.1.1/sdc Placed 245/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 245/2 onto dev r1z1-10.1.1.1/sda Placed 245/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 174/0 onto dev r1z1-10.1.1.1/sdd Placed 174/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 174/1 onto dev r1z1-10.1.1.1/sdb Placed 174/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 174/2 onto dev r1z1-10.1.1.1/sdc Placed 174/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 205/0 onto dev r1z1-10.1.1.1/sda Placed 205/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 205/1 onto dev r1z1-10.1.1.1/sdd Placed 205/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 205/2 onto dev r1z1-10.1.1.1/sdb Placed 205/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 209/0 onto dev r1z1-10.1.1.1/sdc Placed 209/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 209/1 onto dev r1z1-10.1.1.1/sda Placed 209/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 209/2 onto dev r1z1-10.1.1.1/sdd Placed 209/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 226/0 onto dev r1z1-10.1.1.1/sdb Placed 226/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 226/1 onto dev r1z1-10.1.1.1/sdc Placed 226/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 226/2 onto dev r1z1-10.1.1.1/sda Placed 226/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 155/0 onto dev r1z1-10.1.1.1/sdd Placed 155/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 155/1 onto dev r1z1-10.1.1.1/sdb Placed 155/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 155/2 onto dev r1z1-10.1.1.1/sdc Placed 155/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 179/0 onto dev r1z1-10.1.1.1/sda Placed 179/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 179/1 onto dev r1z1-10.1.1.1/sdd Placed 179/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 179/2 onto dev r1z1-10.1.1.1/sdb Placed 179/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 85/0 onto dev r1z1-10.1.1.1/sdc Placed 85/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 85/1 onto dev r1z1-10.1.1.1/sda Placed 85/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 85/2 onto dev r1z1-10.1.1.1/sdd Placed 85/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 89/0 onto dev r1z1-10.1.1.1/sdb Placed 89/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 89/1 onto dev r1z1-10.1.1.1/sdc Placed 89/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 89/2 onto dev r1z1-10.1.1.1/sda Placed 89/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 58/0 onto dev r1z1-10.1.1.1/sdd Placed 58/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 58/1 onto dev r1z1-10.1.1.1/sdb Placed 58/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 58/2 onto dev r1z1-10.1.1.1/sdc Placed 58/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 105/0 onto dev r1z1-10.1.1.1/sda Placed 105/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 105/1 onto dev r1z1-10.1.1.1/sdd Placed 105/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 105/2 onto dev r1z1-10.1.1.1/sdb Placed 105/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 98/0 onto dev r1z1-10.1.1.1/sdc Placed 98/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 98/1 onto dev r1z1-10.1.1.1/sda Placed 98/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 98/2 onto dev r1z1-10.1.1.1/sdd Placed 98/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Assigned 768 parts Assigned 768 parts DEBUG: Finished rebalance plan after 1 attempts Finished rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 4 (Last start was 178) DEBUG: Gather start is 4 (Last start was 178) Gather start is 4 (Last start was 178) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 4 (Last start was 178) DEBUG: Gather start is 4 (Last start was 178) DEBUG: Gather start is 4 (Last start was 178) Gather start is 4 (Last start was 178) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload ... ok test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '462'} test DEBUG: Grant test:full_control FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '446'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options test.unit.proxy.controllers.test_base.TestFuncs.test_options ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT ... ok test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 7.16898 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 6 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 0 dbs in 0.00000 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 30 dbs in 246.81358 seconds (0.12155/s) test-replicator INFO: Removed 9 dbs test-replicator INFO: 25 successes, 1 failures test-replicator INFO: diff:5 diff_capped:4 empty:7 hashmatch:8 no_change:6 remote_merge:2 rsync:3 ts_repl:10 direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:24] "GET https://cluster.domain/bucket/key HTTP/1.0" 200 106 0.000084 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:24] "GET http://cluster.domain/v1/acct/cont/obj HTTP/1.0" 200 112 0.000071 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:24] "GET ftp://cluster.domain/bucket/key HTTP/1.0" 200 126 0.000064 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:24] "GET https://cluster.domain HTTP/1.0" 200 117 0.000070 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:24] "GET http:omg//wtf/bbq HTTP/1.0" 200 112 0.000062 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: tx8395d0b33967471099e7f-00665c9c7c) proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) test ERROR: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD') (txn: test-trans-id) proxy INFO: wsgi exiting (txn: test-trans-id) proxy INFO: (3541997) wsgi exited, is_accepting=True (txn: test-trans-id) proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: test-trans-id) proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "PUT /path HTTP/1.0" 200 105 0.000128 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET ///some-leading-slashes HTTP/1.0" 200 118 0.000108 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET /ohÿboy%what$now%E2%80%bd HTTP/1.0" 200 120 0.000099 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET /some%fFpath%D8%AA HTTP/1.0" 200 113 0.000105 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET / HTTP/1.1" 200 120 0.000113 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 44\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET /%FF HTTP/1.1" 200 123 0.000038 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 47\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "/%FF", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET /ÿ HTTP/1.1" 200 126 0.000097 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 50\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "/\\u00ff", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "PUT /Here%20Is%20A%20SnowMan:☃ HTTP/1.0" 200 181 0.000157 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 86\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\nConnection: close\r\n\r\n{"RAW_PATH_INFO": "/Here%20Is%20A%20SnowMan:\\u00e2\\u0098\\u0083", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "POST /?and%20it=does+nothing+to+params&PALMTREE=ð%9fŒ%b4 HTTP/1.1" 200 178 0.000089 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 101\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": "and%20it=does+nothing+to+params&PALMTREE=\\u00f0%9f\\u008c%b4"}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET // HTTP/1.1" 200 121 0.000095 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 45\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "//", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET //bar HTTP/1.1" 200 124 0.000110 proxy INFO: wsgi exiting test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple ... ok test.unit.common.test_splice.TestTee.test_available test.unit.common.test_splice.TestTee.test_available ... ok test.unit.common.test_splice.TestTee.test_errno test.unit.common.test_splice.TestTee.test_errno ... ok test INFO: Object update sweep completed on /tmp/tmpg7rk51a0/devices/sda1 in 0.01s seconds:, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 2.57s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpjzcpvsab/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c1/o03 /tmp/tmpjzcpvsab/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345391.49876 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpjzcpvsab/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345391.49857 test INFO: Object update sweep completed on /tmp/tmpjzcpvsab/devices/sda1 in 0.11s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.11s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7qmxlz8j/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c1/o08 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/5b8/4e594cf882aac2f7f8f2f4cdade285b8-1717345391.61627 test DEBUG: Update sent for /a/c1/o06 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717345391.61449 test DEBUG: Update sent for /a/c1/o02 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345391.61379 test DEBUG: Update sent for /a/c1/o03 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345391.61397 test DEBUG: Update sent for /a/c1/o07 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/fc8/26ac197975ac1169d403eeb7ae8bbfc8-1717345391.61599 test DEBUG: Update sent for /a/c1/o09 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/bee/db77f283b4b1bb5e5b6ce501ca337bee-1717345391.61647 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717345391.61332 test DEBUG: Update sent for /a/c1/o01 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345391.61355 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717345391.61760 test DEBUG: Update sent for /a/c1/o05 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/dc4/017ff741f591e7220e0836a0c4fcedc4-1717345391.61432 test DEBUG: Update sent for /a/c1/o04 /tmp/tmp7qmxlz8j/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345391.61415 test INFO: Object update sweep completed on /tmp/tmp7qmxlz8j/devices/sda1 in 0.21s seconds:, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.21s, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpqvblpwuk/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c1/o06 /tmp/tmpqvblpwuk/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717345391.83228 test DEBUG: Update sent for /a/c1/o03 /tmp/tmpqvblpwuk/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345391.83179 test DEBUG: Update sent for /a/c1/o01 /tmp/tmpqvblpwuk/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345391.83144 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpqvblpwuk/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345391.83162 test DEBUG: Update sent for /a/c1/o04 /tmp/tmpqvblpwuk/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345391.83196 test INFO: Object update sweep completed on /tmp/tmpqvblpwuk/devices/sda1 in 0.01s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test DEBUG: Update failed for /a/c/o /tmp/tmph3xkdblz/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.85521 test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test DEBUG: Error code 500 is returned from remote server 127.0.0.1:51167/sda1 test DEBUG: Error code 500 is returned from remote server 127.0.0.1:51167/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmph3xkdblz/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.85521 test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test DEBUG: Error code 404 is returned from remote server 127.0.0.1:51167/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmph3xkdblz/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.85521 test.unit.obj.test_updater.TestObjectUpdater.test_run_once test.unit.obj.test_updater.TestObjectUpdater.test_run_once ... ok test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies ... ok proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 48\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "//bar", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:29] "GET //////baz HTTP/1.1" 200 128 0.000056 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 52\r\nDate: Sun, 02 Jun 2024 16:23:29 GMT\r\n\r\n{"RAW_PATH_INFO": "//////baz", "QUERY_STRING": null}' proxy INFO: (3541997) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3541997) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:23:33] "GET /someurl HTTP/1.0" 200 102 0.000096 proxy INFO: wsgi exiting proxy INFO: (3541997) wsgi exited, is_accepting=True Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor test.unit.common.test_splice.TestTee.test_fileno test.unit.common.test_splice.TestTee.test_fileno ... ok test.unit.common.test_splice.TestTee.test_flags_list test.unit.common.test_splice.TestTee.test_flags_list ... ok test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe ... ok test.unit.common.test_splice.TestTee.test_unavailable test.unit.common.test_splice.TestTee.test_unavailable ... ok test.unit.common.test_splice.TestTee.test_unavailable_in_libc test.unit.common.test_splice.TestTee.test_unavailable_in_libc ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_filters test.unit.common.test_utils.TestAuditLocationGenerator.test_filters ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive ... ok test.unit.common.test_utils.TestPunchHole.test_arg_bounds test.unit.common.test_utils.TestPunchHole.test_arg_bounds ... ok test.unit.common.test_utils.TestPunchHole.test_error test.unit.common.test_utils.TestPunchHole.test_error ... ok test.unit.common.test_utils.TestPunchHole.test_no_fallocate test.unit.common.test_utils.TestPunchHole.test_no_fallocate ... ok test.unit.common.test_utils.TestPunchHole.test_punch_hole test.unit.common.test_utils.TestPunchHole.test_punch_hole ... ok test.unit.common.test_utils.TestWatchdog.test_start_stop test.unit.common.test_utils.TestWatchdog.test_start_stop ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_expire test.unit.common.test_utils.TestWatchdog.test_timeout_expire ... ok test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim ... ok cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpitr8xd1m/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpitr8xd1m/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0172 "-" 3542021 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0143 "-" 3542021 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0121 "-" 3542021 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0125 "-" 3542021 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx44b94246d1784c5c9c522-00665c9c72" "proxy-server 3542021" 0.0023 "-" 3542021 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:58635/sdb1/1/a/c" "tx44b94246d1784c5c9c522-00665c9c72" "container-server 3542021" 0.0022 "-" 3542021 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:48979/sda1/1/a/c" "tx44b94246d1784c5c9c522-00665c9c72" "container-server 3542021" 0.0030 "-" 3542021 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx44b94246d1784c5c9c522-00665c9c72" "proxy-server 3542021" 0.0258 "-" 3542021 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx44b94246d1784c5c9c522-00665c9c72" "proxy-server 3542021" 0.0416 "-" 3542021 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx44b94246d1784c5c9c522-00665c9c72 - 0.0557 - - 1717345394.542291641 1717345394.597942114 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx033ccc2931244b1ab2596-00665c9c72" "proxy-server 3542021" 0.0021 "-" 3542021 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:48979/sda1/1/a1/c1" "tx033ccc2931244b1ab2596-00665c9c72" "container-server 3542021" 0.0018 "-" 3542021 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:58635/sdb1/1/a1/c1" "tx033ccc2931244b1ab2596-00665c9c72" "container-server 3542021" 0.0017 "-" 3542021 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx033ccc2931244b1ab2596-00665c9c72" "proxy-server 3542021" 0.0388 "-" 3542021 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx033ccc2931244b1ab2596-00665c9c72" "proxy-server 3542021" 0.0239 "-" 3542021 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx033ccc2931244b1ab2596-00665c9c72 - 0.0532 - - 1717345394.600425959 1717345394.653582811 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx78e78692a9a74e408cd70-00665c9c72" "proxy-server 3542021" 0.0130 "-" 3542021 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:48979/sda1/2/a/c1" "tx78e78692a9a74e408cd70-00665c9c72" "container-server 3542021" 0.0039 "-" 3542021 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:58635/sdb1/2/a/c1" "tx78e78692a9a74e408cd70-00665c9c72" "container-server 3542021" 0.0090 "-" 3542021 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx78e78692a9a74e408cd70-00665c9c72" "proxy-server 3542021" 0.0460 "-" 3542021 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx78e78692a9a74e408cd70-00665c9c72" "proxy-server 3542021" 0.0324 "-" 3542021 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx78e78692a9a74e408cd70-00665c9c72 - 0.0712 - - 1717345394.655946970 1717345394.727171183 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx684b83d0772d48e8b299d-00665c9c72" "proxy-server 3542021" 0.0117 "-" 3542021 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:48979/sda1/1/a/c2" "tx684b83d0772d48e8b299d-00665c9c72" "container-server 3542021" 0.0105 "-" 3542021 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:58635/sdb1/1/a/c2" "tx684b83d0772d48e8b299d-00665c9c72" "container-server 3542021" 0.0019 "-" 3542021 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx684b83d0772d48e8b299d-00665c9c72" "proxy-server 3542021" 0.0452 "-" 3542021 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:14 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx684b83d0772d48e8b299d-00665c9c72" "proxy-server 3542021" 0.0317 "-" 3542021 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/14 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx684b83d0772d48e8b299d-00665c9c72 - 0.0689 - - 1717345394.729746342 1717345394.798597574 2 test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you ... ok test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"something" Device d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" is now d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"something" Device d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" is now d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"something" Device d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" is now d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"something" Device 1 already uses 127.0.0.2:6201/sda2. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" is now d4r2z3-test.test2.com:9000Rr.test2.com:9000/sda300_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.2.1:9000R127.0.2.1:9000/sda100_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.1.1:8000/sda10_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[4001:0:1234::c1c0:abcd:876]:9000R[4::10]:9000/sda300_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:8000/sda30_"other meta data" Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The minimum number of hours before a partition can be reassigned is now set to 24 test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload test.unit.cli.test_ringbuilder.TestCommands.test_set_overload ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region test.unit.cli.test_ringbuilder.TestCommands.test_set_region ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight test.unit.cli.test_ringbuilder.TestCommands.test_set_weight ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345378.56706', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345378.56706', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345378.56706', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345378.57919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345378.57919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345378.49804', 'epoch': '1717345378.49804', 'reported': 0, 'tombstones': -1}], 'c2c1efb7-9226-4f8b-b81a-81ab0b53a9b3-') test DEBUG: /tmp/tmpbmtnp8yh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.49804.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpbmtnp8yh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.49804.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '2b0f841446b34e389129d8920f937fb6', 'c2c1efb7-9226-4f8b-b81a-81ab0b53a9b3-', '1717345378.48000', '1717345378.55665', '0', '', '1717345378.47496', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345378.56706', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345378.56706', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345378.56706', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345378.57919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345378.57919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345378.49804', 'epoch': '1717345378.49804', 'reported': 0, 'tombstones': -1}], 'c2c1efb7-9226-4f8b-b81a-81ab0b53a9b3-') test DEBUG: /tmp/tmpbmtnp8yh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.49804.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpbmtnp8yh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345378.49804.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '417b3f87f15b72fdd1d863db9fb59e21', '49762d65-ca58-42de-a93e-5d4dd8c9028a-', '1717345385.81468', '1717345385.80827', '0', '', '1717345385.80827', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345385.93081', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345385.93081', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345385.93081', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345385.94833', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345385.94833', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345385.83461', 'epoch': '1717345385.83461', 'reported': 0, 'tombstones': -1}], '49762d65-ca58-42de-a93e-5d4dd8c9028a-') test DEBUG: /tmp/tmpmrvg3yy2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345385.83461.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpmrvg3yy2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345385.83461.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '417b3f87f15b72fdd1d863db9fb59e21', '49762d65-ca58-42de-a93e-5d4dd8c9028a-', '1717345385.81468', '1717345385.91487', '0', '', '1717345385.80827', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345385.93081', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345385.93081', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345385.93081', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345385.94833', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345385.94833', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345385.83461', 'epoch': '1717345385.83461', 'reported': 0, 'tombstones': -1}], '49762d65-ca58-42de-a93e-5d4dd8c9028a-') test DEBUG: /tmp/tmpmrvg3yy2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345385.83461.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpmrvg3yy2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345385.83461.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'c5d0bb0903104fad408110669d5b34d1', 'deefb089-76cd-40ca-a046-334bcc0a0a31-', '1717345389.32917', '1717345389.31355', '0', '', '1717345389.31355', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345389.33741', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.33741', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345389.33741', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345389.34548', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.34548', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345389.34518', 'epoch': '1717345389.34518', 'reported': 0, 'tombstones': -1}], 'deefb089-76cd-40ca-a046-334bcc0a0a31-') test DEBUG: /tmp/tmpp_ut6yz3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.34518.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpp_ut6yz3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.34518.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'c5d0bb0903104fad408110669d5b34d1', 'deefb089-76cd-40ca-a046-334bcc0a0a31-', '1717345389.32917', '1717345389.44231', '0', '', '1717345389.31355', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345389.33741', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.33741', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345389.33741', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345389.34548', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.34548', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345389.34518', 'epoch': '1717345389.34518', 'reported': 0, 'tombstones': -1}], 'deefb089-76cd-40ca-a046-334bcc0a0a31-') test DEBUG: /tmp/tmpp_ut6yz3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.34518.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpp_ut6yz3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.34518.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'Content-Md5': '0psj556+oeNaquEdDeMxig==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd29b23e79ebea1e35aaae11d0de3318a', 'Content-Length': '226', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '450'} test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '456'} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48a75983bfd947169b412-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48a75983bfd947169b412-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48a75983bfd947169b412-00665c9c7f) test ERROR: Container HEAD returning 503 for [] (txn: tx48a75983bfd947169b412-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx03763e503c044fc3b50c2-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx03763e503c044fc3b50c2-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx03763e503c044fc3b50c2-00665c9c7f) test ERROR: Container HEAD returning 503 for [] (txn: tx03763e503c044fc3b50c2-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors ... ok test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: tx2795c55d3b0643728e383-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx4fe6b80335684e4d90495-00665c9c82) test DEBUG: Setting object count to 3120 (txn: tx4fe6b80335684e4d90495-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test DEBUG: Setting object count to 3120 (txn: tx770ab18736e74bc1b7e2a-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test DEBUG: Setting object count to 3120 (txn: txc3e4ac3e34c944f99b02c-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test DEBUG: Setting object count to 3120 (txn: txa0b79f0c5cc54415b3d5f-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx274c850bedc04aba82d4d-00665c9c82) test DEBUG: Setting object count to 3120 (txn: tx274c850bedc04aba82d4d-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb8b1fbcdf45446881547-00665c9c82) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: txeb8b1fbcdf45446881547-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test INFO: Timeout waiting on remote server 127.0.0.1:51167/sda1: 99 seconds test DEBUG: Update failed for /a/c/o /tmp/tmph3xkdblz/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.85521 test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.00s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test INFO: Timeout connecting to remote server 127.0.0.1:51167/sda1: 9 seconds test DEBUG: Update failed for /a/c/o /tmp/tmph3xkdblz/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.85521 test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.00s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmph3xkdblz/devices/sda1 (pid: 3541999) test DEBUG: Update sent for /a/c/o /tmp/tmph3xkdblz/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345391.85521 test INFO: Object update sweep completed on /tmp/tmph3xkdblz/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test WARNING: Skipping: test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Object update sweep starting on /tmp/tmplvclyk9s/devices/sda1 (pid: 3541999) test INFO: Object update sweep progress on /tmp/tmplvclyk9s/devices/sda1: 16.00s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update sweep progress on /tmp/tmplvclyk9s/devices/sda1: 32.00s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update sweep completed on /tmp/tmplvclyk9s/devices/sda1 in 44.00s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) test INFO: Object update sweep starting on /tmp/tmpw_hwrh7m/devices/sda1 (pid: 3541999) test INFO: Object update sweep completed on /tmp/tmpw_hwrh7m/devices/sda1 in 0.13s seconds:, 6 successes, 0 failures, 0 quarantines, 6 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3541999) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmppk32ldfm/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmppk32ldfm/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmppk32ldfm/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmppk32ldfm/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0107 "-" 3541999 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0138 "-" 3541999 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0136 "-" 3541999 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0102 "-" 3541999 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb064f270f5f24f08b33b0-00665c9c70" "proxy-server 3541999" 0.0020 "-" 3541999 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:52631/sdb1/0/a/c" "txb064f270f5f24f08b33b0-00665c9c70" "container-server 3541999" 0.0030 "-" 3541999 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:44999/sda1/0/a/c" "txb064f270f5f24f08b33b0-00665c9c70" "container-server 3541999" 0.0024 "-" 3541999 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txb064f270f5f24f08b33b0-00665c9c70" "proxy-server 3541999" 0.0359 "-" 3541999 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txb064f270f5f24f08b33b0-00665c9c70" "proxy-server 3541999" 0.0233 "-" 3541999 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/12 PUT /v1/a/c HTTP/1.0 201 - - t - - - txb064f270f5f24f08b33b0-00665c9c70 - 0.0481 - - 1717345392.203021288 1717345392.251154900 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx2d08c158150e4b0bb225e-00665c9c70" "proxy-server 3541999" 0.0017 "-" 3541999 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:44999/sda1/2/a1/c1" "tx2d08c158150e4b0bb225e-00665c9c70" "container-server 3541999" 0.0018 "-" 3541999 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:52631/sdb1/2/a1/c1" "tx2d08c158150e4b0bb225e-00665c9c70" "container-server 3541999" 0.0025 "-" 3541999 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2d08c158150e4b0bb225e-00665c9c70" "proxy-server 3541999" 0.0224 "-" 3541999 0 test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE test.unit.proxy.test_mem_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:39 -0000', 'Content-Md5': 'i4CS9WiTvEfCFGqAHP8j0A==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8b8092f56893bc47c2146a801cff23d0', 'Content-Length': '1030946'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:55 -0000', 'Content-Md5': '/JzPJjlXCqpRBmlyhGXNDg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'fc9ccf2639570aaa510669728465cd0e', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states ... ok swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. The overload factor is now 19.88% (0.198780) The change will take effect after the next rebalance. Overload must be non-negative. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. 'swedish fish' is not a valid number. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The replica count is now 3.141593. The change will take effect after the next rebalance. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. "test" is not a valid number. Replica count must be at least 1. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 200.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 200.0 d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 200.0 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 200.0 test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone test.unit.cli.test_ringbuilder.TestCommands.test_set_zone ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '448'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path ... ok test INFO: - - - [02/Jun/2024:16:23:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0160 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:04 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0102 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:04 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:04 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0155 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:04 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:04 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:04 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:04 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0203 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:04 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.1343 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0152 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0142 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0086 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0111 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0118 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0158 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0156 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0119 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0018 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0243 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0076 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0044 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0076 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0101 "-" 3542007 3 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0078 "-" 3542007 4 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0233 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0101 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0151 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0214 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0061 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0039 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0166 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0268 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0360 "-" 3542007 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3542007 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0237 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0004 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0183 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0077 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:10 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0036 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0036 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0037 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0070 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0056 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0029 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0051 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0027 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0023 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0020 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0052 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0028 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0043 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'c5d0bb0903104fad408110669d5b34d1', 'ea190d2e-d119-44b2-a9cc-6dfb1435eda4-', '1717345389.51651', '1717345389.51279', '0', '', '1717345389.51279', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345389.53212', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.53212', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345389.53212', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345389.54489', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.54489', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345389.54442', 'epoch': '1717345389.54442', 'reported': 0, 'tombstones': -1}], 'ea190d2e-d119-44b2-a9cc-6dfb1435eda4-') test DEBUG: /tmp/tmpax15lh05/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.54442.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpax15lh05/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.54442.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'c5d0bb0903104fad408110669d5b34d1', 'ea190d2e-d119-44b2-a9cc-6dfb1435eda4-', '1717345389.51651', '1717345389.61953', '0', '', '1717345389.51279', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345389.53212', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.53212', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345389.53212', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345389.54489', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345389.54489', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345389.54442', 'epoch': '1717345389.54442', 'reported': 0, 'tombstones': -1}], 'ea190d2e-d119-44b2-a9cc-6dfb1435eda4-') test DEBUG: /tmp/tmpax15lh05/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.54442.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpax15lh05/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345389.54442.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'c1beadb8fa6c1d058e96a54996695ca0', '70e533a5-2a3b-4ba0-9796-0c5e85aac29b-sda', '1717345396.32343', '1717345396.31642', '0', '', '1717345396.31642', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('70e533a5-2a3b-4ba0-9796-0c5e85aac29b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'a577416bf2fae11bc8af6a8d8667fa26', '36e2a5c7-2075-4c76-99d3-bb35397da795-sda', '1717345396.45237', '1717345396.43966', '0', '', '1717345396.43966', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345396.51506', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345396.51506', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345396.51506', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345396.52452', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345396.52452', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345396.52404', 'epoch': '1717345396.52404', 'reported': 0, 'tombstones': -1}], '36e2a5c7-2075-4c76-99d3-bb35397da795-sda') test DEBUG: /tmp/tmpi4vs4try/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpi4vs4try/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'a577416bf2fae11bc8af6a8d8667fa26', '36e2a5c7-2075-4c76-99d3-bb35397da795-sda', '1717345396.45237', '1717345396.49122', '0', '', '1717345396.43966', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345396.51506', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345396.51506', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345396.51506', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345396.52452', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345396.52452', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345396.52404', 'epoch': '1717345396.52404', 'reported': 0, 'tombstones': -1}], '36e2a5c7-2075-4c76-99d3-bb35397da795-sda') test DEBUG: /tmp/tmpi4vs4try/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpi4vs4try/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'a577416bf2fae11bc8af6a8d8667fa26', '6bf9d621-1bff-439c-af79-bb236ea0d255-sda', '1717345396.72388', '1717345396.71673', '0', '', '1717345396.71673', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345396.76156', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345396.76156', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345396.76156', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345396.77481', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345396.77481', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345396.77422', 'epoch': '1717345396.77422', 'reported': 0, 'tombstones': -1}], '6bf9d621-1bff-439c-af79-bb236ea0d255-sda') test DEBUG: /tmp/tmpsohqr8mg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpsohqr8mg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'a577416bf2fae11bc8af6a8d8667fa26', '6bf9d621-1bff-439c-af79-bb236ea0d255-sda', '1717345396.72388', '1717345396.74389', '0', '', '1717345396.71673', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345396.76156', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345396.76156', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345396.76156', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345396.77481', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345396.77481', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345396.77422', 'epoch': '1717345396.77422', 'reported': 0, 'tombstones': -1}], '6bf9d621-1bff-439c-af79-bb236ea0d255-sda') test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines ... ok test WARNING: Skipping /tmp/tmpxo3nq8f_/devices3/device_is_file/data because [Errno 20] Not a directory: '/tmp/tmpxo3nq8f_/devices3/device_is_file/data' test WARNING: Skipping /tmp/tmpxo3nq8f_/devices4/device/data because test WARNING: Skipping device as it is not mounted test WARNING: Skipping /tmp/tmpl_u73szi/asdf/data because [Errno 20] Not a directory: '/tmp/tmpl_u73szi/asdf/data' test WARNING: Skipping /tmp/tmpl_u73szi/asdf/data because [Errno 20] Not a directory: '/tmp/tmpl_u73szi/asdf/data' test WARNING: Skipping drive as it is not mounted test WARNING: Skipping asdf as it is not mounted test WARNING: Skipping /tmp/tmpft8i3ios/asdf/data because [Errno 20] Not a directory: '/tmp/tmpft8i3ios/asdf/data' test ERROR: Problem finding shard upper in '/tmp/tmphil14sni/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmphil14sni/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmp56xyeirr/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmp56xyeirr/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx0aaeaa42d2cb4c66b85d4-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx2bf911cf0017482e99aab-00665c9c82) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx2bf911cf0017482e99aab-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txf2e0ea532aca433c9b4dd-00665c9c82) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txf2e0ea532aca433c9b4dd-00665c9c82) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf2e0ea532aca433c9b4dd-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2e0ea532aca433c9b4dd-00665c9c82) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txf2e0ea532aca433c9b4dd-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx27f8a752957e44da85602-00665c9c82) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx27f8a752957e44da85602-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx27f8a752957e44da85602-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx27f8a752957e44da85602-00665c9c82) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx27f8a752957e44da85602-00665c9c82) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx0ca23ecc88a0456d9e58a-00665c9c82) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx0ca23ecc88a0456d9e58a-00665c9c82) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx0ca23ecc88a0456d9e58a-00665c9c82) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx0ca23ecc88a0456d9e58a-00665c9c82) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx0ca23ecc88a0456d9e58a-00665c9c82) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txcdc79cc0dc88448280e6a-00665c9c85) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txcdc79cc0dc88448280e6a-00665c9c85) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: txcdc79cc0dc88448280e6a-00665c9c85) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx4e356d1039464ee28d9c1-00665c9c87) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_options_info test.unit.proxy.controllers.test_info.TestInfoController.test_options_info ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: txee0f86881db84ce2a14a9-00665c9c7f) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: txee0f86881db84ce2a14a9-00665c9c7f) test INFO: Client disconnected on read of '/a/c' (txn: txee0f86881db84ce2a14a9-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2848fb1556b24ec2a386b-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2848fb1556b24ec2a386b-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2848fb1556b24ec2a386b-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2848fb1556b24ec2a386b-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: tx6d79f419565444f4bec90-00665c9c7f) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx9553f7c18a7548c6b2336-00665c9c7f) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx66ca192c776946e293e57-00665c9c7f) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd (txn: txc2f3403548ef4bcd9e95b-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: txb0d33d0f21c94a2aac188-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx67cba0d741c143bb878b1-00665c9c7f) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx19b74761d0db44c297295-00665c9c7f) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: txa980ba656dce4831b6969-00665c9c7f) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx932d10744e184aee9508d-00665c9c7f) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx03701b07c64b4af9a67bf-00665c9c7f) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx5bad6e6247fc40c79497b-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: tx7326339de7064aa8bda98-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txde88bdb9a85540248b6c0-00665c9c7f) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txd714df3509ad4e0e85d13-00665c9c7f) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: tx30b90b01b32b4dddb4743-00665c9c7f) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txd5fd0205104746f493a46-00665c9c7f) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx551c4ac8e8cc429684c3d-00665c9c7f) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx94d53de2b8374e87aa7d5-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: tx7c84d5b303634d0695303-00665c9c7f) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx5fdcfb023a884663bd6c0-00665c9c7f) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txee814a2fc515496ba7c77-00665c9c7f) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: Setting object count to 1065 (txn: tx4e356d1039464ee28d9c1-00665c9c87) test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:22:55 -0000', 'Content-Md5': 'PggRsy85Tu07EsbATYB/cA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '3e0811b32f394eed3b12c6c04d807f70', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:05 -0000', 'Content-Md5': 'zDBBHq17IpuHzskOm+GHEQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'cc30411ead7b229b87cec90e9be18711', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:05 -0000', 'Content-Md5': 'hTnOh9sxGnzPiScnD+t0Mg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8539ce87db311a7ccf8927270feb7432', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'Content-Md5': 'mbEtd8ufKpkpCIfMjGD7Tw==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '99b12d77cb9f2a99290887cc8c60fb4f', 'Content-Length': '90'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 220, in PUT self.req.get_acl_response(app, 'PUT', self.container) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private ... ok cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:12 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2d08c158150e4b0bb225e-00665c9c70" "proxy-server 3541999" 0.0397 "-" 3541999 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/15 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx2d08c158150e4b0bb225e-00665c9c70 - 3.2549 - - 1717345392.253041267 1717345395.507975578 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx72126a2a3ec2498f9b0e9-00665c9c73" "proxy-server 3541999" 0.0110 "-" 3541999 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:52631/sdb1/0/a/c1" "tx72126a2a3ec2498f9b0e9-00665c9c73" "container-server 3541999" 0.0160 "-" 3541999 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:44999/sda1/0/a/c1" "tx72126a2a3ec2498f9b0e9-00665c9c73" "container-server 3541999" 0.0024 "-" 3541999 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx72126a2a3ec2498f9b0e9-00665c9c73" "proxy-server 3541999" 0.0561 "-" 3541999 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx72126a2a3ec2498f9b0e9-00665c9c73" "proxy-server 3541999" 0.0406 "-" 3541999 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/15 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx72126a2a3ec2498f9b0e9-00665c9c73 - 0.0799 - - 1717345395.510171890 1717345395.590109348 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdb5d114ab54a4ea6b84a5-00665c9c73" "proxy-server 3541999" 0.0103 "-" 3541999 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:44999/sda1/2/a/c2" "txdb5d114ab54a4ea6b84a5-00665c9c73" "container-server 3541999" 0.0025 "-" 3541999 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:52631/sdb1/2/a/c2" "txdb5d114ab54a4ea6b84a5-00665c9c73" "container-server 3541999" 0.0096 "-" 3541999 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txdb5d114ab54a4ea6b84a5-00665c9c73" "proxy-server 3541999" 0.0605 "-" 3541999 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:15 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txdb5d114ab54a4ea6b84a5-00665c9c73" "proxy-server 3541999" 0.0336 "-" 3541999 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/15 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txdb5d114ab54a4ea6b84a5-00665c9c73 - 0.0863 - - 1717345395.592626333 1717345395.678953886 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: txce9a7b663142459285580-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txcc789617ce09413db2ad2-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: tx283032f0729d4793a364d-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx283032f0729d4793a364d-00665c9c77) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx283032f0729d4793a364d-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txd68a25196f9e4f608041b-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txd68a25196f9e4f608041b-00665c9c77) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txd68a25196f9e4f608041b-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx052d1eb1a53547d7a7403-00665c9c77) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx052d1eb1a53547d7a7403-00665c9c77) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx3e4245dd849245828e42d-00665c9c77) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx3e4245dd849245828e42d-00665c9c77) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx9de1b21136264cb4bedd2-00665c9c77) test INFO: Client disconnected on read of '/a/c' (txn: tx9de1b21136264cb4bedd2-00665c9c77) test INFO: Client disconnected on read of '/a/c' (txn: tx279944e0af874bf3814f1-00665c9c77) test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST test.unit.proxy.test_mem_server.TestContainerController.test_POST ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user ... ok Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 3.14 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 6.28 Search value matched 0 devices. The on-disk ring builder is unchanged. d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_unknown test.unit.cli.test_ringbuilder.TestCommands.test_unknown ... ok test ERROR: Failed to unlink '/tmp/tmpivqo8yfi/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpivqo8yfi/containers/part/suffix/hash/container_1717345437.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpivqo8yfi/containers/part/suffix/hash/container_1717345437.00000.db': ['/tmp/tmpivqo8yfi/containers/part/suffix/hash/container.db', '/tmp/tmpivqo8yfi/containers/part/suffix/hash/container_1717345438.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpivqo8yfi/containers/part/suffix/hash/container_1717345438.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp1h51v2qr/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp1h51v2qr/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp1h51v2qr/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp1h51v2qr/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0124 "-" 3541997 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0134 "-" 3541997 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0109 "-" 3541997 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0104 "-" 3541997 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txafab05f82ae24fb4b703e-00665c9c9e" "proxy-server 3541997" 0.0032 "-" 3541997 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:39313/sdb1/1/a/c" "txafab05f82ae24fb4b703e-00665c9c9e" "container-server 3541997" 0.0019 "-" 3541997 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:35307/sda1/1/a/c" "txafab05f82ae24fb4b703e-00665c9c9e" "container-server 3541997" 0.0017 "-" 3541997 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txafab05f82ae24fb4b703e-00665c9c9e" "proxy-server 3541997" 0.0231 "-" 3541997 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txafab05f82ae24fb4b703e-00665c9c9e" "proxy-server 3541997" 0.0380 "-" 3541997 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/58 PUT /v1/a/c HTTP/1.0 201 - - t - - - txafab05f82ae24fb4b703e-00665c9c9e - 0.0524 - - 1717345438.146206617 1717345438.198573828 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txbeaa8bf13a2c4f61922ed-00665c9c9e" "proxy-server 3541997" 0.0021 "-" 3541997 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35307/sda1/1/a1/c1" "txbeaa8bf13a2c4f61922ed-00665c9c9e" "container-server 3541997" 0.0018 "-" 3541997 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:39313/sdb1/1/a1/c1" "txbeaa8bf13a2c4f61922ed-00665c9c9e" "container-server 3541997" 0.0036 "-" 3541997 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbeaa8bf13a2c4f61922ed-00665c9c9e" "proxy-server 3541997" 0.0335 "-" 3541997 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbeaa8bf13a2c4f61922ed-00665c9c9e" "proxy-server 3541997" 0.0225 "-" 3541997 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/58 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txbeaa8bf13a2c4f61922ed-00665c9c9e - 0.0473 - - 1717345438.200906277 1717345438.248232365 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb822769c0b604d05a8882-00665c9c9e" "proxy-server 3541997" 0.0074 "-" 3541997 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:35307/sda1/2/a/c1" "txb822769c0b604d05a8882-00665c9c9e" "container-server 3541997" 0.0019 "-" 3541997 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:39313/sdb1/2/a/c1" "txb822769c0b604d05a8882-00665c9c9e" "container-server 3541997" 0.0055 "-" 3541997 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb822769c0b604d05a8882-00665c9c9e" "proxy-server 3541997" 0.0249 "-" 3541997 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb822769c0b604d05a8882-00665c9c9e" "proxy-server 3541997" 0.0390 "-" 3541997 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/58 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txb822769c0b604d05a8882-00665c9c9e - 0.0572 - - 1717345438.250480890 1717345438.307641268 1 test.unit.proxy.test_server.TestAccountController.test_DELETE test.unit.proxy.test_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_server.TestAccountController.test_GET test.unit.proxy.test_server.TestAccountController.test_GET ... ok test.unit.proxy.test_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD test.unit.proxy.test_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_OPTIONS test.unit.proxy.test_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_POST_metadata test.unit.proxy.test_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT test.unit.proxy.test_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_status test.unit.proxy.test_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_server.TestAccountController.test_connection_refused test.unit.proxy.test_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_server.TestAccountController.test_other_socket_error test.unit.proxy.test_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings ... ok test INFO: - - - [02/Jun/2024:16:23:11 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0053 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmpgm22uwdk/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0043 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0047 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0183 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0046 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0050 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0046 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0043 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0043 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0039 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0043 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0084 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0044 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:15 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0044 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3542007 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3542007 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0228 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0086 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:18 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0121 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0169 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0044 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0185 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0011 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0139 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0076 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0218 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0176 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0173 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0017 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:19 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0140 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0205 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0102 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0256 "-" 3542007 - test ERROR: ERROR Account update failed with 127.0.0.1:38235/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0246 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0025 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0005 "-" 3542007 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3542007 3 test INFO: - - - [02/Jun/2024:16:23:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3542007 3 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found ... ok test DEBUG: /tmp/tmpsohqr8mg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpsohqr8mg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7a916d5fcdda05db698b8c7fb572b077', '34546fa6-0226-47e6-b9d1-7328fbe1eb8a-sda', '1717345405.15697', '1717345405.15148', '0', '', '1717345405.15148', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('34546fa6-0226-47e6-b9d1-7328fbe1eb8a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmpmqim06m_/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7a916d5fcdda05db698b8c7fb572b077', 'e2953cd7-9282-43fb-a82e-036f3b8cf81c-sda', '1717345405.31822', '1717345405.31345', '0', '', '1717345405.31345', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('e2953cd7-9282-43fb-a82e-036f3b8cf81c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7a916d5fcdda05db698b8c7fb572b077', 'cef2e934-84a3-48e3-bffb-60df7360d025-sda', '1717345405.49574', '1717345405.48951', '0', '', '1717345405.48951', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('cef2e934-84a3-48e3-bffb-60df7360d025-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmpme8tqb2_/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7a916d5fcdda05db698b8c7fb572b077', 'd91d4d61-85a0-4ece-bf63-ea1efb05d9de-sda', '1717345405.78058', '1717345405.77697', '0', '', '1717345405.77697', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('d91d4d61-85a0-4ece-bf63-ea1efb05d9de-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f74d892c-9722-44b0-ae9d-d86a6da1b42a-sda', '1717345405.97731', '1717345405.96794', '0', '', '1717345405.96794', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpfi_np2le/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2a1784e2-daf8-4d5c-a116-dc7d6b65bbcf-sdb', '1717345406.10614', '1717345407.00000', '0', '', '1717345407.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2a1784e2-daf8-4d5c-a116-dc7d6b65bbcf-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2a1784e2-daf8-4d5c-a116-dc7d6b65bbcf-sdb', '1717345406.10614', '1717345407.00000', '0', '', '1717345407.00000', 0, 0) test ERROR: Quarantining DB /tmp/tmpfi_np2le/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2a1784e2-daf8-4d5c-a116-dc7d6b65bbcf-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06299 seconds (15.87426/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2l7100px/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '5474582a-cfa8-4834-adbb-7eb8d2d3a237-sda', '1717345406.20167', '1717345406', '0', '', '1717345406', 0, 1) test DEBUG: /tmp/tmp2l7100px/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '5474582a-cfa8-4834-adbb-7eb8d2d3a237-sda', '1717345406.20167', '1717345406', '0', '', '1717345406', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5474582a-cfa8-4834-adbb-7eb8d2d3a237-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09560 seconds (10.46004/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxdo_i54a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ef44d6966c4c159ec53d88952e43ce6d', 'e9636703-fcec-44e5-92e5-178b3d7e87be-sda', '1717345406.33709', '1717345406', '0', '', '1717345406', 1, 1) test DEBUG: /tmp/tmpxdo_i54a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345408.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'e9636703-fcec-44e5-92e5-178b3d7e87be-sda') test DEBUG: /tmp/tmpxdo_i54a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0109241s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ef44d6966c4c159ec53d88952e43ce6d', 'e9636703-fcec-44e5-92e5-178b3d7e87be-sda', '1717345406.33709', '1717345406', '0', '', '1717345406', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e9636703-fcec-44e5-92e5-178b3d7e87be-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08367 seconds (11.95117/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1cadiuof/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2f39915a-495b-4572-8657-a0f6cbaced46-sda', '1717345406.46265', '1717345406', '0', '', '1717345406', 0, 1) test DEBUG: /tmp/tmp1cadiuof/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2f39915a-495b-4572-8657-a0f6cbaced46-sda', '1717345406.46265', '1717345406', '0', '', '1717345406', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2f39915a-495b-4572-8657-a0f6cbaced46-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06797 seconds (14.71209/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphm5b79e5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ef44d6966c4c159ec53d88952e43ce6d', '5ffa9efd-7cbb-4c66-8277-997b49291536-sda', '1717345406.56541', '1717345406', '0', '', '1717345406', 1, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'Content-Md5': '9Z+cE+YYsVYXLtR89GLUSA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'f59f9c13e618b156172ed47cf462d448', 'Content-Length': '88'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx86e260bd967d47deb303c-00665c9c9e" "proxy-server 3541997" 0.0064 "-" 3541997 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:39313/sdb1/1/a/c2" "tx86e260bd967d47deb303c-00665c9c9e" "container-server 3541997" 0.0019 "-" 3541997 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:35307/sda1/1/a/c2" "tx86e260bd967d47deb303c-00665c9c9e" "container-server 3541997" 0.0058 "-" 3541997 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx86e260bd967d47deb303c-00665c9c9e" "proxy-server 3541997" 0.0372 "-" 3541997 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:23:58 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx86e260bd967d47deb303c-00665c9c9e" "proxy-server 3541997" 0.0262 "-" 3541997 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/23/58 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx86e260bd967d47deb303c-00665c9c9e - 0.0560 - - 1717345438.309911251 1717345438.365895748 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': True, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 4}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.2, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl ... ok test INFO: Client disconnected on read of '/a/c' (txn: tx279944e0af874bf3814f1-00665c9c77) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txc5c4c1263ca14c87841b4-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx6a622178b1c3428c8da12-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txff32c81d19544284bb2ce-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txff32c81d19544284bb2ce-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txff32c81d19544284bb2ce-00665c9c7c) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: txff32c81d19544284bb2ce-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa7236bad8a4b432db05f9-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txa7236bad8a4b432db05f9-00665c9c7c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txa7236bad8a4b432db05f9-00665c9c7c) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: txa7236bad8a4b432db05f9-00665c9c7c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: tx8fe6018456014976a3088-00665c9c80) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: tx8fe6018456014976a3088-00665c9c80) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: tx8fe6018456014976a3088-00665c9c80) test ERROR: Account HEAD returning 503 for [] (txn: tx8fe6018456014976a3088-00665c9c80) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx3743e89698204bbc9f36e-00665c9c80) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.2:2002/sdc (txn: txc434bcdfc97d4cb4b3b2d-00665c9c80) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txc434bcdfc97d4cb4b3b2d-00665c9c80) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx11300351c1fe42e394a6d-00665c9c80) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx272763f4f99f49fba5ff0-00665c9c80) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf49d7129cde24fbd99642-00665c9c80) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx346094c2bea44e3ab48c0-00665c9c80) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txade4f79dcdce47e58f3ba-00665c9c81) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx5f5397361c464fc6a639b-00665c9c81) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx5f5397361c464fc6a639b-00665c9c81) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx15c9149addd6467681495-00665c9c81) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx15c9149addd6467681495-00665c9c81) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx15c9149addd6467681495-00665c9c81) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx15c9149addd6467681495-00665c9c81) test.unit.proxy.test_mem_server.TestContainerController.test_PUT test.unit.proxy.test_mem_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission ... ok test DEBUG: /tmp/tmphm5b79e5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ef44d6966c4c159ec53d88952e43ce6d', '5ffa9efd-7cbb-4c66-8277-997b49291536-sda', '1717345406.56541', '1717345406', '0', '', '1717345406', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5ffa9efd-7cbb-4c66-8277-997b49291536-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.43730 seconds (0.41029/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgwcmmfjf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ce3cc60d-47f0-4bd4-a342-82677ecc44d3-sda', '1717345412.76737', '1717345412', '0', '', '1717345412', 0, 0) test DEBUG: /tmp/tmpgwcmmfjf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ce3cc60d-47f0-4bd4-a342-82677ecc44d3-sda', '1717345412.76737', '1717345412', '0', '', '1717345412', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ce3cc60d-47f0-4bd4-a342-82677ecc44d3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03531 seconds (28.32259/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp52zwnc4c/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '180dafa38025fea76904139478ade219', 'b2baecd1-bf65-49fe-a75d-e2839f22fdc4-sda', '1717345412.84269', '1717345412', '0', '', '1717345412', 1, 0) test DEBUG: /tmp/tmp52zwnc4c/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345415.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'b2baecd1-bf65-49fe-a75d-e2839f22fdc4-sda') test DEBUG: /tmp/tmp52zwnc4c/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00897312s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '180dafa38025fea76904139478ade219', 'b2baecd1-bf65-49fe-a75d-e2839f22fdc4-sda', '1717345412.84269', '1717345412', '0', '', '1717345412', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b2baecd1-bf65-49fe-a75d-e2839f22fdc4-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05905 seconds (16.93591/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxqocd5bz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '65bdf642-8e65-4ddf-917f-71e997917122-sda', '1717345412.93776', '1717345412', '0', '', '1717345412', 0, 1) test DEBUG: /tmp/tmpxqocd5bz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '65bdf642-8e65-4ddf-917f-71e997917122-sda', '1717345412.93776', '1717345412', '0', '', '1717345412', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('65bdf642-8e65-4ddf-917f-71e997917122-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04306 seconds (23.22371/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmppp8nj25r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec1414ca598132a31cf91cd9737fb4ea', 'a39a96a5-4fc3-41e8-9896-1bb8e5634e88-sda', '1717345413.01539', '1717345413', '0', '', '1717345413', 1, 0) test DEBUG: /tmp/tmppp8nj25r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec1414ca598132a31cf91cd9737fb4ea', 'a39a96a5-4fc3-41e8-9896-1bb8e5634e88-sda', '1717345413.01539', '1717345413', '0', '', '1717345413', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a39a96a5-4fc3-41e8-9896-1bb8e5634e88-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05916 seconds (16.90417/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpv_71uwal/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ca33f3f4-3146-458a-a2b4-31fb5b7da09d-sda', '1717345413.12642', '1717345415', '0', '', '1717345415', 0, 1) test DEBUG: /tmp/tmpv_71uwal/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ca33f3f4-3146-458a-a2b4-31fb5b7da09d-sda', '1717345413.12642', '1717345415', '0', '', '1717345415', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ca33f3f4-3146-458a-a2b4-31fb5b7da09d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03645 seconds (27.43143/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp__36aq3t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec1414ca598132a31cf91cd9737fb4ea', '0ab6ca7b-2e33-495e-9a1f-ef0458e14ba2-sda', '1717345413.19621', '1717345415', '0', '', '1717345415', 1, 1) test DEBUG: /tmp/tmp__36aq3t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345416.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '0ab6ca7b-2e33-495e-9a1f-ef0458e14ba2-sda') test DEBUG: /tmp/tmp__36aq3t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00901747s) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error ... ok test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'Content-Md5': 'g8a5/BKgecQpG1SedXwMBA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '83c6b9fc12a079c4291b549e757c0c04', 'Content-Length': '289'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0122 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0054 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0050 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0129 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0103 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0015 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0115 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0021 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0020 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0021 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0200 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0069 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0014 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0069 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0053 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0068 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0052 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default ... ok test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx15c9149addd6467681495-00665c9c81) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx15c9149addd6467681495-00665c9c81) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx15c9149addd6467681495-00665c9c81) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx15c9149addd6467681495-00665c9c81) test WARNING: Could not autocreate account '/account' (txn: tx15c9149addd6467681495-00665c9c81) test INFO: autocreate account '/account' (txn: tx50247a2e60a94748af308-00665c9c85) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txb71c283d914141bd9effa-00665c9c85) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txb71c283d914141bd9effa-00665c9c85) test ERROR: Account HEAD returning 503 for [503, 503] (txn: txb71c283d914141bd9effa-00665c9c85) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: txb71c283d914141bd9effa-00665c9c85) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: txb71c283d914141bd9effa-00665c9c85) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: txb71c283d914141bd9effa-00665c9c85) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txb71c283d914141bd9effa-00665c9c85) test WARNING: Could not autocreate account '/account' (txn: txb71c283d914141bd9effa-00665c9c85) test INFO: autocreate account '/account' (txn: tx294c0b1c21fc48579bdd8-00665c9c85) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txc44fd532f814465e9ecc3-00665c9c85) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txc44fd532f814465e9ecc3-00665c9c85) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txc44fd532f814465e9ecc3-00665c9c85) test INFO: autocreate account '/account' (txn: tx1a94530163994bc9ad3c9-00665c9c85) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx9d3d893d71514ee6a565d-00665c9c85) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx9d3d893d71514ee6a565d-00665c9c85) test INFO: autocreate account '/account' (txn: tx9d3d893d71514ee6a565d-00665c9c85) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx6eeda08b20fb43bc9e739-00665c9c85) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx6eeda08b20fb43bc9e739-00665c9c85) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: txc893114fe17c470f867f9-00665c9c88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: txbd462e1ff2cc4189bb805-00665c9c88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4caeaac5dfbc47518481f-00665c9c88) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8e98673456cd4df997525-00665c9c88) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa5757447b1643fbaad98-00665c9c88) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx13481a4ab6034df0b3867-00665c9c88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx9f714a9508a54714876a8-00665c9c7f) test INFO: Client disconnected on read of '/a/c' (txn: tx0975bd88f5af49dca8e68-00665c9c7f) test INFO: Client disconnected on read of '/a/c' (txn: txbb74fe115adf405a99391-00665c9c7f) test INFO: Client disconnected on read of '/a/c' (txn: txbb74fe115adf405a99391-00665c9c7f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant AllUsers WRITE permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. Unknown command: unknown swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate test.unit.cli.test_ringbuilder.TestCommands.test_validate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder test.unit.cli.test_ringbuilder.TestCommands.test_write_builder ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_ring ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys ... ok test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_update test.unit.common.middleware.s3api.test_utils.TestConfig.test_update ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success ... ok test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_stubs test.unit.common.test_daemon.TestWorkerDaemon.test_stubs ... ok test.unit.common.test_db.TestDatabaseConnectionError.test_str test.unit.common.test_db.TestDatabaseConnectionError.test_str ... ok test.unit.common.test_db.TestExampleBroker.test_commit_pending test.unit.common.test_db.TestExampleBroker.test_commit_pending ... ok test.unit.common.test_db.TestExampleBroker.test_delete_db test.unit.common.test_db.TestExampleBroker.test_delete_db ... ok test.unit.common.test_db.TestExampleBroker.test_get_info test.unit.common.test_db.TestExampleBroker.test_get_info ... ok test.unit.common.test_db.TestExampleBroker.test_get_max_row test.unit.common.test_db.TestExampleBroker.test_get_max_row ... ok test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata ... ok test.unit.common.test_db.TestExampleBroker.test_get_syncs test.unit.common.test_db.TestExampleBroker.test_get_syncs ... ok test.unit.common.test_db.TestExampleBroker.test_maybe_get test.unit.common.test_db.TestExampleBroker.test_maybe_get ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_put_timestamp test.unit.common.test_db.TestExampleBroker.test_put_timestamp ... ok test.unit.common.test_db.TestExampleBroker.test_status_changed_at test.unit.common.test_db.TestExampleBroker.test_status_changed_at ... ok test.unit.common.test_swob.TestEtag.test_normalize_bytes test.unit.common.test_swob.TestEtag.test_normalize_bytes ... ok test.unit.common.test_swob.TestEtag.test_normalize_etag test.unit.common.test_swob.TestEtag.test_normalize_etag ... ok test.unit.common.test_swob.TestRange.test_range test.unit.common.test_swob.TestRange.test_range ... ok test.unit.common.test_swob.TestRange.test_range_invalid_syntax test.unit.common.test_swob.TestRange.test_range_invalid_syntax ... ok test.unit.common.test_swob.TestRange.test_ranges_for_large_length test.unit.common.test_swob.TestRange.test_ranges_for_large_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length test.unit.common.test_swob.TestRange.test_ranges_for_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_edges test.unit.common.test_swob.TestRange.test_ranges_for_length_edges ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_multi test.unit.common.test_swob.TestRange.test_ranges_for_length_multi ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many ... ok test.unit.common.test_swob.TestRange.test_str test.unit.common.test_swob.TestRange.test_str ... ok test.unit.common.test_swob.TestRange.test_upsidedown_range test.unit.common.test_swob.TestRange.test_upsidedown_range ... ok test.unit.common.test_swob.TestRequest.test_401_invalid_path test.unit.common.test_swob.TestRequest.test_401_invalid_path ... ok test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path ... ok test.unit.common.test_swob.TestRequest.test_401_unauthorized test.unit.common.test_swob.TestRequest.test_401_unauthorized ... ok test.unit.common.test_swob.TestRequest.test_401_valid_account_path test.unit.common.test_swob.TestRequest.test_401_valid_account_path ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted ... ok test.unit.common.test_swob.TestRequest.test_accept_header test.unit.common.test_swob.TestRequest.test_accept_header ... ok test.unit.common.test_swob.TestRequest.test_allow_reserved_names test.unit.common.test_swob.TestRequest.test_allow_reserved_names ... ok test.unit.common.test_swob.TestRequest.test_as_referer test.unit.common.test_swob.TestRequest.test_as_referer ... ok test.unit.common.test_swob.TestRequest.test_bad_path_info_pop test.unit.common.test_swob.TestRequest.test_bad_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_bad_range test.unit.common.test_swob.TestRequest.test_bad_range ... ok test.unit.common.test_swob.TestRequest.test_blank test.unit.common.test_swob.TestRequest.test_blank ... ok test.unit.common.test_swob.TestRequest.test_blank_body_precedence test.unit.common.test_swob.TestRequest.test_blank_body_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_parsing test.unit.common.test_swob.TestRequest.test_blank_parsing ... ok test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args ... ok test.unit.common.test_swob.TestRequest.test_copy_get test.unit.common.test_swob.TestRequest.test_copy_get ... ok test.unit.common.test_swob.TestRequest.test_datetime_properties test.unit.common.test_swob.TestRequest.test_datetime_properties ... ok test.unit.common.test_swob.TestRequest.test_get_response test.unit.common.test_swob.TestRequest.test_get_response ... ok test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args ... ok test.unit.common.test_swob.TestRequest.test_message_length_just_content_length test.unit.common.test_swob.TestRequest.test_message_length_just_content_length ... ok test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding ... ok test.unit.common.test_swob.TestRequest.test_not_401 test.unit.common.test_swob.TestRequest.test_not_401 ... ok test.unit.common.test_swob.TestRequest.test_params test.unit.common.test_swob.TestRequest.test_params ... ok test.unit.common.test_swob.TestRequest.test_path test.unit.common.test_swob.TestRequest.test_path ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop test.unit.common.test_swob.TestRequest.test_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_last test.unit.common.test_swob.TestRequest.test_path_info_pop_last ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_none test.unit.common.test_swob.TestRequest.test_path_info_pop_none ... ok test.unit.common.test_swob.TestRequest.test_path_qs test.unit.common.test_swob.TestRequest.test_path_qs ... ok test.unit.common.test_swob.TestRequest.test_path_question_mark test.unit.common.test_swob.TestRequest.test_path_question_mark ... ok test.unit.common.test_swob.TestRequest.test_properties test.unit.common.test_swob.TestRequest.test_properties ... ok test.unit.common.test_swob.TestRequest.test_split_path test.unit.common.test_swob.TestRequest.test_split_path ... ok test.unit.common.test_swob.TestRequest.test_swift_entity_path test.unit.common.test_swob.TestRequest.test_swift_entity_path ... ok test.unit.common.test_swob.TestRequest.test_timestamp test.unit.common.test_swob.TestRequest.test_timestamp ... ok test.unit.common.test_swob.TestRequest.test_timestamp_invalid test.unit.common.test_swob.TestRequest.test_timestamp_invalid ... ok test.unit.common.test_swob.TestRequest.test_timestamp_missing test.unit.common.test_swob.TestRequest.test_timestamp_missing ... ok test.unit.common.test_swob.TestRequest.test_unicode_params test.unit.common.test_swob.TestRequest.test_unicode_params ... ok test.unit.common.test_swob.TestRequest.test_unicode_path test.unit.common.test_swob.TestRequest.test_unicode_path ... ok test.unit.common.test_swob.TestRequest.test_unicode_query test.unit.common.test_swob.TestRequest.test_unicode_query ... ok test.unit.common.test_swob.TestRequest.test_url test.unit.common.test_swob.TestRequest.test_url ... ok test.unit.common.test_swob.TestRequest.test_url2 test.unit.common.test_swob.TestRequest.test_url2 ... ok test.unit.common.test_swob.TestRequest.test_wsgify test.unit.common.test_swob.TestRequest.test_wsgify ... ok test.unit.common.test_swob.TestRequest.test_wsgify_method test.unit.common.test_swob.TestRequest.test_wsgify_method ... ok test.unit.common.test_swob.TestRequest.test_wsgify_raise test.unit.common.test_swob.TestRequest.test_wsgify_raise ... ok test.unit.common.test_utils.TestShardRange.test_constants test.unit.common.test_utils.TestShardRange.test_constants ... ok test.unit.common.test_utils.TestShardRange.test_copy test.unit.common.test_utils.TestShardRange.test_copy ... ok test.unit.common.test_utils.TestShardRange.test_deleted_setter test.unit.common.test_utils.TestShardRange.test_deleted_setter ... ok test.unit.common.test_utils.TestShardRange.test_epoch_setter test.unit.common.test_utils.TestShardRange.test_epoch_setter ... ok test.unit.common.test_utils.TestShardRange.test_find_ancestors test.unit.common.test_utils.TestShardRange.test_find_ancestors ... ok test.unit.common.test_utils.TestShardRange.test_find_root test.unit.common.test_utils.TestShardRange.test_find_root ... ok test.unit.common.test_utils.TestShardRange.test_increment_meta test.unit.common.test_utils.TestShardRange.test_increment_meta ... ok test.unit.common.test_utils.TestShardRange.test_is_child_of test.unit.common.test_utils.TestShardRange.test_is_child_of ... ok test.unit.common.test_utils.TestShardRange.test_make_path test.unit.common.test_utils.TestShardRange.test_make_path ... ok test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_min_max_bounds test.unit.common.test_utils.TestShardRange.test_min_max_bounds ... ok test.unit.common.test_utils.TestShardRange.test_name test.unit.common.test_utils.TestShardRange.test_name ... ok test.unit.common.test_utils.TestShardRange.test_name_validation test.unit.common.test_utils.TestShardRange.test_name_validation ... ok test.unit.common.test_utils.TestShardRange.test_repr test.unit.common.test_utils.TestShardRange.test_repr ... ok test.unit.common.test_utils.TestShardRange.test_resolve_state test.unit.common.test_utils.TestShardRange.test_resolve_state ... ok test.unit.common.test_utils.TestShardRange.test_row_count test.unit.common.test_utils.TestShardRange.test_row_count ... ok test.unit.common.test_utils.TestShardRange.test_set_deleted test.unit.common.test_utils.TestShardRange.test_set_deleted ... ok test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation ... ok test.unit.common.test_utils.TestShardRange.test_sort_key test.unit.common.test_utils.TestShardRange.test_sort_key ... ok test.unit.common.test_utils.TestShardRange.test_sort_key_order test.unit.common.test_utils.TestShardRange.test_sort_key_order ... ok test.unit.common.test_utils.TestShardRange.test_state_setter test.unit.common.test_utils.TestShardRange.test_state_setter ... ok test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_timestamp_setter test.unit.common.test_utils.TestShardRange.test_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_to_from_dict test.unit.common.test_utils.TestShardRange.test_to_from_dict ... ok test.unit.common.test_utils.TestShardRange.test_update_meta test.unit.common.test_utils.TestShardRange.test_update_meta ... ok test.unit.common.test_utils.TestShardRange.test_update_state test.unit.common.test_utils.TestShardRange.test_update_state ... ok test.unit.common.test_utils.TestShardRange.test_update_tombstones test.unit.common.test_utils.TestShardRange.test_update_tombstones ... ok test.unit.common.test_utils.TestSpliterator.test_big_input_string test.unit.common.test_utils.TestSpliterator.test_big_input_string ... ok test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries ... ok test.unit.common.test_utils.TestSpliterator.test_closing test.unit.common.test_utils.TestSpliterator.test_closing ... ok test.unit.common.test_utils.TestSpliterator.test_no_empty_strings test.unit.common.test_utils.TestSpliterator.test_no_empty_strings ... ok test.unit.common.test_utils.TestSpliterator.test_overlap test.unit.common.test_utils.TestSpliterator.test_overlap ... ok test.unit.common.test_utils.TestSpliterator.test_running_out test.unit.common.test_utils.TestSpliterator.test_running_out ... ok test.unit.common.test_utils.TestSpliterator.test_string test.unit.common.test_utils.TestSpliterator.test_string ... ok test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestRateLimiterBucket.test_bool test.unit.obj.test_updater.TestRateLimiterBucket.test_bool ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_len test.unit.obj.test_updater.TestRateLimiterBucket.test_len ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec1414ca598132a31cf91cd9737fb4ea', '0ab6ca7b-2e33-495e-9a1f-ef0458e14ba2-sda', '1717345413.19621', '1717345415', '0', '', '1717345415', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('0ab6ca7b-2e33-495e-9a1f-ef0458e14ba2-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.92399 seconds (0.34200/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgn_1kh19/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a5bc970f19a8860f432646de105ae5a5', '82c539b4-a4a1-4b72-99c4-402f151a005c-sda', '1717345419.63026', '1717345419', '1717345420', '', '1717345420', 1, 0) test DEBUG: /tmp/tmpgn_1kh19/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345422.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '82c539b4-a4a1-4b72-99c4-402f151a005c-sda') test DEBUG: /tmp/tmpgn_1kh19/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00990677s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a5bc970f19a8860f432646de105ae5a5', '82c539b4-a4a1-4b72-99c4-402f151a005c-sda', '1717345419.63026', '1717345419', '1717345420', '', '1717345420', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('82c539b4-a4a1-4b72-99c4-402f151a005c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09003 seconds (11.10696/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpbunlkcyq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a5bc970f19a8860f432646de105ae5a5', '64e19e94-bab7-4bd0-b343-8e72d27a6dcd-sda', '1717345419.75826', '1717345419', '1717345420', '', '1717345420', 1, 0) test DEBUG: /tmp/tmpbunlkcyq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a5bc970f19a8860f432646de105ae5a5', '64e19e94-bab7-4bd0-b343-8e72d27a6dcd-sda', '1717345419.75826', '1717345419', '1717345420', '', '1717345420', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('64e19e94-bab7-4bd0-b343-8e72d27a6dcd-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09085 seconds (11.00704/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2syoyv7a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '99b096df-b4f6-4076-95ed-f1935d9e5bbb-sda', '1717345419.89003', '1717345421', '1717345420', '', '1717345421', 0, 1) test DEBUG: /tmp/tmp2syoyv7a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '99b096df-b4f6-4076-95ed-f1935d9e5bbb-sda', '1717345419.89003', '1717345421', '1717345420', '', '1717345421', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('99b096df-b4f6-4076-95ed-f1935d9e5bbb-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04005 seconds (24.96752/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp80mzqje2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '098685e66b49886e387c814f67226e53', '241ef2b3-e2cc-4833-9b3c-e801f901f9b3-sda', '1717345419.98502', '1717345421', '1717345420', '', '1717345421', 1, 1) test DEBUG: /tmp/tmp80mzqje2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345423.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '241ef2b3-e2cc-4833-9b3c-e801f901f9b3-sda') test DEBUG: /tmp/tmp80mzqje2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.010653s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '098685e66b49886e387c814f67226e53', '241ef2b3-e2cc-4833-9b3c-e801f901f9b3-sda', '1717345419.98502', '1717345421', '1717345420', '', '1717345421', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('241ef2b3-e2cc-4833-9b3c-e801f901f9b3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07240 seconds (13.81194/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpauo0ud0e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8363347a-5915-4330-9c7d-cf3040df0481-sda', '1717345420.10191', '1717345422', '1717345421', '', '1717345422', 0, 1) test DEBUG: /tmp/tmpauo0ud0e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8363347a-5915-4330-9c7d-cf3040df0481-sda', '1717345420.10191', '1717345422', '1717345421', '', '1717345422', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8363347a-5915-4330-9c7d-cf3040df0481-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05125 seconds (19.51180/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpo48v4955/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'bc10c6c7c538a8358e417e85ed3446ab', '0d4eab98-9ed8-453e-ab85-e059e3f6f51b-sda', '1717345420.19042', '1717345422', '1717345421', '', '1717345422', 1, 1) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx9e87c47aae1f49e3ac3dc-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx248de6fbd49f4013ab2a9-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx94074cd331e3484f8aa7b-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx94074cd331e3484f8aa7b-00665c9c8b) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx94074cd331e3484f8aa7b-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx0c3362d1eeb944ce8a308-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx0c3362d1eeb944ce8a308-00665c9c8b) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx0c3362d1eeb944ce8a308-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txd30cfded4d1d4d93803c9-00665c9c8b) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txd30cfded4d1d4d93803c9-00665c9c8b) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txd30cfded4d1d4d93803c9-00665c9c8b) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txd30cfded4d1d4d93803c9-00665c9c8b) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txa71b0cffa4624caaabcf5-00665c9c8b) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txa71b0cffa4624caaabcf5-00665c9c8b) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txa71b0cffa4624caaabcf5-00665c9c8b) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txa71b0cffa4624caaabcf5-00665c9c8b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txcf372f755b2d4fcd84352-00665c9c8e) test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'Content-Md5': '2Oj7tEKhlNCBlzgaM7Czbw==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd8e8fbb442a194d08197381a33b0b36f', 'Content-Length': '230'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission ... ok test INFO: - - - [02/Jun/2024:16:23:29 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0009 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0058 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0049 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmps0ljefae/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0015 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0033 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:33 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0125 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0053 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0186 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0033 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error Problem occurred while reading builder file: /tmp/tmphmzot42h/tmpgorkrifs. Generic error occurred Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sdb_"" with 1.0 weight got id 4 Device d5r1z1-127.0.0.1:6200R127.0.0.1:6200/sdc_"" with 1.0 weight got id 5 Reassigned 192 (300.00%) partitions. Balance is now 109.38. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 109.38 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 371.88. Dispersion is now 2.08 ------------------------------------------------------------------------------- NOTE: Dispersion of 2.083333 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Note: using /tmp/tmpq053g_wk/tmpu0fcki8w.builder instead of /tmp/tmpq053g_wk/tmpu0fcki8w.ring.gz as builder file Reassigned 4 (6.25%) partitions. Balance is now 100.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 100.00 indicates you should push this ring, wait at least 24 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 24 (37.50%) partitions. Balance is now 1.04. Dispersion is now 0.00 Note: using /tmp/tmpx0wer55b/tmpsban9fk0.builder instead of /tmp/tmpx0wer55b/tmpsban9fk0.ring.gz as builder file Note: using /tmp/tmpokciuxg0/tmp1ojwgwwj.builder instead of /tmp/tmpokciuxg0/tmp1ojwgwwj.ring.gz as builder file Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: User tenant_id has system reader authorizing test DEBUG: tenant mismatch: AUTH_tenant_id2 != tenant_id test DEBUG: allow user with role(s) admin,swiftoperator as account admin test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx1e0d52b7875140c8931ca-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf080be5850a6419d9a72c-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txfc9a5fc8d6264afaace20-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txef0379bbbec74c7dbd26e-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txf60bf3212a00466c82780-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx4c3e94545ca84aca82b44-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe4292b6dad7547fdb6808-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx763cea38d74640b1a6fef-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx2a992769886f467494447-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx33a0998a621d4e8cb97ce-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txeb83e3e46b7244018a4c7-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txc04278578848424ca2009-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx000dde55c14e428cbb883-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txcab0de0f7a814e699f55a-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx12c749bd54684973bb4c2-00665c9ca9) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:23:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value ... ok test DEBUG: /tmp/tmpo48v4955/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'bc10c6c7c538a8358e417e85ed3446ab', '0d4eab98-9ed8-453e-ab85-e059e3f6f51b-sda', '1717345420.19042', '1717345422', '1717345421', '', '1717345422', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('0d4eab98-9ed8-453e-ab85-e059e3f6f51b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.20797 seconds (0.45290/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpzizzwr1v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '6f1bb5ca-ef4a-42fd-b10c-36b51c9f336f-sda', '1717345424.76087', '1717345427', '1717345426', '', '1717345427', 0, 0) test DEBUG: /tmp/tmpzizzwr1v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '6f1bb5ca-ef4a-42fd-b10c-36b51c9f336f-sda', '1717345424.76087', '1717345427', '1717345426', '', '1717345427', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('6f1bb5ca-ef4a-42fd-b10c-36b51c9f336f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04947 seconds (20.21523/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp4vsyw6ls/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '426dd14682bcc1273cf02f3e5bc2ba48', 'd3047307-90fe-4d7b-8ba3-77581a8fd21d-sda', '1717345424.87278', '1717345427', '1717345426', '', '1717345427', 1, 1) test DEBUG: /tmp/tmp4vsyw6ls/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345429.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'd3047307-90fe-4d7b-8ba3-77581a8fd21d-sda') test DEBUG: /tmp/tmp4vsyw6ls/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00912738s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '426dd14682bcc1273cf02f3e5bc2ba48', 'd3047307-90fe-4d7b-8ba3-77581a8fd21d-sda', '1717345424.87278', '1717345427', '1717345426', '', '1717345427', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d3047307-90fe-4d7b-8ba3-77581a8fd21d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09988 seconds (10.01213/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphnei6hgc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '30eb3224-4d72-4899-aa08-d7fd3d4a7f5a-sda', '1717345425.03021', '1717345428', '1717345427', '', '1717345428', 0, 0) test DEBUG: /tmp/tmphnei6hgc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '30eb3224-4d72-4899-aa08-d7fd3d4a7f5a-sda', '1717345425.03021', '1717345428', '1717345427', '', '1717345428', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('30eb3224-4d72-4899-aa08-d7fd3d4a7f5a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07915 seconds (12.63381/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpo_im4eyp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '937cb9926da694df0aef10416496ea7e', 'c6f9b3f8-9653-4c1b-be88-c00d7bd0084b-sda', '1717345425.16526', '1717345428', '1717345427', '', '1717345428', 1, 1) test DEBUG: /tmp/tmpo_im4eyp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '937cb9926da694df0aef10416496ea7e', 'c6f9b3f8-9653-4c1b-be88-c00d7bd0084b-sda', '1717345425.16526', '1717345428', '1717345427', '', '1717345428', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c6f9b3f8-9653-4c1b-be88-c00d7bd0084b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10939 seconds (9.14125/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwg3ot8gc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '651f1275-2bdc-490f-9005-8730be32b906-sda', '1717345425.36252', '1717345428', '1717345427', '', '1717345428', 0, 1) test DEBUG: /tmp/tmpwg3ot8gc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '651f1275-2bdc-490f-9005-8730be32b906-sda', '1717345425.36252', '1717345428', '1717345427', '', '1717345428', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('651f1275-2bdc-490f-9005-8730be32b906-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05394 seconds (18.53972/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpmymirsof/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '426dd14682bcc1273cf02f3e5bc2ba48', 'e48f66a7-a697-4fc4-9293-77bd6ac9377a-sda', '1717345425.46405', '1717345428', '1717345427', '', '1717345428', 1, 1) test DEBUG: /tmp/tmpmymirsof/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345429.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'e48f66a7-a697-4fc4-9293-77bd6ac9377a-sda') test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json ... ok test INFO: - - - [02/Jun/2024:16:23:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0007 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0004 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0305 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0076 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0074 "-" 3542007 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0166 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0004 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0126 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3542007 2 test DEBUG: Unlinked retiring db '/tmp/tmploytxe4v/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0160 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0171 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0047 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0027 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0161 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0129 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:23:42 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:23:42 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:23:42 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0004 "-" 3542007 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:23:42 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0099 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:23:42 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0004 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0144 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0005 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0040 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0006 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0229 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:42 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0039 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission ... ok test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe91bef0a8cc042d3a932f-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa55ffe3055b243b78d019-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txd84cfc05192e47aa99e37-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb6823f86474f4220ae254-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd26d2599592f4aa4be221-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txcc21eac081e145e1baa20-00665c9ca9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx742c9c86aec6400e8eca2-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txbe710d26d95c492eb5138-00665c9ca9) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx00b7769631874493a90d9-00665c9ca9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txb5d2bebdb5754fbe80334-00665c9ca9) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: tx7cacfd9a61a2488faa225-00665c9ca9) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txcf9d236956e74d9091ed5-00665c9ca9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx24f4f1cb30d84d5995d43-00665c9caa) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txf1236ec0b7674eafaa95b-00665c9caa) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: tx3f9457c5ecb4460d98613-00665c9caa) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx83689b727bb5441883067-00665c9caa) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx53d8d924cff94612898d7-00665c9caa) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: txce0f42403d16459caf35d-00665c9caa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx34198d7a436f4931a1d1d-00665c9caa) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx85e19be8bf1b4ff9869c2-00665c9caa) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7913b3f961ca4be4ae10b-00665c9caa) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txeb3585c6d6c04c279ed31-00665c9caa) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbb6cc79609424637b42c0-00665c9caa) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txdefe5cd8d6dd414289cde-00665c9caa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: txd1e8f3f2d31640f7b44c2-00665c9caa) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd1e8f3f2d31640f7b44c2-00665c9caa) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 501 Not Implemented test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txdb554da056e244ec95df1-00665c9c8e) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txdb554da056e244ec95df1-00665c9c8e) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txdb554da056e244ec95df1-00665c9c8e) test ERROR: Account HEAD returning 503 for [] (txn: txdb554da056e244ec95df1-00665c9c8e) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txac7187557f9c428b9ef57-00665c9c91) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txac7187557f9c428b9ef57-00665c9c91) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txac7187557f9c428b9ef57-00665c9c91) test ERROR: Account HEAD returning 503 for [] (txn: txac7187557f9c428b9ef57-00665c9c91) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx61cbeb584ddf47f894ec1-00665c9c91) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txa445a966cc7f4040a1a12-00665c9c91) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txa445a966cc7f4040a1a12-00665c9c91) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa445a966cc7f4040a1a12-00665c9c91) test ERROR: Account HEAD returning 503 for [] (txn: txa445a966cc7f4040a1a12-00665c9c91) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx27a120322e0f4d7b94ba5-00665c9c91) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx27a120322e0f4d7b94ba5-00665c9c91) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx27a120322e0f4d7b94ba5-00665c9c91) test ERROR: Account HEAD returning 503 for [] (txn: tx27a120322e0f4d7b94ba5-00665c9c91) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txacce6d3e5fdc46a99d648-00665c9c91) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: tx7d51ccd4706e41f5af3cc-00665c9c94) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: tx7d51ccd4706e41f5af3cc-00665c9c94) test INFO: Client disconnected on read of '/a/c' (txn: tx7d51ccd4706e41f5af3cc-00665c9c94) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txddb3e77cc1a740d1a241d-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx53184657dd7343b483fb4-00665c9c94) test INFO: Client disconnected on read of '/a/c' (txn: tx53184657dd7343b483fb4-00665c9c94) test INFO: Client disconnected on read of '/a/c/' (txn: tx53184657dd7343b483fb4-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx51b87d0e93874bd1866e0-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx51b87d0e93874bd1866e0-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx51b87d0e93874bd1866e0-00665c9c94) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx51b87d0e93874bd1866e0-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txc1ffb185f665428a84640-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txc1ffb185f665428a84640-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txc1ffb185f665428a84640-00665c9c94) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txc1ffb185f665428a84640-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx4ec5282489824659a9675-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx4ec5282489824659a9675-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx4ec5282489824659a9675-00665c9c94) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx4ec5282489824659a9675-00665c9c94) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx241baaff83374017821b1-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx241baaff83374017821b1-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx241baaff83374017821b1-00665c9c97) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx241baaff83374017821b1-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx7af4f36f39084292839d4-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx7af4f36f39084292839d4-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx7af4f36f39084292839d4-00665c9c97) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx7af4f36f39084292839d4-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txa6b4b6723db648aeb1a29-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txa6b4b6723db648aeb1a29-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txa6b4b6723db648aeb1a29-00665c9c97) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txa6b4b6723db648aeb1a29-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txb029b63ecafd43148e866-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txb029b63ecafd43148e866-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txb029b63ecafd43148e866-00665c9c97) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txb029b63ecafd43148e866-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txc5c7d411d6904215b232a-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txc5c7d411d6904215b232a-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txc5c7d411d6904215b232a-00665c9c97) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txc5c7d411d6904215b232a-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txc8ac0dd6981c426497e57-00665c9c97) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: txc8ac0dd6981c426497e57-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txc8ac0dd6981c426497e57-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txc8ac0dd6981c426497e57-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txc8ac0dd6981c426497e57-00665c9c97) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txc8ac0dd6981c426497e57-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx04eaeb397b404d3c882a8-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx04eaeb397b404d3c882a8-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx04eaeb397b404d3c882a8-00665c9c97) test ERROR: Container GET returning 503 for [503, 503] (txn: tx04eaeb397b404d3c882a8-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test ERROR: Container GET returning 503 for [503, 503] (txn: tx970b3ffdfcdf4325ad563-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txc59dc314644548d19175d-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txc59dc314644548d19175d-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txc59dc314644548d19175d-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: txc59dc314644548d19175d-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx9553c08702c44a98823ed-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx9553c08702c44a98823ed-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx9553c08702c44a98823ed-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: tx9553c08702c44a98823ed-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txe83fef45073e4627b5767-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txe83fef45073e4627b5767-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txe83fef45073e4627b5767-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: txe83fef45073e4627b5767-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7dd2001ce65e4734b0d70-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7dd2001ce65e4734b0d70-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7dd2001ce65e4734b0d70-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: tx7dd2001ce65e4734b0d70-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx5f1e9dc48f894267bcb22-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx5f1e9dc48f894267bcb22-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx5f1e9dc48f894267bcb22-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: tx5f1e9dc48f894267bcb22-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx14d0821706dd4fba8aff7-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx14d0821706dd4fba8aff7-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx14d0821706dd4fba8aff7-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: tx14d0821706dd4fba8aff7-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txcce0a6e2bbbb4cbeaeed6-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txcce0a6e2bbbb4cbeaeed6-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txcce0a6e2bbbb4cbeaeed6-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: txcce0a6e2bbbb4cbeaeed6-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txfcfa8e3178464febb3ac2-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txfcfa8e3178464febb3ac2-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txfcfa8e3178464febb3ac2-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: txfcfa8e3178464febb3ac2-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txfabac11845074b49ba11d-00665c9c97) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txfabac11845074b49ba11d-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txfabac11845074b49ba11d-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: txfabac11845074b49ba11d-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txcb4879785fcb43568b0e2-00665c9c97) test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error ... ok test DEBUG: /tmp/tmpmymirsof/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0107563s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '426dd14682bcc1273cf02f3e5bc2ba48', 'e48f66a7-a697-4fc4-9293-77bd6ac9377a-sda', '1717345425.46405', '1717345428', '1717345427', '', '1717345428', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e48f66a7-a697-4fc4-9293-77bd6ac9377a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.77037 seconds (0.36096/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgsojc2ia/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e8cff7c6-4cc0-436b-b405-c1e2fd120e5a-sda', '1717345428.27860', '1717345431', '1717345430', '', '1717345431', 0, 1) test DEBUG: /tmp/tmpgsojc2ia/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e8cff7c6-4cc0-436b-b405-c1e2fd120e5a-sda', '1717345428.27860', '1717345431', '1717345430', '', '1717345431', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e8cff7c6-4cc0-436b-b405-c1e2fd120e5a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05355 seconds (18.67327/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpuzn9dcce/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0b4ebfbd33dfb685bc8888f2c0030cb8', 'e73ff9a8-fcc6-419a-946f-db5b4d4f9352-sda', '1717345428.37996', '1717345431', '1717345430', '', '1717345431', 1, 1) test DEBUG: /tmp/tmpuzn9dcce/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0b4ebfbd33dfb685bc8888f2c0030cb8', 'e73ff9a8-fcc6-419a-946f-db5b4d4f9352-sda', '1717345428.37996', '1717345431', '1717345430', '', '1717345431', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e73ff9a8-fcc6-419a-946f-db5b4d4f9352-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07277 seconds (13.74238/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp0yepktrr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'cb3d94cd-1b86-49f9-9a92-cd4fa0063a4c-sda', '1717345431.21046', '1717345435', '1717345434', '', '1717345435', 0, 1) test DEBUG: /tmp/tmp0yepktrr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'cb3d94cd-1b86-49f9-9a92-cd4fa0063a4c-sda', '1717345431.21046', '1717345435', '1717345434', '', '1717345435', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('cb3d94cd-1b86-49f9-9a92-cd4fa0063a4c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05999 seconds (16.67050/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp9ser7341/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'dc857cb4ca68c98c685bc70f0af867b7', 'dc775005-9520-4b6b-9a0c-87c937b969b3-sda', '1717345431.32266', '1717345435', '1717345434', '', '1717345435', 1, 1) test DEBUG: /tmp/tmp9ser7341/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345436.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'dc775005-9520-4b6b-9a0c-87c937b969b3-sda') test DEBUG: /tmp/tmp9ser7341/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0108142s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'dc857cb4ca68c98c685bc70f0af867b7', 'dc775005-9520-4b6b-9a0c-87c937b969b3-sda', '1717345431.32266', '1717345435', '1717345434', '', '1717345435', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('dc775005-9520-4b6b-9a0c-87c937b969b3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07450 seconds (13.42206/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpqenu32ms/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7021219e-4541-48c9-bb74-66fcdc2679a6-sda', '1717345431.45355', '1717345435', '1717345434', '', '1717345435', 0, 1) test DEBUG: /tmp/tmpqenu32ms/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7021219e-4541-48c9-bb74-66fcdc2679a6-sda', '1717345431.45355', '1717345435', '1717345434', '', '1717345435', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7021219e-4541-48c9-bb74-66fcdc2679a6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04334 seconds (23.07384/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpt948ulg0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'dc857cb4ca68c98c685bc70f0af867b7', 'f2caeaf6-6f8b-4f6d-876f-0f2a011848d0-sda', '1717345431.55190', '1717345435', '1717345434', '', '1717345435', 1, 0) test DEBUG: /tmp/tmpt948ulg0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'dc857cb4ca68c98c685bc70f0af867b7', 'f2caeaf6-6f8b-4f6d-876f-0f2a011848d0-sda', '1717345431.55190', '1717345435', '1717345434', '', '1717345435', 1, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 ... ok test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txcb4879785fcb43568b0e2-00665c9c97) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txcb4879785fcb43568b0e2-00665c9c97) test ERROR: Container GET returning 503 for [] (txn: txcb4879785fcb43568b0e2-00665c9c97) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa0620e4e3a4140db817c2-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa0620e4e3a4140db817c2-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa0620e4e3a4140db817c2-00665c9c9a) test ERROR: Container GET returning 503 for [] (txn: txa0620e4e3a4140db817c2-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txc676f4aa9b4640abbe42e-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txc676f4aa9b4640abbe42e-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txc676f4aa9b4640abbe42e-00665c9c9a) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txc676f4aa9b4640abbe42e-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7d86c5b024d944308f04e-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7d86c5b024d944308f04e-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7d86c5b024d944308f04e-00665c9c9a) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx7d86c5b024d944308f04e-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx02087bf07f9b463cb29e2-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx02087bf07f9b463cb29e2-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx02087bf07f9b463cb29e2-00665c9c9a) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx02087bf07f9b463cb29e2-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txe9c0493c026140f0a37f3-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txe9c0493c026140f0a37f3-00665c9c9a) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txe9c0493c026140f0a37f3-00665c9c9a) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txe9c0493c026140f0a37f3-00665c9c9a) test INFO: Client disconnected on read of '/a/c' (txn: tx43a1171a8b7345048a928-00665c9c9a) test INFO: Client disconnected on read of '/a/c/' (txn: tx43a1171a8b7345048a928-00665c9c9a) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txb9f997e127274b2aa18e1-00665c9c9a) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txb9f997e127274b2aa18e1-00665c9c9a) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txb9f997e127274b2aa18e1-00665c9c9a) test ERROR: Account HEAD returning 503 for [] (txn: txb9f997e127274b2aa18e1-00665c9c9a) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx5c71efdca2ba42f2b7e65-00665c9c9a) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx5c71efdca2ba42f2b7e65-00665c9c9a) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx5c71efdca2ba42f2b7e65-00665c9c9a) test ERROR: Account HEAD returning 503 for [] (txn: tx5c71efdca2ba42f2b7e65-00665c9c9a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx4ff629f547c745ae82f40-00665c9c9d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx7e413cdb0c674daa93ae5-00665c9c9d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx52e14e111f4242b585fb5-00665c9c9d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission ... ok test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0186 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0141 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0180 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0125 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0091 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0138 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0108 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3542007 3 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542007 3 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0091 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3542007 4 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3542007 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0121 "-" 3542007 4 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0109 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0137 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0231 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0018 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0016 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0126 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0113 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0129 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0115 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0022 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0141 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0113 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0097 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:23:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0014 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0119 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0004 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:48 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0109 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f2caeaf6-6f8b-4f6d-876f-0f2a011848d0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.29945 seconds (0.43489/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp7jyc3qz4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd4fa6e0f-a441-41a2-95c7-ba13afcf2d8b-sda', '1717345436.95187', '1717345441', '1717345440', '', '1717345441', 0, 0) test DEBUG: /tmp/tmp7jyc3qz4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd4fa6e0f-a441-41a2-95c7-ba13afcf2d8b-sda', '1717345436.95187', '1717345441', '1717345440', '', '1717345441', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d4fa6e0f-a441-41a2-95c7-ba13afcf2d8b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04429 seconds (22.57789/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkfan0zvj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e5a614a6e349c956ac3274484fe0dff7', 'f1d86e21-d435-41e7-8880-34f31ca90ed7-sda', '1717345437.05914', '1717345442', '1717345441', '', '1717345442', 1, 0) test DEBUG: /tmp/tmpkfan0zvj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345443.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'f1d86e21-d435-41e7-8880-34f31ca90ed7-sda') test DEBUG: /tmp/tmpkfan0zvj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00843501s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e5a614a6e349c956ac3274484fe0dff7', 'f1d86e21-d435-41e7-8880-34f31ca90ed7-sda', '1717345437.05914', '1717345442', '1717345441', '', '1717345442', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f1d86e21-d435-41e7-8880-34f31ca90ed7-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07527 seconds (13.28571/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpza2cxrld/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b135ca65-07b7-4af6-be40-d398affe4023-sda', '1717345437.19921', '1717345442', '1717345441', '', '1717345442', 0, 0) test DEBUG: /tmp/tmpza2cxrld/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b135ca65-07b7-4af6-be40-d398affe4023-sda', '1717345437.19921', '1717345442', '1717345441', '', '1717345442', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b135ca65-07b7-4af6-be40-d398affe4023-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06130 seconds (16.31197/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwwqfl4av/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e5a614a6e349c956ac3274484fe0dff7', 'a1e11de5-69f0-4f37-8d1d-76b3f3eb1fa1-sda', '1717345437.33035', '1717345442', '1717345441', '', '1717345442', 1, 0) test DEBUG: /tmp/tmpwwqfl4av/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e5a614a6e349c956ac3274484fe0dff7', 'a1e11de5-69f0-4f37-8d1d-76b3f3eb1fa1-sda', '1717345437.33035', '1717345442', '1717345441', '', '1717345442', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a1e11de5-69f0-4f37-8d1d-76b3f3eb1fa1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08470 seconds (11.80621/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '5c5c96ce-7744-4fb9-ae40-413b204cb44c-sda', '1717345437.48061', '1717345437.00000', '0', '', '1717345437.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '5c5c96ce-7744-4fb9-ae40-413b204cb44c-sda', '1717345437.48061', '1717345438.00000', '0', '', '1717345437.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4e6f5f8567195f96d878de76588e64ec', '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda', '1717345437.52534', '1717345437.52187', '0', '', '1717345437.52187', 11, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345430.59187', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission ... ok test INFO: - - - [02/Jun/2024:16:23:48 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:53 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0037 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0144 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0002 "-" 3542007 nada test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0002 "-" 3542007 999 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_legacy" 201 - "-" "-" "-" 0.0225 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c_legacy" 204 - "-" "-" "-" 0.0016 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0106 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c_one" 204 - "-" "-" "-" 0.0016 "-" 3542007 1 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0100 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c_two" 204 - "-" "-" "-" 0.0017 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0127 "-" 3542007 3 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c_three" 204 - "-" "-" "-" 0.0017 "-" 3542007 3 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c_four" 201 - "-" "-" "-" 0.0110 "-" 3542007 4 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c_four" 204 - "-" "-" "-" 0.0066 "-" 3542007 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0002 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0115 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0206 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0032 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0122 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0004 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0139 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0059 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0032 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0030 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0137 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0110 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0111 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0056 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0112 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0103 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0147 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0019 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0111 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0097 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0107 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0097 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:23:56 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o_0', 'created_at': '1717345437.54800', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o_1', 'created_at': '1717345437.54896', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o_2', 'created_at': '1717345437.54921', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o_3', 'created_at': '1717345437.54944', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o_4', 'created_at': '1717345437.54967', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o_5', 'created_at': '1717345437.54990', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o_6', 'created_at': '1717345437.55013', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o_7', 'created_at': '1717345437.55036', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o_8', 'created_at': '1717345437.55059', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o_9', 'created_at': '1717345437.55091', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing_0', 'created_at': '1717345437.55120', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 12, 'name': '/a/c/o_missing_1', 'created_at': '1717345437.59165', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 13, 'name': '/a/c/o_missing_2', 'created_at': '1717345437.60927', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 14, 'name': '/a/c/o_missing_3', 'created_at': '1717345437.63835', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 15, 'name': '/a/c/o_missing_4', 'created_at': '1717345437.65470', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 16, 'name': '/a/c/o_missing_5', 'created_at': '1717345437.70111', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 17, 'name': '/a/c/o_missing_6', 'created_at': '1717345437.72441', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 18, 'name': '/a/c/o_missing_7', 'created_at': '1717345437.74855', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 19, 'name': '/a/c/o_missing_8', 'created_at': '1717345437.76451', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 20, 'name': '/a/c/o_missing_9', 'created_at': '1717345437.78219', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (21, 'd73865f42a2bad3494f671b4d1903b98', '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda', '1717345437.52534', '1717345437.52187', '0', '', '1717345437.52187', 21, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': '/a/c/o_missing_10', 'created_at': '1717345437.79589', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8830e067-322b-4a78-bdb7-8c6dba181ca7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp3lr8ou58/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd7a55211-6435-45aa-9054-6f65dbf07414-sda', '1717345439.89737', '1717345440', '0', '', '1717345440', 0, 0) test DEBUG: /tmp/tmp3lr8ou58/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd7a55211-6435-45aa-9054-6f65dbf07414-sda', '1717345439.89737', '1717345440', '0', '', '1717345440', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d7a55211-6435-45aa-9054-6f65dbf07414-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04481 seconds (22.31779/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6dvcsuey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '073b62c081a0c83567a46d0a5cb30bd8', 'a4f3252b-338d-4bb3-a446-44b16fbb725e-sda', '1717345439.96879', '1717345440', '0', '', '1717345440', 1, 0) test DEBUG: /tmp/tmp6dvcsuey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345441.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a4f3252b-338d-4bb3-a446-44b16fbb725e-sda') test DEBUG: /tmp/tmp6dvcsuey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0105586s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '073b62c081a0c83567a46d0a5cb30bd8', 'a4f3252b-338d-4bb3-a446-44b16fbb725e-sda', '1717345439.96879', '1717345440', '0', '', '1717345440', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a4f3252b-338d-4bb3-a446-44b16fbb725e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp6dvcsuey/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10187 seconds (9.81683/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 400 Bad Request test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345433.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345433.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345433.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345433.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error ... ok test INFO: - - - [02/Jun/2024:16:23:56 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0099 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0163 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0031 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0027 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0123 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0089 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0038 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0114 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0106 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0039 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0115 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0002 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0107 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0003 "-" 3542007 0 test INFO: - - - [02/Jun/2024:16:24:01 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0108 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0007 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0005 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:02 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0097 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3542007 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0381 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:02 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0125 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:02 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0099 "-" 3542007 - test INFO: - - - [02/Jun/2024:16:24:02 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0117 "-" 3542007 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:02 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0111 "-" 3542007 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types ... ok test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestSweepStats.test_copy test.unit.obj.test_updater.TestSweepStats.test_copy ... ok test.unit.obj.test_updater.TestSweepStats.test_reset test.unit.obj.test_updater.TestSweepStats.test_reset ... ok test.unit.obj.test_updater.TestSweepStats.test_since test.unit.obj.test_updater.TestSweepStats.test_since ... ok test.unit.obj.test_updater.TestSweepStats.test_str test.unit.obj.test_updater.TestSweepStats.test_str ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345437.97486', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:24:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0202 "-" 3542007 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:24:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:06 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0008 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:06 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3542007 2 test INFO: - - - [02/Jun/2024:16:24:06 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0112 "-" 3542007 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txfc1d6564ef864464999c8-00665c9cab) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txfc1d6564ef864464999c8-00665c9cab) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txfc1d6564ef864464999c8-00665c9cab) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: txfc1d6564ef864464999c8-00665c9cab) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txfc1d6564ef864464999c8-00665c9cab) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txfc1d6564ef864464999c8-00665c9cab) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txfc1d6564ef864464999c8-00665c9cab) test INFO: Client disconnected on read of '/a/c' (txn: txfc1d6564ef864464999c8-00665c9cab) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2fcbfbbddb204542848d0-00665c9cab) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2fcbfbbddb204542848d0-00665c9cab) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx2fcbfbbddb204542848d0-00665c9cab) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: txfbc7181fd451428cbd1f8-00665c9cab) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: txef376da82b0646b399fa1-00665c9cab) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx166a66ffeb214c4da2ddc-00665c9cab) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: txeec9271048b54786bf8c5-00665c9cab) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txca9b0a1965ba4c3784bae-00665c9cab) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx836f0a5f9f5341f8b52cc-00665c9cab) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx836f0a5f9f5341f8b52cc-00665c9cab) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: tx315fc318d35f4e67bab1c-00665c9cab) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: txdf4a60b352224a1a9ca8f-00665c9cab) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx46525797b2874a94b68ad-00665c9cab) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: txc7b2623bc70a49d688665-00665c9cab) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txe8c6d97272724b89bc2fb-00665c9cab) test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin ... ok test DEBUG: Replicating db /tmp/tmp6dvcsuey/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '6d34489036dd88d831435a8784cfe360', '83e51fc0-9661-42bc-982b-b806c596d817-sda', '1717345440.04922', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('83e51fc0-9661-42bc-982b-b806c596d817-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '6d34489036dd88d831435a8784cfe360', '83e51fc0-9661-42bc-982b-b806c596d817-sda', '1717345440.04922', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('83e51fc0-9661-42bc-982b-b806c596d817-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpc2sde22s/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '6a7d0a06-c2c5-4bee-b4cd-4f56d9826a00-sda', '1717345442.42445', '1717345443', '0', '', '1717345443', 0, 1) test DEBUG: /tmp/tmpc2sde22s/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '6a7d0a06-c2c5-4bee-b4cd-4f56d9826a00-sda', '1717345442.42445', '1717345443', '0', '', '1717345443', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('6a7d0a06-c2c5-4bee-b4cd-4f56d9826a00-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04180 seconds (23.92501/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp54xjal6q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1211f31981a7682f4b4ad2d35c104c6b', '9c7218b0-eb14-4796-b6a3-a73bb939eaea-sda', '1717345442.50004', '1717345443', '0', '', '1717345443', 1, 0) test DEBUG: /tmp/tmp54xjal6q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1211f31981a7682f4b4ad2d35c104c6b', '9c7218b0-eb14-4796-b6a3-a73bb939eaea-sda', '1717345442.50004', '1717345443', '0', '', '1717345443', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('9c7218b0-eb14-4796-b6a3-a73bb939eaea-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp54xjal6q/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07481 seconds (13.36700/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp54xjal6q/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '5199651a37eb42f1452cdb598b1aabaa', '51c14389-4941-4bb6-b3ba-4be7087cc728-sda', '1717345442.56367', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('51c14389-4941-4bb6-b3ba-4be7087cc728-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '5199651a37eb42f1452cdb598b1aabaa', '51c14389-4941-4bb6-b3ba-4be7087cc728-sda', '1717345442.56367', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('51c14389-4941-4bb6-b3ba-4be7087cc728-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxfmd5in9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '626a5623-1d04-465c-a49a-1c188b7afd09-sda', '1717345444.20123', '1717345445', '1717345446', '', '1717345446', 0, 1) test DEBUG: /tmp/tmpxfmd5in9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '626a5623-1d04-465c-a49a-1c188b7afd09-sda', '1717345444.20123', '1717345445', '1717345446', '', '1717345446', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('626a5623-1d04-465c-a49a-1c188b7afd09-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03448 seconds (29.00055/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwnh1ulq5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9d73980b98a5bdad14362d90fee41743', '45b93c03-5981-4174-b2c7-289d18d4a7b9-sda', '1717345444.27018', '1717345445', '1717345446', '', '1717345446', 1, 1) test DEBUG: /tmp/tmpwnh1ulq5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345447.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '45b93c03-5981-4174-b2c7-289d18d4a7b9-sda') test DEBUG: /tmp/tmpwnh1ulq5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00832224s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9d73980b98a5bdad14362d90fee41743', '45b93c03-5981-4174-b2c7-289d18d4a7b9-sda', '1717345444.27018', '1717345445', '1717345446', '', '1717345446', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('45b93c03-5981-4174-b2c7-289d18d4a7b9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpwnh1ulq5/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08467 seconds (11.81020/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpwnh1ulq5/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'bc92d70f94691d76ad9578fb7ab58c12', '7ec59e48-9ed5-4e75-a38c-d9ac1bb7dff2-sda', '1717345444.34834', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('7ec59e48-9ed5-4e75-a38c-d9ac1bb7dff2-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:23:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345444.51148', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy ... ok test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx141cdbe834f04f1f9e2e9-00665c9cab) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx141cdbe834f04f1f9e2e9-00665c9cab) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: tx8fa6fdeea6e6459a8cc43-00665c9cad) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txfc991c6f6fbb43fd808ec-00665c9cad) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx7c774d685c534c8dbfdbb-00665c9cad) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: tx47be7b31e8db4865af53e-00665c9cad) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txd93a71259fa44213b6b37-00665c9cad) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txf3ef2ba000b84e8d8e398-00665c9cad) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txf3ef2ba000b84e8d8e398-00665c9cad) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txfaf996b2f4934420a6535-00665c9cb0) test INFO: Client disconnected on read of '/a/c' (txn: txc3b55a4e3f644cffbf689-00665c9cb0) test INFO: Client disconnected on read of '/a/c' (txn: tx293b27a2252740288e034-00665c9cb0) test INFO: Client disconnected on read of '/a/c' (txn: tx293b27a2252740288e034-00665c9cb0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response ... ok REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'bc92d70f94691d76ad9578fb7ab58c12', '7ec59e48-9ed5-4e75-a38c-d9ac1bb7dff2-sda', '1717345444.34834', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('7ec59e48-9ed5-4e75-a38c-d9ac1bb7dff2-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpiylcvv4p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4e4c925c-42f4-4aef-b8c3-1c0991f70862-sda', '1717345445.99679', '1717345446', '1717345447', '', '1717345447', 0, 0) test DEBUG: /tmp/tmpiylcvv4p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4e4c925c-42f4-4aef-b8c3-1c0991f70862-sda', '1717345445.99679', '1717345446', '1717345447', '', '1717345447', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('4e4c925c-42f4-4aef-b8c3-1c0991f70862-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03895 seconds (25.67484/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp56fretb8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6482137c2998abf052927b0c8390841d', '04663474-03cf-4464-9306-4c7b8fb6030f-sda', '1717345446.06802', '1717345447', '1717345448', '', '1717345448', 1, 0) test DEBUG: /tmp/tmp56fretb8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6482137c2998abf052927b0c8390841d', '04663474-03cf-4464-9306-4c7b8fb6030f-sda', '1717345446.06802', '1717345447', '1717345448', '', '1717345448', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04663474-03cf-4464-9306-4c7b8fb6030f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp56fretb8/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06299 seconds (15.87576/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp56fretb8/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e007f05494f95fb2d738a08a9a908bb7', 'b11342f6-9e34-48fd-9932-d0bd9b57c5ef-sda', '1717345446.12833', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('b11342f6-9e34-48fd-9932-d0bd9b57c5ef-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e007f05494f95fb2d738a08a9a908bb7', 'b11342f6-9e34-48fd-9932-d0bd9b57c5ef-sda', '1717345446.12833', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('b11342f6-9e34-48fd-9932-d0bd9b57c5ef-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgmdt_fpr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f7d5e58b-e4fe-4e10-b8be-f7d463a9c803-sda', '1717345447.91168', '1717345447', '1717345448', '', '1717345448', 0, 1) test DEBUG: /tmp/tmpgmdt_fpr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f7d5e58b-e4fe-4e10-b8be-f7d463a9c803-sda', '1717345447.91168', '1717345447', '1717345448', '', '1717345448', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f7d5e58b-e4fe-4e10-b8be-f7d463a9c803-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03778 seconds (26.46809/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp031amlap/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd0cee3b6-8b1e-43c4-b217-2071c21840b9-sda', '1717345447.98195', '1717345447', '1717345448', '', '1717345448', 0, 0) test DEBUG: /tmp/tmp031amlap/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd0cee3b6-8b1e-43c4-b217-2071c21840b9-sda', '1717345447.98195', '1717345447', '1717345448', '', '1717345448', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d0cee3b6-8b1e-43c4-b217-2071c21840b9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03949 seconds (25.32220/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpop9e7kvh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2d665b01-3f87-4965-a630-c026b700c03a-sda', '1717345448.07205', '1717345450', '0', '', '1717345450', 0, 0) test DEBUG: /tmp/tmpop9e7kvh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2d665b01-3f87-4965-a630-c026b700c03a-sda', '1717345448.07205', '1717345450', '0', '', '1717345450', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2d665b01-3f87-4965-a630-c026b700c03a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04389 seconds (22.78220/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpvvnj4f2g/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec9d17b99ae3cc2712079fcf144d87f1', 'ca303988-1293-48db-ab59-6c2f04c0543e-sda', '1717345448.14871', '1717345450', '0', '', '1717345450', 1, 1) test DEBUG: /tmp/tmpvvnj4f2g/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345444.53274', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345447.06941', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy ... ok test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (201, 404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (201, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec9d17b99ae3cc2712079fcf144d87f1', 'ca303988-1293-48db-ab59-6c2f04c0543e-sda', '1717345448.14871', '1717345450', '0', '', '1717345450', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ca303988-1293-48db-ab59-6c2f04c0543e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpvvnj4f2g/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.47220 seconds (0.40450/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpvvnj4f2g/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '29b6e1cc67a67ad27efd4913798b5575', '821454a9-90bd-4c68-a0a0-41294d24629b-sda', '1717345450.61333', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('821454a9-90bd-4c68-a0a0-41294d24629b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '29b6e1cc67a67ad27efd4913798b5575', '821454a9-90bd-4c68-a0a0-41294d24629b-sda', '1717345450.61333', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('821454a9-90bd-4c68-a0a0-41294d24629b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '109a573d-02b3-459a-9259-3e3aa2f39470-sda', '1717345452.89442', '1717345452.88961', '0', '', '1717345452.88961', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpbgyut617/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2704251e-33ae-43ee-92ba-2923f368907c-sda', '1717345452.91686', '1717345452.91238', '0', '', '1717345452.91238', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2704251e-33ae-43ee-92ba-2923f368907c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2704251e-33ae-43ee-92ba-2923f368907c-sda', '1717345452.91686', '1717345452.91238', '0', '', '1717345452.91238', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2704251e-33ae-43ee-92ba-2923f368907c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05068 seconds (19.73146/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2065078ff42f85ef6e6ccabe5e21967e', '247fdbd7-edb7-4b31-954d-ec34519f0949-sda', '1717345452.99487', '1717345452.98989', '0', '', '1717345452.98989', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('247fdbd7-edb7-4b31-954d-ec34519f0949-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'c4ca6f04b64cc1d55f4e7db4872375e3', '17bad78c-6fa9-4cd2-a13c-0c2df6e3807b-sda', '1717345453.06757', '1717345453.06396', '0', '', '1717345453.06396', 11, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o_0', 'created_at': '1717345453.08417', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c/o_1', 'created_at': '1717345453.08513', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c/o_2', 'created_at': '1717345453.08537', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c/o_3', 'created_at': '1717345453.08560', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c/o_4', 'created_at': '1717345453.08589', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c/o_5', 'created_at': '1717345453.08612', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c/o_6', 'created_at': '1717345453.08635', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c/o_7', 'created_at': '1717345453.08657', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c/o_8', 'created_at': '1717345453.08687', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c/o_9', 'created_at': '1717345453.08712', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing', 'created_at': '1717345453.08737', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '17bad78c-6fa9-4cd2-a13c-0c2df6e3807b-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpegcbss_s/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4debe211-6abe-49d6-b653-d6d68c1453aa-sda', '1717345453.18025', '1717345456', '0', '', '1717345456', 0, 1) test DEBUG: /tmp/tmpegcbss_s/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4debe211-6abe-49d6-b653-d6d68c1453aa-sda', '1717345453.18025', '1717345456', '0', '', '1717345456', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('4debe211-6abe-49d6-b653-d6d68c1453aa-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06338 seconds (15.77705/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpzelulu2t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '8a8c737e1330eb485b3471e86133a7df', '3ec4f00d-47f6-4099-86aa-5b7f6a743f10-sda', '1717345453.28901', '1717345456', '0', '', '1717345456', 1, 1) test DEBUG: /tmp/tmpzelulu2t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345457.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '3ec4f00d-47f6-4099-86aa-5b7f6a743f10-sda') test DEBUG: /tmp/tmpzelulu2t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00904632s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345449.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345456.98053', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:24:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345456.99587', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '8a8c737e1330eb485b3471e86133a7df', '3ec4f00d-47f6-4099-86aa-5b7f6a743f10-sda', '1717345453.28901', '1717345456', '0', '', '1717345456', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3ec4f00d-47f6-4099-86aa-5b7f6a743f10-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpzelulu2t/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.84944 seconds (0.54070/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpzelulu2t/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '70322367c8f56c03413aeb93fdc4f98a', '9f005bf3-4114-4909-9410-80c7fee971d9-sda', '1717345455.13304', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9f005bf3-4114-4909-9410-80c7fee971d9-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '70322367c8f56c03413aeb93fdc4f98a', '9f005bf3-4114-4909-9410-80c7fee971d9-sda', '1717345455.13304', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9f005bf3-4114-4909-9410-80c7fee971d9-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpvoadiw7t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '965898d4-cd24-4405-981d-2a3067cd9ce5-sda', '1717345455.21595', '1717345458', '0', '', '1717345458', 0, 0) test DEBUG: /tmp/tmpvoadiw7t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '965898d4-cd24-4405-981d-2a3067cd9ce5-sda', '1717345455.21595', '1717345458', '0', '', '1717345458', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('965898d4-cd24-4405-981d-2a3067cd9ce5-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03941 seconds (25.37475/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpn_ztbu6u/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'cc64d7a4c98d68196924af68cfe98b1a', '1623c6fb-d5c9-42c1-860c-de17ba13a4ce-sda', '1717345455.29597', '1717345458', '0', '', '1717345458', 1, 0) test DEBUG: /tmp/tmpn_ztbu6u/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'cc64d7a4c98d68196924af68cfe98b1a', '1623c6fb-d5c9-42c1-860c-de17ba13a4ce-sda', '1717345455.29597', '1717345458', '0', '', '1717345458', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('1623c6fb-d5c9-42c1-860c-de17ba13a4ce-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpn_ztbu6u/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06852 seconds (14.59495/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpn_ztbu6u/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '47c9ee0f4333b134496348aec9e15537', '73bfa7db-7114-42ae-988a-bef401f8b290-sda', '1717345455.35832', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('73bfa7db-7114-42ae-988a-bef401f8b290-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '47c9ee0f4333b134496348aec9e15537', '73bfa7db-7114-42ae-988a-bef401f8b290-sda', '1717345455.35832', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('73bfa7db-7114-42ae-988a-bef401f8b290-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpolnn13n7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '49e4afaf-ac56-4d4a-be95-e990ee5aa54b-sda', '1717345457.34306', '1717345457', '1717345461', '', '1717345461', 0, 0) test DEBUG: /tmp/tmpolnn13n7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '49e4afaf-ac56-4d4a-be95-e990ee5aa54b-sda', '1717345457.34306', '1717345457', '1717345461', '', '1717345461', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('49e4afaf-ac56-4d4a-be95-e990ee5aa54b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03452 seconds (28.96930/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgi9yk0r1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f985ddc198967329247b0acceab8171c', 'd1d3a983-6025-4bc3-81ec-84993f0db6d2-sda', '1717345457.42485', '1717345457', '1717345461', '', '1717345461', 1, 1) test DEBUG: /tmp/tmpgi9yk0r1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345462.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'd1d3a983-6025-4bc3-81ec-84993f0db6d2-sda') test DEBUG: /tmp/tmpgi9yk0r1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0182066s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f985ddc198967329247b0acceab8171c', 'd1d3a983-6025-4bc3-81ec-84993f0db6d2-sda', '1717345457.42485', '1717345457', '1717345461', '', '1717345461', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d1d3a983-6025-4bc3-81ec-84993f0db6d2-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpgi9yk0r1/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08320 seconds (12.01853/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete ... ok test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (201, 404, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (201, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345461.24843', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345461.26360', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission ... ok test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpgi9yk0r1/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '0dfa0e54661ec85157b45e99bc4af1e2', '97ce305a-5ad6-46d6-baa0-541ff7ed8b3e-sda', '1717345457.52557', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('97ce305a-5ad6-46d6-baa0-541ff7ed8b3e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '0dfa0e54661ec85157b45e99bc4af1e2', '97ce305a-5ad6-46d6-baa0-541ff7ed8b3e-sda', '1717345457.52557', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('97ce305a-5ad6-46d6-baa0-541ff7ed8b3e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpammxhh5r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '016c5712-33c0-4dfc-9727-69a666505b4e-sda', '1717345460.26763', '1717345460', '1717345464', '', '1717345464', 0, 1) test DEBUG: /tmp/tmpammxhh5r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '016c5712-33c0-4dfc-9727-69a666505b4e-sda', '1717345460.26763', '1717345460', '1717345464', '', '1717345464', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('016c5712-33c0-4dfc-9727-69a666505b4e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04040 seconds (24.75444/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpcdq1ol80/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'de2cf21ce204325812027eef5a1590a3', '023b740a-6a84-4c3e-b459-8a78a0a386ab-sda', '1717345460.35234', '1717345460', '1717345464', '', '1717345464', 1, 0) test DEBUG: /tmp/tmpcdq1ol80/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'de2cf21ce204325812027eef5a1590a3', '023b740a-6a84-4c3e-b459-8a78a0a386ab-sda', '1717345460.35234', '1717345460', '1717345464', '', '1717345464', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('023b740a-6a84-4c3e-b459-8a78a0a386ab-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpcdq1ol80/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07253 seconds (13.78805/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpcdq1ol80/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '43903c6bfde7e1e86ef5a6519bee9456', '62dffdc3-6c14-47cb-bded-ac08f28add8e-sda', '1717345460.44215', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('62dffdc3-6c14-47cb-bded-ac08f28add8e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '43903c6bfde7e1e86ef5a6519bee9456', '62dffdc3-6c14-47cb-bded-ac08f28add8e-sda', '1717345460.44215', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('62dffdc3-6c14-47cb-bded-ac08f28add8e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpehivwqgg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '37ea63e5-4fc9-4267-b2fb-8fd2309e9864-sda', '1717345462.32796', '1717345462', '0', '', '1717345462', 0, 1) test DEBUG: /tmp/tmpehivwqgg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '37ea63e5-4fc9-4267-b2fb-8fd2309e9864-sda', '1717345462.32796', '1717345462', '0', '', '1717345462', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('37ea63e5-4fc9-4267-b2fb-8fd2309e9864-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03472 seconds (28.80299/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpu_qxv15d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f0782ae5d3108c044d4ec4ec6f51ba85', '5ca772b5-a53f-483a-a524-7a61ad293417-sda', '1717345462.40417', '1717345462', '0', '', '1717345462', 1, 0) test DEBUG: /tmp/tmpu_qxv15d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345466.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5ca772b5-a53f-483a-a524-7a61ad293417-sda') test DEBUG: /tmp/tmpu_qxv15d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00904822s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f0782ae5d3108c044d4ec4ec6f51ba85', '5ca772b5-a53f-483a-a524-7a61ad293417-sda', '1717345462.40417', '1717345462', '0', '', '1717345462', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5ca772b5-a53f-483a-a524-7a61ad293417-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpu_qxv15d/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07876 seconds (12.69626/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpu_qxv15d/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '33c0942b7b8d809a386cba1ff2d0e79d', 'edea0ad8-590c-4918-9749-72ccb6fc624b-sda', '1717345462.49707', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('edea0ad8-590c-4918-9749-72ccb6fc624b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission ... ok REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '33c0942b7b8d809a386cba1ff2d0e79d', 'edea0ad8-590c-4918-9749-72ccb6fc624b-sda', '1717345462.49707', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('edea0ad8-590c-4918-9749-72ccb6fc624b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwycpf1er/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a6c04ccb-a058-418c-8dd7-0b5320eb8394-sda', '1717345464.28382', '1717345464', '0', '', '1717345464', 0, 1) test DEBUG: /tmp/tmpwycpf1er/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a6c04ccb-a058-418c-8dd7-0b5320eb8394-sda', '1717345464.28382', '1717345464', '0', '', '1717345464', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a6c04ccb-a058-418c-8dd7-0b5320eb8394-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04888 seconds (20.45908/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpeft35pc1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '563f332ed207b2cf025c0415f87c8449', '3a67a34e-26fd-4249-8231-6ff7e2126940-sda', '1717345464.37563', '1717345464', '0', '', '1717345464', 1, 1) test DEBUG: /tmp/tmpeft35pc1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '563f332ed207b2cf025c0415f87c8449', '3a67a34e-26fd-4249-8231-6ff7e2126940-sda', '1717345464.37563', '1717345464', '0', '', '1717345464', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3a67a34e-26fd-4249-8231-6ff7e2126940-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpeft35pc1/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06780 seconds (14.74996/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpeft35pc1/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'af2b9f90fb3fd525a0fbe968052e8675', 'abef17f9-5dc5-453c-be6e-cfb175042dc9-sda', '1717345464.51551', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('abef17f9-5dc5-453c-be6e-cfb175042dc9-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'af2b9f90fb3fd525a0fbe968052e8675', 'abef17f9-5dc5-453c-be6e-cfb175042dc9-sda', '1717345464.51551', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('abef17f9-5dc5-453c-be6e-cfb175042dc9-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmplx6p3kaq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fdc3904f-ef76-4840-ae63-5029c52c854c-sda', '1717345466.10923', '1717345466', '1717345468', '', '1717345468', 0, 0) test DEBUG: /tmp/tmplx6p3kaq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fdc3904f-ef76-4840-ae63-5029c52c854c-sda', '1717345466.10923', '1717345466', '1717345468', '', '1717345468', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fdc3904f-ef76-4840-ae63-5029c52c854c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03291 seconds (30.38791/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpyffql340/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'be9d4e358b27b460b5215ae4dabf1e7a', '47ca6a32-745f-4a71-8fed-7f7a6d8eb673-sda', '1717345466.18476', '1717345466', '1717345468', '', '1717345468', 1, 0) test DEBUG: /tmp/tmpyffql340/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345471.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '47ca6a32-745f-4a71-8fed-7f7a6d8eb673-sda') test DEBUG: /tmp/tmpyffql340/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00859332s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'be9d4e358b27b460b5215ae4dabf1e7a', '47ca6a32-745f-4a71-8fed-7f7a6d8eb673-sda', '1717345466.18476', '1717345466', '1717345468', '', '1717345468', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('47ca6a32-745f-4a71-8fed-7f7a6d8eb673-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpyffql340/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06831 seconds (14.63835/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpyffql340/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7738b319423eb0f272cd43cc59d5396d', '79cc947c-683f-4636-9d40-7c91beca587a-sda', '1717345466.27143', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('79cc947c-683f-4636-9d40-7c91beca587a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7738b319423eb0f272cd43cc59d5396d', '79cc947c-683f-4636-9d40-7c91beca587a-sda', '1717345466.27143', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('79cc947c-683f-4636-9d40-7c91beca587a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp8qyvi02k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.46418', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.46777', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.47129', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.47391', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.47785', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.48199', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.48538', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.48854', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.49127', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.50797', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345466.51183', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded ... ok test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: Container GET returning 503 for [] (txn: tx93ecc462f03443a28e34e-00665c9c9e) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: txb1adc7c23a754168b820d-00665c9cb0) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: txb1adc7c23a754168b820d-00665c9cb0) test INFO: Client disconnected on read of '/a/c' (txn: tx818ab9a0e80e4f99af9d4-00665c9cb0) test INFO: Client disconnected on read of '/a/c' (txn: tx351bc607bfba402fae54b-00665c9cb0) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: Container GET returning 503 for [] (txn: txf73df4dd645e42d580baa-00665c9cb9) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx619b906c9c14431898413-00665c9cba) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx619b906c9c14431898413-00665c9cba) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx619b906c9c14431898413-00665c9cba) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: tx619b906c9c14431898413-00665c9cba) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx619b906c9c14431898413-00665c9cba) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx619b906c9c14431898413-00665c9cba) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx619b906c9c14431898413-00665c9cba) test INFO: Client disconnected on read of '/a/c' (txn: tx8c012af7a6bb46df9a733-00665c9cba) test INFO: Client disconnected on read of '/a/c' (txn: tx266a7370960a4901be52d-00665c9cba) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx011feb903fcd4e36ab52c-00665c9cba) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx011feb903fcd4e36ab52c-00665c9cba) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx011feb903fcd4e36ab52c-00665c9cba) test ERROR: Container GET returning 503 for [] (txn: tx011feb903fcd4e36ab52c-00665c9cba) test INFO: Client disconnected on read of '/a/c' (txn: tx011feb903fcd4e36ab52c-00665c9cba) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: Container GET returning 503 for [] (txn: tx4139e838faa0437f9978e-00665c9cba) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx587fee12472c469886ee5-00665c9cba) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx587fee12472c469886ee5-00665c9cba) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx587fee12472c469886ee5-00665c9cba) test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:full_control FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '119c584a-f1b3-4726-bea5-d6952758486d-sda', '1717345466.32184', '1717345466', '1717345468', '', '1717345468', 0, 1) test DEBUG: /tmp/tmp8qyvi02k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '119c584a-f1b3-4726-bea5-d6952758486d-sda', '1717345466.32184', '1717345466', '1717345468', '', '1717345468', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('119c584a-f1b3-4726-bea5-d6952758486d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.43193 seconds (0.69836/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_lac9o30/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b557b70c4688deeff5e3a5587008bf2f', 'e1cd988f-cabb-42db-849b-bda33cba3bdf-sda', '1717345467.79237', '1717345467', '1717345469', '', '1717345469', 1, 0) test DEBUG: /tmp/tmp_lac9o30/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b557b70c4688deeff5e3a5587008bf2f', 'e1cd988f-cabb-42db-849b-bda33cba3bdf-sda', '1717345467.79237', '1717345467', '1717345469', '', '1717345469', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e1cd988f-cabb-42db-849b-bda33cba3bdf-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp_lac9o30/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06825 seconds (14.65288/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp_lac9o30/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'cc64e53b5f90f131d37e5fece5081815', 'dab4d010-8282-47d4-8f1c-77a71809899d-sda', '1717345467.87767', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('dab4d010-8282-47d4-8f1c-77a71809899d-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'cc64e53b5f90f131d37e5fece5081815', 'dab4d010-8282-47d4-8f1c-77a71809899d-sda', '1717345467.87767', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('dab4d010-8282-47d4-8f1c-77a71809899d-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpvoepp_nc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e9ab4074-8641-4531-8a0d-4d6667d01e29-sda', '1717345469.23560', '1717345472', '1717345471', '', '1717345472', 0, 0) test DEBUG: /tmp/tmpvoepp_nc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e9ab4074-8641-4531-8a0d-4d6667d01e29-sda', '1717345469.23560', '1717345472', '1717345471', '', '1717345472', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e9ab4074-8641-4531-8a0d-4d6667d01e29-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03618 seconds (27.64222/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpix_y7z8m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0e8577901eb5579cf62e24d69f4f49d2', '2a9bc66d-c922-418b-a966-159b4c7d99f0-sda', '1717345469.33544', '1717345472', '1717345471', '', '1717345472', 1, 1) test DEBUG: /tmp/tmpix_y7z8m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345475.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '2a9bc66d-c922-418b-a966-159b4c7d99f0-sda') test DEBUG: /tmp/tmpix_y7z8m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00834727s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0e8577901eb5579cf62e24d69f4f49d2', '2a9bc66d-c922-418b-a966-159b4c7d99f0-sda', '1717345469.33544', '1717345472', '1717345471', '', '1717345472', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2a9bc66d-c922-418b-a966-159b4c7d99f0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpix_y7z8m/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08553 seconds (11.69165/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpix_y7z8m/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '49b5da4dc1e92b5739cc759fd93b9c8b', '687aeb72-5ac2-4a0e-bc46-deae22075d06-sda', '1717345469.45208', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('687aeb72-5ac2-4a0e-bc46-deae22075d06-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '49b5da4dc1e92b5739cc759fd93b9c8b', '687aeb72-5ac2-4a0e-bc46-deae22075d06-sda', '1717345469.45208', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('687aeb72-5ac2-4a0e-bc46-deae22075d06-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpsl11uilg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c6837982-eb90-47da-8ce6-477e3700b1ef-sda', '1717345469.51212', '1717345472', '1717345471', '', '1717345472', 0, 1) test DEBUG: /tmp/tmpsl11uilg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c6837982-eb90-47da-8ce6-477e3700b1ef-sda', '1717345469.51212', '1717345472', '1717345471', '', '1717345472', 0, 1) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345470.19353', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T162430Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345470.20948', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash ... ok test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx24b80a8c422a43d3af424-00665c9cb0) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx24b80a8c422a43d3af424-00665c9cb0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx24b80a8c422a43d3af424-00665c9cb0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx24b80a8c422a43d3af424-00665c9cb0) test ERROR: Container GET returning 503 for [] (txn: tx24b80a8c422a43d3af424-00665c9cb0) test INFO: Client disconnected on read of '/a/c' (txn: tx24b80a8c422a43d3af424-00665c9cb0) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: Container GET returning 503 for [] (txn: txf6943662c09e47ee9dfe9-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: Container GET returning 503 for [] (txn: txc458bfd927a2468cb492a-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txe662a4c38463408e9ae33-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txe662a4c38463408e9ae33-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txe662a4c38463408e9ae33-00665c9cb4) test ERROR: Container GET returning 503 for [] (txn: txe662a4c38463408e9ae33-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx3d46b74166ac466293f51-00665c9cb4) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx3d46b74166ac466293f51-00665c9cb4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx0926ab24c66544fb8cbde-00665c9cb4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx0926ab24c66544fb8cbde-00665c9cb4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx0926ab24c66544fb8cbde-00665c9cb4) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx0926ab24c66544fb8cbde-00665c9cb4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx303fdeb30d5b4524a3ff2-00665c9cb4) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txa0924e61bc4345a9b0cb8-00665c9cb4) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txa0924e61bc4345a9b0cb8-00665c9cb4) test INFO: Client disconnected on read of '/a/c' (txn: txa0924e61bc4345a9b0cb8-00665c9cb4) test ERROR: Container GET returning 503 for [] (txn: txbca8301da54a4926a89fc-00665c9cb4) test INFO: Client disconnected on read of '/a/c' (txn: txbca8301da54a4926a89fc-00665c9cb4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx0896563008714281b3d63-00665c9cb6) test INFO: Client disconnected on read of '/a/c' (txn: tx0896563008714281b3d63-00665c9cb6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available ... ok swift: The slowdown option is deprecated in favor of containers_per_second. This option may be ignored in a future release. swift: The slowdown option is deprecated in favor of containers_per_second. This option may be ignored in a future release. test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx587fee12472c469886ee5-00665c9cba) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx587fee12472c469886ee5-00665c9cba) test ERROR: Container GET returning 503 for [] (txn: tx587fee12472c469886ee5-00665c9cba) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx9414cbfe8fc34444aa805-00665c9cbb) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx9414cbfe8fc34444aa805-00665c9cbb) test ERROR: Container GET returning 503 for [] (txn: tx9414cbfe8fc34444aa805-00665c9cbb) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx88242f79e8ca4b538bb1d-00665c9cbb) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx9cee0d16919546cda4838-00665c9cbb) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx9cee0d16919546cda4838-00665c9cbb) test ERROR: Container GET returning 503 for [500, 500] (txn: tx9cee0d16919546cda4838-00665c9cbb) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx0fefd7d362c9449687365-00665c9cbb) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txc23d9d006a354fb881c4f-00665c9cbc) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txc23d9d006a354fb881c4f-00665c9cbc) test INFO: Client disconnected on read of '/a/c' (txn: txc23d9d006a354fb881c4f-00665c9cbc) test ERROR: Container GET returning 503 for [] (txn: tx26ab6b53c4604deaa7e3b-00665c9cbc) test INFO: Client disconnected on read of '/a/c' (txn: tx26ab6b53c4604deaa7e3b-00665c9cbc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx8fdab7694d2c428081851-00665c9cbd) test INFO: Client disconnected on read of '/a/c' (txn: tx8fdab7694d2c428081851-00665c9cbd) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc058fdf552e24999bd229-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc058fdf552e24999bd229-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc058fdf552e24999bd229-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc058fdf552e24999bd229-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc058fdf552e24999bd229-00665c9cbd) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: txc058fdf552e24999bd229-00665c9cbd) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0f80448560ee4bde812ac-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0f80448560ee4bde812ac-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0f80448560ee4bde812ac-00665c9cbd) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0f80448560ee4bde812ac-00665c9cbd) test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 ... ok test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T162432Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345472.62229', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T162432Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345472.63781', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0f80448560ee4bde812ac-00665c9cbd) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: tx0f80448560ee4bde812ac-00665c9cbd) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag 'a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to get commit status of PUT to /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1750, in await_response self.resp = self.conn.getexpect() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 854, in getexpect expect_status = self._status.get_expect_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 757, in get_expect_status raise expect_status Exception: kaboom! (txn: tx99148257ef6a4ae7a842a-00665c9cc0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to get commit status of PUT to /v1/a/c/o: Timeout (Nones) (txn: txa5038464ca954b779e552-00665c9cc0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: tx5e64cc403c9b400583667-00665c9cc0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: tx75dc464b0d7c48b7a3cb9-00665c9cc0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx74ecee8bab8045d29529d-00665c9cb6) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: tx74ecee8bab8045d29529d-00665c9cb6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: txfa3526dc02ac4be68ad63-00665c9cbc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c6837982-eb90-47da-8ce6-477e3700b1ef-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.59312 seconds (0.62770/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphvdbb552/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7d2f3107f1b2228577306a29322ddf14', 'e07fec35-dbda-4ac1-b150-88787bf37c9c-sda', '1717345471.16968', '1717345474', '1717345473', '', '1717345474', 1, 0) test DEBUG: /tmp/tmphvdbb552/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7d2f3107f1b2228577306a29322ddf14', 'e07fec35-dbda-4ac1-b150-88787bf37c9c-sda', '1717345471.16968', '1717345474', '1717345473', '', '1717345474', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e07fec35-dbda-4ac1-b150-88787bf37c9c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmphvdbb552/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09003 seconds (11.10684/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmphvdbb552/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7b000f7692aa8038150486fae936afaa', '38eeba72-e969-498d-b016-5d737bbf33d3-sda', '1717345471.29285', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('38eeba72-e969-498d-b016-5d737bbf33d3-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7b000f7692aa8038150486fae936afaa', '38eeba72-e969-498d-b016-5d737bbf33d3-sda', '1717345471.29285', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('38eeba72-e969-498d-b016-5d737bbf33d3-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0e8577901eb5579cf62e24d69f4f49d2', '243af8ca-6c0e-423d-a9c6-5a5a8154b0db-sda', '1717345473.00510', '1717345473.00000', '0', '{"x-container-meta-test": ["foo", "1717345473.00000"]}', '1717345473.00000', 1, 0) test DEBUG: replicator-rpc-sync time for info: 0.01s test DEBUG: replicator-rpc-sync time for update_metadata: 0.02s test DEBUG: replicator-rpc-sync time for merge_timestamps: 0.01s test DEBUG: replicator-rpc-sync time for get_sync: 0.00s test DEBUG: replicator-rpc-sync time for merge_syncs: 0.00s REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.11466', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.11466', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11466', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345473.11475', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.11478', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345473.11478', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11478', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345473.13831', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0210288s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.11466', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.11466', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11466', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345473.11475', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.11478', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345473.11478', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11478', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.11466', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.11466', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11466', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 13, 'bytes_used': 123, 'meta_timestamp': '1717345473.21466', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.11478', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345473.11478', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11478', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.11466', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.11466', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11466', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345473.23919', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.11478', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345473.11478', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11478', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.26316', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.26316', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345473.26316', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345473.23919', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.11478', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345473.11478', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11478', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.26316', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.26316', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345473.26316', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345473.23919', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.28667', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 30, 'meta_timestamp': '1717345473.28667', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.28667', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345475.75373', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345475.76731', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission ... ok proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3173, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 6776, in read raise Exception('exception message') Exception: exception message (txn: tx0132217c9f334a419b61d-00665c9cc0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: tx9ba8afd58aaf406f9e83b-00665c9cc3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.13:1013/sdn (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server ERROR: Not enough object servers ack'ed (got 0) (txn: tx336c937267a54e73abb51-00665c9cc4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server ERROR: Not enough object servers ack'ed (got 3) (txn: tx78034c79ad2d45afa90fd-00665c9cc4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: txa52fc6666505446eb0418-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: txa52fc6666505446eb0418-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: txa52fc6666505446eb0418-00665c9cc4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx5926409d3f984997a63ea-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx5926409d3f984997a63ea-00665c9cc4) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx5926409d3f984997a63ea-00665c9cc4) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_01 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_02 fake-swift ERROR: Got status 404 listing container /AUTH_test/c fake-swift ERROR: Too many bytes for /v1/AUTH_test/mancon/manifest; truncating in /v1/AUTH_test/c/seg_03 with 5 bytes left fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, max LO GET time of 86400s exceeded fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had a21075a36eeddd084e17611a238c7101, but object MD5 was actually 3902ee7e149eb8313a34757e89e21af6 fake-swift ERROR: Bad response length for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/c/seg_01 test ERROR: Error retrieving recon data: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 228, in raise_Exception raise Exception Exception test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap ... ok test.unit.common.test_swob.TestAccept.test_accept_invalid test.unit.common.test_swob.TestAccept.test_accept_invalid ... ok test.unit.common.test_swob.TestAccept.test_accept_json test.unit.common.test_swob.TestAccept.test_accept_json ... ok test.unit.common.test_swob.TestAccept.test_accept_plain test.unit.common.test_swob.TestAccept.test_accept_plain ... ok test.unit.common.test_swob.TestAccept.test_accept_xml test.unit.common.test_swob.TestAccept.test_accept_xml ... ok test.unit.common.test_swob.TestAccept.test_repr test.unit.common.test_swob.TestAccept.test_repr ... ok test.unit.common.test_utils.TestClosingIterator.test_close test.unit.common.test_utils.TestClosingIterator.test_close ... ok test.unit.common.test_utils.TestClosingIterator.test_close_gen test.unit.common.test_utils.TestClosingIterator.test_close_gen ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_exception test.unit.common.test_utils.TestClosingIterator.test_close_on_exception ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration ... ok test.unit.common.test_utils.TestClosingIterator.test_close_others test.unit.common.test_utils.TestClosingIterator.test_close_others ... ok test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter ... ok test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable ... ok test.unit.common.test_utils.TestClosingIterator.test_nested_iters test.unit.common.test_utils.TestClosingIterator.test_nested_iters ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_delta test.unit.common.utils.test_timestamp.TestTimestamp.test_delta ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_false test.unit.common.utils.test_timestamp.TestTimestamp.test_false ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_float test.unit.common.utils.test_timestamp.TestTimestamp.test_float ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_int test.unit.common.utils.test_timestamp.TestTimestamp.test_int ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_raw test.unit.common.utils.test_timestamp.TestTimestamp.test_raw ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_true test.unit.common.utils.test_timestamp.TestTimestamp.test_true ... ok test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx0391a4bd6a7c4e1c87d9c-00665c9cc2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx734a598310ac4bfaaa73a-00665c9cc2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpsbca7mgr/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpsbca7mgr/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpsbca7mgr/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpsbca7mgr/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0133 "-" 3542013 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0121 "-" 3542013 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0109 "-" 3542013 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0119 "-" 3542013 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6feea66238c44947b4296-00665c9cc2" "proxy-server 3542013" 0.0023 "-" 3542013 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:41239/sda1/1/a/c" "tx6feea66238c44947b4296-00665c9cc2" "container-server 3542013" 0.0035 "-" 3542013 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:48763/sdb1/1/a/c" "tx6feea66238c44947b4296-00665c9cc2" "container-server 3542013" 0.0018 "-" 3542013 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6feea66238c44947b4296-00665c9cc2" "proxy-server 3542013" 0.0226 "-" 3542013 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6feea66238c44947b4296-00665c9cc2" "proxy-server 3542013" 0.0382 "-" 3542013 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/34 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx6feea66238c44947b4296-00665c9cc2 - 0.0502 - - 1717345474.933790922 1717345474.983966351 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:34 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx1592b2e6603f4df8a1b5d-00665c9cc2" "proxy-server 3542013" 0.0021 "-" 3542013 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:48763/sdb1/1/a1/c1" "tx1592b2e6603f4df8a1b5d-00665c9cc2" "container-server 3542013" 0.0018 "-" 3542013 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:41239/sda1/1/a1/c1" "tx1592b2e6603f4df8a1b5d-00665c9cc2" "container-server 3542013" 0.0015 "-" 3542013 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1592b2e6603f4df8a1b5d-00665c9cc2" "proxy-server 3542013" 0.0337 "-" 3542013 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1592b2e6603f4df8a1b5d-00665c9cc2" "proxy-server 3542013" 0.0205 "-" 3542013 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/35 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx1592b2e6603f4df8a1b5d-00665c9cc2 - 0.0463 - - 1717345474.986005306 1717345475.032284737 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1ddc121cdf344807ad44f-00665c9cc3" "proxy-server 3542013" 0.0150 "-" 3542013 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:48763/sdb1/2/a/c1" "tx1ddc121cdf344807ad44f-00665c9cc3" "container-server 3542013" 0.0019 "-" 3542013 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:41239/sda1/2/a/c1" "tx1ddc121cdf344807ad44f-00665c9cc3" "container-server 3542013" 0.0069 "-" 3542013 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1ddc121cdf344807ad44f-00665c9cc3" "proxy-server 3542013" 0.0513 "-" 3542013 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1ddc121cdf344807ad44f-00665c9cc3" "proxy-server 3542013" 0.0308 "-" 3542013 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/35 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx1ddc121cdf344807ad44f-00665c9cc3 - 0.0769 - - 1717345475.034234047 1717345475.111108303 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx948c66ba392d4e2e8b5ed-00665c9cc3" "proxy-server 3542013" 0.0079 "-" 3542013 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:41239/sda1/1/a/c2" "tx948c66ba392d4e2e8b5ed-00665c9cc3" "container-server 3542013" 0.0017 "-" 3542013 2 test.unit.proxy.test_server.TestContainerController.test_CORS_valid test.unit.proxy.test_server.TestContainerController.test_CORS_valid ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE test.unit.proxy.test_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_GET_no_content test.unit.proxy.test_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Expect: 100-continue on /a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3038, in _make_putter return MIMEPutter.connect( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2001, in connect raise FooterNotSupported() swift.common.exceptions.FooterNotSupported (txn: txc0ba2a9413354d1ebd065-00665c9cc7) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: txbf17447a48b64ebd98138-00665c9cc9) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txbf17447a48b64ebd98138-00665c9cc9) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txbf17447a48b64ebd98138-00665c9cc9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx0929ba8e0c494255af237-00665c9cc9) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: tx0929ba8e0c494255af237-00665c9cc9) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx0929ba8e0c494255af237-00665c9cc9) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx0929ba8e0c494255af237-00665c9cc9) proxy-server ERROR: Object PUT returning 503 for [503, 201, 503, 503, 201, 201, 201, 201, 503, 201, 201, 201, 201, 201] (txn: tx0929ba8e0c494255af237-00665c9cc9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp__qkee37/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp__qkee37/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp__qkee37/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp__qkee37/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:41 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0120 "-" 3542007 - test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345479.94087', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:39 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345479.95821', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:41 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0091 "-" 3542007 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0104 "-" 3542007 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0089 "-" 3542007 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx70bd4b98d0b94c8aa36f5-00665c9ccb" "proxy-server 3542007" 0.0016 "-" 3542007 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:57693/sda1/1/a/c" "tx70bd4b98d0b94c8aa36f5-00665c9ccb" "container-server 3542007" 0.0028 "-" 3542007 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:56215/sdb1/1/a/c" "tx70bd4b98d0b94c8aa36f5-00665c9ccb" "container-server 3542007" 0.0014 "-" 3542007 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx70bd4b98d0b94c8aa36f5-00665c9ccb" "proxy-server 3542007" 0.0182 "-" 3542007 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx70bd4b98d0b94c8aa36f5-00665c9ccb" "proxy-server 3542007" 0.0311 "-" 3542007 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/43 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx70bd4b98d0b94c8aa36f5-00665c9ccb - 0.0410 - - 1717345483.734187365 1717345483.775171757 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txfaeead53bb93442fa069a-00665c9ccb" "proxy-server 3542007" 0.0017 "-" 3542007 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:57693/sda1/1/a1/c1" "txfaeead53bb93442fa069a-00665c9ccb" "container-server 3542007" 0.0015 "-" 3542007 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:56215/sdb1/1/a1/c1" "txfaeead53bb93442fa069a-00665c9ccb" "container-server 3542007" 0.0013 "-" 3542007 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfaeead53bb93442fa069a-00665c9ccb" "proxy-server 3542007" 0.0157 "-" 3542007 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfaeead53bb93442fa069a-00665c9ccb" "proxy-server 3542007" 0.0289 "-" 3542007 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/43 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txfaeead53bb93442fa069a-00665c9ccb - 0.0384 - - 1717345483.776992559 1717345483.815436602 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx87401f4fdb1d4819a0ce1-00665c9ccb" "proxy-server 3542007" 0.0058 "-" 3542007 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:56215/sdb1/2/a/c1" "tx87401f4fdb1d4819a0ce1-00665c9ccb" "container-server 3542007" 0.0015 "-" 3542007 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:57693/sda1/2/a/c1" "tx87401f4fdb1d4819a0ce1-00665c9ccb" "container-server 3542007" 0.0060 "-" 3542007 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx87401f4fdb1d4819a0ce1-00665c9ccb" "proxy-server 3542007" 0.0224 "-" 3542007 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx87401f4fdb1d4819a0ce1-00665c9ccb" "proxy-server 3542007" 0.0347 "-" 3542007 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/43 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx87401f4fdb1d4819a0ce1-00665c9ccb - 0.0489 - - 1717345483.817239523 1717345483.866098166 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf01ead965c054431af2e6-00665c9ccb" "proxy-server 3542007" 0.0066 "-" 3542007 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:56215/sdb1/1/a/c2" "txf01ead965c054431af2e6-00665c9ccb" "container-server 3542007" 0.0014 "-" 3542007 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:57693/sda1/1/a/c2" "txf01ead965c054431af2e6-00665c9ccb" "container-server 3542007" 0.0064 "-" 3542007 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf01ead965c054431af2e6-00665c9ccb" "proxy-server 3542007" 0.0210 "-" 3542007 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:43 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf01ead965c054431af2e6-00665c9ccb" "proxy-server 3542007" 0.0337 "-" 3542007 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/43 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txf01ead965c054431af2e6-00665c9ccb - 0.0488 - - 1717345483.867900610 1717345483.916666269 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345478.81369', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy ... ok test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test ERROR: Error parsing recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 225, in raise_ValueError raise ValueError ValueError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication ... ok swift: ERROR 500 b'' Trying to GET /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: ERROR 500 b'' Trying to GET /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: ERROR 500 b'' Trying to GET /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: Account GET returning 503 for [500, 500, 500] swift: Client disconnected on read of '/AUTH_bob' acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:35 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:48763/sdb1/1/a/c2" "tx948c66ba392d4e2e8b5ed-00665c9cc3" "container-server 3542013" 0.0141 "-" 3542013 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:38 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx948c66ba392d4e2e8b5ed-00665c9cc3" "proxy-server 3542013" 2.8900 "-" 3542013 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:24:38 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx948c66ba392d4e2e8b5ed-00665c9cc3" "proxy-server 3542013" 2.9056 "-" 3542013 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/24/38 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx948c66ba392d4e2e8b5ed-00665c9cc3 - 2.9234 - - 1717345475.113070250 1717345478.036423683 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: txfc8e67fde73343b88d534-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txf9f45325c87845cf8f197-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: txd5370ba87f3343bdadb77-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: txd5370ba87f3343bdadb77-00665c9cc7) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txd5370ba87f3343bdadb77-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txd12555b6bbda4106ac030-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txd12555b6bbda4106ac030-00665c9cc7) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txd12555b6bbda4106ac030-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: txae070e6aca984bc09302b-00665c9cc7) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txae070e6aca984bc09302b-00665c9cc7) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txd4b2a4a4d455440d991d4-00665c9cc7) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txd4b2a4a4d455440d991d4-00665c9cc7) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx228e747f30464c3186464-00665c9cc7) test INFO: Client disconnected on read of '/a/c' (txn: tx228e747f30464c3186464-00665c9cc7) test INFO: Client disconnected on read of '/a/c' (txn: tx92fdf430311b4c0a98b30-00665c9cc7) test INFO: Client disconnected on read of '/a/c' (txn: tx92fdf430311b4c0a98b30-00665c9cc7) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txbe975375bbbd4d35a1bab-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txb0dde2a58dce4fbe91f28-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx2a1b9003b0b14e95ba7d2-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx2a1b9003b0b14e95ba7d2-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx2a1b9003b0b14e95ba7d2-00665c9cc8) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx2a1b9003b0b14e95ba7d2-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx340041b5393443e684ec5-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx340041b5393443e684ec5-00665c9cc8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx340041b5393443e684ec5-00665c9cc8) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx340041b5393443e684ec5-00665c9cc8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestContainerController.test_OPTIONS test.unit.proxy.test_server.TestContainerController.test_OPTIONS ... ok test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.proxy.test_server.TestContainerController.test_POST test.unit.proxy.test_server.TestContainerController.test_POST ... ok test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata ... ok swift: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to GET /v1/a: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request#012 conn = http_connect(#012 ^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect#012 i, status = next(conn_id_and_code_iter)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^#012StopIteration swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to GET /v1/a: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request#012 conn = http_connect(#012 ^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect#012 i, status = next(conn_id_and_code_iter)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^#012StopIteration test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_server.TestContainerController.test_POST_metadata test.unit.proxy.test_server.TestContainerController.test_POST_metadata ... ok swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: Account GET returning 503 for (204, 404, 503) swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: Account GET returning 503 for (204, 503, 503) swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: Account GET returning 503 for (404, 503, 503) swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: ERROR 503 Trying to DELETE /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: Account GET returning 503 for (503, 503, 503) test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl ... ok swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: Account GET returning 503 for (204, 404, 503) swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: Account GET returning 503 for (204, 503, 503) swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: Account GET returning 503 for (404, 503, 503) swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to POST /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: Account GET returning 503 for (503, 503, 503) swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: Account GET returning 503 for (201, 404, 503) swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: Account GET returning 503 for (201, 503, 503) swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: Account GET returning 503 for (404, 503, 503) swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.0:1000/sda swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.1:1001/sdb swift: ERROR 503 Trying to PUT /AUTH_bob From Account Server 10.0.0.2:1002/sdc swift: Account GET returning 503 for (503, 503, 503) swift: Client disconnected on read of '/a' swift: Client disconnected on read of '/a' swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to POST /a: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request#012 conn = http_connect(#012 ^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect#012 i, status = next(conn_id_and_code_iter)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^#012StopIteration swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to PUT /a: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request#012 conn = http_connect(#012 ^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect#012 i, status = next(conn_id_and_code_iter)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^#012StopIteration test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345480.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345480.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345480.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345480.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx4dd3c6e291b04c4e91155-00665c9cd4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sdb (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sdt (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sds (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sdx (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sdy (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sdr (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sdu (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sdq (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sdw (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sdz (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sdp (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sdv (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sdo (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: txacf7b61fd2ce41098e352-00665c9ccb) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: txacf7b61fd2ce41098e352-00665c9ccb) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: txacf7b61fd2ce41098e352-00665c9ccb) test ERROR: Account HEAD returning 503 for [] (txn: txacf7b61fd2ce41098e352-00665c9ccb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx43d81e83784c419bb08f7-00665c9ccc) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa9137bad15d74be2804fd-00665c9ccc) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txa9137bad15d74be2804fd-00665c9ccc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx18572b8baeb24981a7036-00665c9ccc) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc21a18fef4600955d4-00665c9ccc) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx08896b3907b444e5b9381-00665c9ccc) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb0905ddc2d9b49aeb04e4-00665c9ccc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txdc1588e502bb498696abc-00665c9ccd) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx56c33be9af904fcb91ac9-00665c9ccd) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx56c33be9af904fcb91ac9-00665c9ccd) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4862c0b944af42778ba74-00665c9ccd) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx4862c0b944af42778ba74-00665c9ccd) test WARNING: Could not autocreate account '/account' (txn: tx4862c0b944af42778ba74-00665c9ccd) test.unit.proxy.test_server.TestContainerController.test_PUT test.unit.proxy.test_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info ... ok test.unit.common.middleware.test_acl.TestACL.test_clean_acl test.unit.common.middleware.test_acl.TestACL.test_clean_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_parse_acl test.unit.common.middleware.test_acl.TestACL.test_parse_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset ... ok test.unit.common.test_db.TestGetDBConnection.test_invalid_path test.unit.common.test_db.TestGetDBConnection.test_invalid_path ... ok test.unit.common.test_db.TestGetDBConnection.test_locked_db test.unit.common.test_db.TestGetDBConnection.test_locked_db ... ok test.unit.common.test_db.TestGetDBConnection.test_normal_case test.unit.common.test_db.TestGetDBConnection.test_normal_case ... ok test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked ... ok test.unit.common.test_exceptions.TestExceptions.test_client_exception test.unit.common.test_exceptions.TestExceptions.test_client_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_exception test.unit.common.test_exceptions.TestExceptions.test_replication_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update ... ok test.unit.common.test_manager.TestServer.test_conf_dir test.unit.common.test_manager.TestServer.test_conf_dir ... ok test.unit.common.test_manager.TestServer.test_conf_files test.unit.common.test_manager.TestServer.test_conf_files ... ok test.unit.common.test_manager.TestServer.test_create_server test.unit.common.test_manager.TestServer.test_create_server ... ok test.unit.common.test_manager.TestServer.test_expirer_conf_files test.unit.common.test_manager.TestServer.test_expirer_conf_files ... ok test.unit.common.test_manager.TestServer.test_get_conf_file_name test.unit.common.test_manager.TestServer.test_get_conf_file_name ... ok test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_pid_file_name test.unit.common.test_manager.TestServer.test_get_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_running_pids test.unit.common.test_manager.TestServer.test_get_running_pids ... ok test.unit.common.test_manager.TestServer.test_interact test.unit.common.test_manager.TestServer.test_interact ... ok test.unit.common.test_manager.TestServer.test_iter_pid_files test.unit.common.test_manager.TestServer.test_iter_pid_files ... ok test.unit.common.test_manager.TestServer.test_kill_running_pids test.unit.common.test_manager.TestServer.test_kill_running_pids ... ok test.unit.common.test_manager.TestServer.test_launch test.unit.common.test_manager.TestServer.test_launch ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir test.unit.common.test_manager.TestServer.test_named_conf_dir ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files ... ok test.unit.common.test_manager.TestServer.test_proxy_conf_dir test.unit.common.test_manager.TestServer.test_proxy_conf_dir ... ok test.unit.common.test_manager.TestServer.test_server_equality test.unit.common.test_manager.TestServer.test_server_equality ... ok test.unit.common.test_manager.TestServer.test_server_repr test.unit.common.test_manager.TestServer.test_server_repr ... ok test.unit.common.test_manager.TestServer.test_server_to_string test.unit.common.test_manager.TestServer.test_server_to_string ... ok test.unit.common.test_manager.TestServer.test_signal_pids test.unit.common.test_manager.TestServer.test_signal_pids ... ok test.unit.common.test_manager.TestServer.test_spawn test.unit.common.test_manager.TestServer.test_spawn ... ok test.unit.common.test_manager.TestServer.test_status test.unit.common.test_manager.TestServer.test_status ... ok test.unit.common.test_manager.TestServer.test_stop test.unit.common.test_manager.TestServer.test_stop ... ok test.unit.common.test_manager.TestServer.test_wait test.unit.common.test_manager.TestServer.test_wait ... ok test.unit.common.test_utils.TestNamespace.test_bounds_serialization test.unit.common.test_utils.TestNamespace.test_bounds_serialization ... ok test.unit.common.test_utils.TestNamespace.test_comparisons test.unit.common.test_utils.TestNamespace.test_comparisons ... ok test.unit.common.test_utils.TestNamespace.test_contains test.unit.common.test_utils.TestNamespace.test_contains ... ok test.unit.common.test_utils.TestNamespace.test_end_marker test.unit.common.test_utils.TestNamespace.test_end_marker ... ok test.unit.common.test_utils.TestNamespace.test_entire_namespace test.unit.common.test_utils.TestNamespace.test_entire_namespace ... ok test.unit.common.test_utils.TestNamespace.test_expand test.unit.common.test_utils.TestNamespace.test_expand ... ok test.unit.common.test_utils.TestNamespace.test_includes test.unit.common.test_utils.TestNamespace.test_includes ... ok test.unit.common.test_utils.TestNamespace.test_lower_setter test.unit.common.test_utils.TestNamespace.test_lower_setter ... ok test.unit.common.test_utils.TestNamespace.test_name test.unit.common.test_utils.TestNamespace.test_name ... ok test.unit.common.test_utils.TestNamespace.test_name_unexpected_format test.unit.common.test_utils.TestNamespace.test_name_unexpected_format ... ok test.unit.common.test_utils.TestNamespace.test_total_ordering test.unit.common.test_utils.TestNamespace.test_total_ordering ... ok test.unit.common.test_utils.TestNamespace.test_unicode_name test.unit.common.test_utils.TestNamespace.test_unicode_name ... ok test.unit.common.test_utils.TestNamespace.test_upper_setter test.unit.common.test_utils.TestNamespace.test_upper_setter ... ok test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345473.26316', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.26316', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345473.26316', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345473.23919', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.28667', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717345473.31495', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.28667', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345476.50412', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.26316', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.26316', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345473.23919', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.28667', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717345473.31495', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.28667', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpe6mp5a55/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b939080422ef3fce852c933c539c8c67', 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda', '1717345473.09728', '1717345473.09372', '0', '', '1717345473.09372', 1, 0) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345476.50412', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345473.26316', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.26316', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345473.11475', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345473.23919', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.11475', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345473.28667', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717345473.31495', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345473.28667', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345473.21047', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345473.21047', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345473.21037', 'epoch': '1717345473.21037', 'reported': 0, 'tombstones': -1}], 'da9fafc5-d00c-4b78-83b2-5c88d835150e-sda') test DEBUG: /tmp/tmpe6mp5a55/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.0:1000/sda test WARNING: /tmp/tmpe6mp5a55/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.0:1000/sda; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'c8f9d7bec63fbc4cf7bcffe163c12b82', '017f1858-21a3-4fa7-9c05-aef3a54a8a62-sda', '1717345476.56732', '1717345476.56365', '0', '', '1717345476.56365', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': 'a/c', 'timestamp': '1717345476.58609', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.58609', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345476.58581', 'epoch': '1717345476.58581', 'reported': 0, 'tombstones': -1}], '017f1858-21a3-4fa7-9c05-aef3a54a8a62-sda') test ERROR: ERROR Bad response 500 from 10.0.0.1 test DEBUG: /tmp/tmp100qr6jz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345476.58542', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '017f1858-21a3-4fa7-9c05-aef3a54a8a62-sda') test DEBUG: /tmp/tmp100qr6jz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0107517s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4d1d604e-7122-4d8e-9e73-4d6697c58f3c-sda', '1717345476.66242', '1717345476.65900', '0', '', '1717345476.65900', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345476.67973', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345476.67973', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345476.67973', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345476.67979', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345476.67979', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345476.67979', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345476.67982', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345476.67982', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345476.67982', 'epoch': None, 'reported': 0, 'tombstones': -1}], '4d1d604e-7122-4d8e-9e73-4d6697c58f3c-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0796b9e1-4864-4185-b8ab-a8c52bd498de-sda', '1717345478.52127', '1717345478.51799', '0', '', '1717345478.51799', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345478.53799', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345478.53799', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.53799', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345478.53805', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345478.53805', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.53805', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345478.53807', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345478.53807', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.53807', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65859', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65859', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65859', 'epoch': None, 'reported': 0, 'tombstones': -1}], '0796b9e1-4864-4185-b8ab-a8c52bd498de-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9ff0ad0d-043f-4155-ad59-adaabf13bd7c-sda', '1717345478.58146', '1717345478.57817', '0', '', '1717345478.57817', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345478.60117', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345478.60117', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.60117', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345478.60127', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345478.60127', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.60127', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345478.60130', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345478.60130', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.60130', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65859', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65859', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65859', 'epoch': None, 'reported': 0, 'tombstones': -1}], '9ff0ad0d-043f-4155-ad59-adaabf13bd7c-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0de77f9e-7a28-4a4a-9245-8947488f9d77-sda', '1717345478.63960', '1717345478.63649', '0', '', '1717345478.63649', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345478.65376', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345478.65376', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.65376', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345478.65381', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345478.65381', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.65381', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345478.65384', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345478.65384', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.65384', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65859', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65859', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65859', 'epoch': None, 'reported': 0, 'tombstones': -1}], '0de77f9e-7a28-4a4a-9245-8947488f9d77-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '18ecd74e-b722-42bd-ac90-e02e0225cc8d-sda', '1717345478.69166', '1717345478.68836', '0', '', '1717345478.68836', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345478.70598', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345478.70598', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.70598', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345478.70603', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345478.70603', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.70603', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345478.70605', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345478.70605', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.70605', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65859', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65859', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65859', 'epoch': None, 'reported': 0, 'tombstones': -1}], '18ecd74e-b722-42bd-ac90-e02e0225cc8d-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3814f067-02b7-4a9e-a1eb-3f3f309c2ce1-sda', '1717345478.77612', '1717345478.77295', '0', '', '1717345478.77295', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345478.79117', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345478.79117', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.79117', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345478.79122', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345478.79122', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.79122', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345478.79125', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345478.79125', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345478.79125', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65878', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65878', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65878', 'epoch': '1717345476.65881', 'reported': 0, 'tombstones': -1}], '3814f067-02b7-4a9e-a1eb-3f3f309c2ce1-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '91b58520-88b4-4be2-946e-86cb012b061e-sda', '1717345480.49817', '1717345480.49515', '0', '', '1717345480.49515', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345480.51762', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345480.51762', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.51762', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345480.51768', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345480.51768', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.51768', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345480.51770', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345480.51770', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.51770', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65878', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65878', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65878', 'epoch': '1717345476.65881', 'reported': 0, 'tombstones': -1}], '91b58520-88b4-4be2-946e-86cb012b061e-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4b1656d4-dc6a-41e5-84ac-f710be9ef441-sda', '1717345480.55356', '1717345480.55054', '0', '', '1717345480.55054', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345480.56908', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345480.56908', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.56908', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345480.56913', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345480.56913', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.56913', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345480.56915', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345480.56915', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.56915', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65878', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65878', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65878', 'epoch': '1717345476.65881', 'reported': 0, 'tombstones': -1}], '4b1656d4-dc6a-41e5-84ac-f710be9ef441-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2f895a71-0c48-4a5e-a37e-b7fb33894084-sda', '1717345480.60783', '1717345480.60479', '0', '', '1717345480.60479', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345480.62340', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345480.62340', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.62340', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345480.62345', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345480.62345', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.62345', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345480.62347', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345480.62347', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.62347', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65890', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65890', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65890', 'epoch': '1717345476.65892', 'reported': 0, 'tombstones': -1}], '2f895a71-0c48-4a5e-a37e-b7fb33894084-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fe98ff65-a6df-459b-8df0-3686541aaa61-sda', '1717345480.66404', '1717345480.66084', '0', '', '1717345480.66084', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success ... ok test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler ... ok test.unit.common.test_internal_client.TestGetAuth.test_invalid_version test.unit.common.test_internal_client.TestGetAuth.test_invalid_version ... ok test.unit.common.test_internal_client.TestGetAuth.test_ok test.unit.common.test_internal_client.TestGetAuth.test_ok ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_init test.unit.common.test_memcached.TestMemcacheCommand.test_init ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_percent test.unit.common.test_utils.TestFsHasFreeSpace.test_percent ... ok test.unit.common.test_utils.TestStringAlong.test_happy test.unit.common.test_utils.TestStringAlong.test_happy ... ok test.unit.common.test_utils.TestStringAlong.test_unhappy test.unit.common.test_utils.TestStringAlong.test_unhappy ... ok test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts ... ok test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", swift: Client disconnected on read of '/a' (txn: txd3db977abaad4753a2da4-00665c9ce5) swift: Client disconnected on read of '/a' (txn: txd3db977abaad4753a2da4-00665c9ce5) test INFO: autocreate account '/account' (txn: txf585959e531c4692a9a23-00665c9ccd) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test ERROR: Account HEAD returning 503 for [503, 503] (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test WARNING: Could not autocreate account '/account' (txn: tx8f1e13a29aa847549b1ad-00665c9cd0) test INFO: autocreate account '/account' (txn: tx2eca5ae59745490cbd064-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txb8f376ee728b4ad585b32-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txb8f376ee728b4ad585b32-00665c9cd0) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txb8f376ee728b4ad585b32-00665c9cd0) test INFO: autocreate account '/account' (txn: tx4649d4a7b7fa4e10985df-00665c9cd0) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx8c81d91d5277424494b91-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx8c81d91d5277424494b91-00665c9cd0) test INFO: autocreate account '/account' (txn: tx8c81d91d5277424494b91-00665c9cd0) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx8386cc8f431149bab7103-00665c9cd0) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx8386cc8f431149bab7103-00665c9cd0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: txcb17eff5cf24464386520-00665c9cd2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: tx6a000389d6754347b6e00-00665c9cd2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc4288f2d5c104592b0b86-00665c9cd2) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2cac36ed1ef401c904bd-00665c9cd2) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx99d81ff30b7646069d3ba-00665c9cd2) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5be38affc9054756ad8e7-00665c9cd2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txfdd624086e044f43a8d6b-00665c9cd2) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx3e230d3c3f5b4b8d915ce-00665c9cd2) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx300e796b1db8447c995fd-00665c9cd2) test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345484.33320', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ Unable to locate config number 5 for account-server Unable to locate config number 5 for object-server Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 1 Signal object-replicator pid: 11 signal: 15 Signal object-replicator pid: 12 signal: 15 Signal proxy-server pid: 1 signal: 1 output Removing pid file /tmp/tmpttoq5da6/var/run/one-server.pid with wrong pid 3 output Removing pid file /tmp/tmpttoq5da6/var/run/zero-server.pid with invalid pid output Removing pid file /tmp/tmpttoq5da6/var/run/invalid-server.pid with invalid pid Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 3 signal: 15 test INFO: Begin container audit "once" mode test DEBUG: Audit passed for /tmp/tmpp6v7brlu/sda/containers/0/0/0/test.db test INFO: Container audit "once" mode completed: 0.00s test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx287db699f28047dda2dd8-00665c9cda) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx58eaee4bc9844ce782862-00665c9cda) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txd0593b23276543f784223-00665c9cda) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx026f426d41894095af130-00665c9cda) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txbbd12aac149840caa2948-00665c9cda) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa2427344765d468eb1e2f-00665c9cda) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx96385b5bf6dd4777a5096-00665c9cda) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe1a9e175ce6440679b1ca-00665c9cda) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1d341d71d0af4d8295e63-00665c9cda) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7fa82539468d4a80bebac-00665c9cda) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx3462306fb274442f96d7c-00665c9cda) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx6d1f948f75e44b79b8351-00665c9cda) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server ERROR: Object returning 503 for [] (txn: tx14b0ebc8af924889b675d-00665c9cd5) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server ERROR: Object returning 503 for [] (txn: txb6c7d84decc84b3393429-00665c9cd7) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission ... ok test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx300e796b1db8447c995fd-00665c9cd2) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx300e796b1db8447c995fd-00665c9cd2) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx29c95c3583294dc8b0603-00665c9cd3) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx29c95c3583294dc8b0603-00665c9cd3) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx29c95c3583294dc8b0603-00665c9cd3) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx1b69740197bb4e38808a9-00665c9cd3) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx1b69740197bb4e38808a9-00665c9cd3) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx1b69740197bb4e38808a9-00665c9cd3) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx1b69740197bb4e38808a9-00665c9cd3) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx043a3fb01ad34a73800b9-00665c9cd3) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx043a3fb01ad34a73800b9-00665c9cd3) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx043a3fb01ad34a73800b9-00665c9cd3) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx043a3fb01ad34a73800b9-00665c9cd3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx35e397aacd30436496683-00665c9cd5) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txc3ad1050babc45b1b3646-00665c9cd5) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txc3ad1050babc45b1b3646-00665c9cd5) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc3ad1050babc45b1b3646-00665c9cd5) test ERROR: Account HEAD returning 503 for [] (txn: txc3ad1050babc45b1b3646-00665c9cd5) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx65edfbb2bc7a488d99df8-00665c9cd5) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx65edfbb2bc7a488d99df8-00665c9cd5) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx65edfbb2bc7a488d99df8-00665c9cd5) test ERROR: Account HEAD returning 503 for [] (txn: tx65edfbb2bc7a488d99df8-00665c9cd5) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txd8ea292a21b44fa58eaac-00665c9cd5) test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345489.26251', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Ignoring 404 from primary (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server ERROR: Object returning 503 for [] (txn: tx526789ce338b46668097a-00665c9cd7) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server ERROR: Object returning 503 for [] (txn: txa3951765874e4e60b243d-00665c9cd9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345480.68060', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345480.68060', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.68060', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345480.68065', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345480.68065', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.68065', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345480.68068', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345480.68068', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345480.68068', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65878', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65878', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65878', 'epoch': '1717345476.65881', 'reported': 0, 'tombstones': -1}], 'fe98ff65-a6df-459b-8df0-3686541aaa61-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '52d1e3de-1878-4fac-9d2c-c4bdd526ae13-sda', '1717345484.51871', '1717345484.51555', '0', '', '1717345484.51555', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345484.53419', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345484.53419', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.53419', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345484.53424', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345484.53424', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.53424', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345484.53426', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345484.53426', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.53426', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65878', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65878', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65878', 'epoch': '1717345476.65881', 'reported': 0, 'tombstones': -1}], '52d1e3de-1878-4fac-9d2c-c4bdd526ae13-sda') test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '503f0773-69fe-49be-895f-a27572099ef7-sda', '1717345484.58797', '1717345484.58473', '0', '', '1717345484.58473', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345484.60268', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345484.60268', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.60268', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345484.60272', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345484.60272', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.60272', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345484.60279', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345484.60279', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.60279', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345476.65898', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65859', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65859', 'epoch': None, 'reported': 0, 'tombstones': -1}], '503f0773-69fe-49be-895f-a27572099ef7-sda') test WARNING: Ignoring remote osr w/o epoch, own_sr: {'name': 'a/c', 'timestamp': '1717345476.65878', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65878', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65878', 'epoch': '1717345476.65881', 'reported': 0, 'tombstones': -1}, remote_sr: {'name': 'a/c', 'timestamp': '1717345476.65898', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345476.65859', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345476.65859', 'epoch': None, 'reported': 0, 'tombstones': -1}, source: repl_req test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmp9effaoj1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'c28cc60b67491b846dab898a39e99158', 'd24a65ab-3b10-4315-9e29-39071e3b931a-sda', '1717345484.64127', '1717345484.63782', '0', '', '1717345484.63782', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345484.65738', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], 'd24a65ab-3b10-4315-9e29-39071e3b931a-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test DEBUG: Replicating db /tmp/tmpn7eizrnj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '79e453ae-af8c-4f73-a340-ca3d8832ac42-sda', '1717345484.69609', '1717345484.69242', '0', '', '1717345484.69242', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test ERROR: ERROR syncing /tmp/tmpn7eizrnj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db with node {'ip': '10.0.0.1', 'replication_ip': '10.0.0.1', 'port': 1001, 'replication_port': 1001, 'device': 'sdb', 'zone': 1, 'region': 1, 'id': 1, 'weight': 1, 'index': 1}: Timeout (0.001s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b3d012ea82e8e18c8e9418d66c37d807', 'ea674b05-173b-47f3-b3ab-7b6be50f1324-sda', '1717345484.73879', '1717345484.73450', '0', '', '1717345484.73450', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345484.75824', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], 'ea674b05-173b-47f3-b3ab-7b6be50f1324-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpfjubanfc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txac6f2cb16ced48ca88aaf-00665c9cda) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx0b6e2580dec5465e9cd01-00665c9cdc) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx2df9fb6f7cde49a6a3f19-00665c9cdc) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3c6abaebd94b4d07942c5-00665c9cdc) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1bda4d2f96db4f65bb2b9-00665c9cdc) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx00101421dd0c439fae27d-00665c9cdc) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx57dbe3cb41ae4e559f6ad-00665c9cdc) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx02c8ecb277a64c3bbf811-00665c9cdc) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx507ff281859f4c8a9ccb3-00665c9cdc) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe0bea00fb4324e748a51a-00665c9cdc) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa8e23785638e4946b935c-00665c9cdc) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txf8b65ceaa6ef439885e8b-00665c9cdc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: tx11308558b7f044869b427-00665c9cde) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: txbede9b4d44b04525a58e5-00665c9cde) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txe8973e7bfaea40ba89716-00665c9cde) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txef2d9cbf7c8740dfb255a-00665c9cdf) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txd815196a3b7b43c1bb348-00665c9cdf) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: txea1a1310503b4acd977e9-00665c9cdf) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx438de0871ee94c8e899c8-00665c9cdf) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx47193f94fbd0420491906-00665c9cdf) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: tx0ee80d2d11fd4d5482375-00665c9cdf) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txfded59e83ed541c18a001-00665c9cdf) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txfadbbbd226144fffb1672-00665c9cdf) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd8b14296b39f48daa80ad-00665c9cdf) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx6fe4dffd14e5400eb3907-00665c9cdf) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx55b227e19a55434dadf9c-00665c9cdf) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx2734f3e750fe41609e72a-00665c9cdf) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345489.28254', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345491.15186', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:47 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0cee3f435374afb911b9-00665c9cd9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server ERROR: Object returning 503 for [] (txn: tx0ef4795104d64077a2dc5-00665c9cdb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx7104a725829c4f718baab-00665c9cdb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8524757e-51ae-444c-ac8b-1e04f08c008f-sda', '1717345484.79366', '1717345484.79028', '0', '', '1717345484.79028', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8524757e-51ae-444c-ac8b-1e04f08c008f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8524757e-51ae-444c-ac8b-1e04f08c008f-sda', '1717345484.79366', '1717345484.79028', '0', '', '1717345484.79028', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8524757e-51ae-444c-ac8b-1e04f08c008f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 3.72580 seconds (0.26840/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpfjubanfc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8524757e-51ae-444c-ac8b-1e04f08c008f-sda', '1717345484.79366', '1717345484.79028', '0', '', '1717345484.79028', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345488.54016', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345488.54016', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345488.54016', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345484.79994', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345484.79994', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.79994', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345484.79996', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345484.79996', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.79996', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345484.80013', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345484.80013', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345484.79998', 'epoch': '1717345484.79998', 'reported': 0, 'tombstones': -1}], '8524757e-51ae-444c-ac8b-1e04f08c008f-sda') test DEBUG: /tmp/tmpfjubanfc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpfjubanfc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8524757e-51ae-444c-ac8b-1e04f08c008f-sda', '1717345484.79366', '1717345484.79028', '0', '', '1717345484.79028', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345488.54016', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345488.54016', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345488.54016', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345484.79994', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345484.79994', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.79994', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345484.79996', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345484.79996', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345484.79996', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345484.80013', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345484.80013', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345484.79998', 'epoch': '1717345484.79998', 'reported': 0, 'tombstones': -1}], '8524757e-51ae-444c-ac8b-1e04f08c008f-sda') test DEBUG: /tmp/tmpfjubanfc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.2:1002/sdc test WARNING: /tmp/tmpfjubanfc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.2:1002/sdc; have shard ranges and will wait for cleaving test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03030 seconds (33.00771/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '1d89d0a8-5496-489f-9936-8159d215031a-sda', '1717345488.59561', '1717345488.59234', '1717345488.61361', '', '1717345488.61361', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpsb6bajiz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2f2a6dde-ece5-4277-a77e-ee9fc5d1242e-sda', '1717345488.67467', '1717345488.00000', '0', '', '1717345488.00000', 0, 0) test DEBUG: /tmp/tmpsb6bajiz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2f2a6dde-ece5-4277-a77e-ee9fc5d1242e-sda', '1717345488.67467', '1717345488.00000', '0', '', '1717345488.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2f2a6dde-ece5-4277-a77e-ee9fc5d1242e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04443 seconds (22.50484/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpn8skjvfm/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', 'c9276111-d867-4d9c-93f4-fb8d577de90a-sda', '1717345488.76360', '1717345489.00000', '0', '{"X-Container-Sync-To": ["a", "1717345490.00000"], "X-Container-Sync-Key": ["secret", "1717345491.00000"]}', '1717345489.00000', 0, 0) REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('c9276111-d867-4d9c-93f4-fb8d577de90a-sda', 'c48d89b33840539f9c5442ab3b10027d.db') REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', 'c9276111-d867-4d9c-93f4-fb8d577de90a-sda', '1717345488.76360', '1717345489.00000', '0', '{"X-Container-Sync-To": ["a", "1717345490.00000"], "X-Container-Sync-Key": ["secret", "1717345491.00000"]}', '1717345489.00000', 0, 0) REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('c9276111-d867-4d9c-93f4-fb8d577de90a-sda', 'c48d89b33840539f9c5442ab3b10027d.db') test DEBUG: Replicating db /tmp/tmpn8skjvfm/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'de664fb4-4f35-442a-b163-c62d6f4ff8d0-sda', '1717345488.75132', '1717345488.00000', '0', '', '1717345488.00000', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_conf_params test.unit.container.test_updater.TestContainerUpdater.test_conf_params ... ok test.unit.container.test_updater.TestContainerUpdater.test_creation test.unit.container.test_updater.TestContainerUpdater.test_creation ... ok test.unit.container.test_updater.TestContainerUpdater.test_error_in_process test.unit.container.test_updater.TestContainerUpdater.test_error_in_process ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: txf6ba24def7af4425956d0-00665c9ce2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf6ba24def7af4425956d0-00665c9ce2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpz8__ff8f/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpz8__ff8f/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpz8__ff8f/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpz8__ff8f/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0124 "-" 3542019 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0093 "-" 3542019 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0092 "-" 3542019 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0095 "-" 3542019 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx06843e593dd64fe4bbded-00665c9ce2" "proxy-server 3542019" 0.0018 "-" 3542019 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:59701/sda1/1/a/c" "tx06843e593dd64fe4bbded-00665c9ce2" "container-server 3542019" 0.0014 "-" 3542019 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:57395/sdb1/1/a/c" "tx06843e593dd64fe4bbded-00665c9ce2" "container-server 3542019" 0.0013 "-" 3542019 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx06843e593dd64fe4bbded-00665c9ce2" "proxy-server 3542019" 0.0180 "-" 3542019 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx06843e593dd64fe4bbded-00665c9ce2" "proxy-server 3542019" 0.0309 "-" 3542019 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/06 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx06843e593dd64fe4bbded-00665c9ce2 - 0.0406 - - 1717345506.623571873 1717345506.664129734 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx1e8c7bcaf12b4ed69c20a-00665c9ce2" "proxy-server 3542019" 0.0015 "-" 3542019 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:59701/sda1/1/a1/c1" "tx1e8c7bcaf12b4ed69c20a-00665c9ce2" "container-server 3542019" 0.0015 "-" 3542019 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:57395/sdb1/1/a1/c1" "tx1e8c7bcaf12b4ed69c20a-00665c9ce2" "container-server 3542019" 0.0014 "-" 3542019 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1e8c7bcaf12b4ed69c20a-00665c9ce2" "proxy-server 3542019" 0.0180 "-" 3542019 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1e8c7bcaf12b4ed69c20a-00665c9ce2" "proxy-server 3542019" 0.0334 "-" 3542019 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/06 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx1e8c7bcaf12b4ed69c20a-00665c9ce2 - 0.0426 - - 1717345506.665896893 1717345506.708482981 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txeae5a359ab474717a0469-00665c9ce2" "proxy-server 3542019" 0.0072 "-" 3542019 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59701/sda1/2/a/c1" "txeae5a359ab474717a0469-00665c9ce2" "container-server 3542019" 0.0062 "-" 3542019 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:57395/sdb1/2/a/c1" "txeae5a359ab474717a0469-00665c9ce2" "container-server 3542019" 0.0013 "-" 3542019 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txeae5a359ab474717a0469-00665c9ce2" "proxy-server 3542019" 0.0423 "-" 3542019 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txeae5a359ab474717a0469-00665c9ce2" "proxy-server 3542019" 0.0252 "-" 3542019 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/06 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txeae5a359ab474717a0469-00665c9ce2 - 0.0584 - - 1717345506.710194588 1717345506.768603086 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9debe338d5ab44e9af20e-00665c9ce2" "proxy-server 3542019" 0.0062 "-" 3542019 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:57395/sdb1/1/a/c2" "tx9debe338d5ab44e9af20e-00665c9ce2" "container-server 3542019" 0.0014 "-" 3542019 2 test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:59701/sda1/1/a/c2" "tx9debe338d5ab44e9af20e-00665c9ce2" "container-server 3542019" 0.0058 "-" 3542019 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx9debe338d5ab44e9af20e-00665c9ce2" "proxy-server 3542019" 0.0203 "-" 3542019 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:06 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx9debe338d5ab44e9af20e-00665c9ce2" "proxy-server 3542019" 0.0317 "-" 3542019 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/06 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx9debe338d5ab44e9af20e-00665c9ce2 - 0.0460 - - 1717345506.770352125 1717345506.816309690 2 test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txe7f7296c249f4bdeb0437-00665c9cd5) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txe7f7296c249f4bdeb0437-00665c9cd5) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe7f7296c249f4bdeb0437-00665c9cd5) test ERROR: Account HEAD returning 503 for [] (txn: txe7f7296c249f4bdeb0437-00665c9cd5) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx388d650187d146e78fa02-00665c9cd7) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx388d650187d146e78fa02-00665c9cd7) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx388d650187d146e78fa02-00665c9cd7) test ERROR: Account HEAD returning 503 for [] (txn: tx388d650187d146e78fa02-00665c9cd7) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txb7df43de249547c1bf0f1-00665c9cd9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: txde493294e9ac44b48d921-00665c9cd9) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: txde493294e9ac44b48d921-00665c9cd9) test INFO: Client disconnected on read of '/a/c' (txn: txde493294e9ac44b48d921-00665c9cd9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx00a09ca7eb2c464eb7ebc-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txaf71d4ec445544a6ab581-00665c9cd9) test INFO: Client disconnected on read of '/a/c' (txn: txaf71d4ec445544a6ab581-00665c9cd9) test INFO: Client disconnected on read of '/a/c/' (txn: txaf71d4ec445544a6ab581-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx8acc5e116914444e96b9d-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx8acc5e116914444e96b9d-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx8acc5e116914444e96b9d-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx8acc5e116914444e96b9d-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txc1360696e83248f7afc87-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txc1360696e83248f7afc87-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txc1360696e83248f7afc87-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txc1360696e83248f7afc87-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx5e613501e1354a9295956-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx5e613501e1354a9295956-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx5e613501e1354a9295956-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx5e613501e1354a9295956-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txe1bb4ef6b0574445b6b9a-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txe1bb4ef6b0574445b6b9a-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txe1bb4ef6b0574445b6b9a-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txe1bb4ef6b0574445b6b9a-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx37562cdb043e4fc898b3d-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx37562cdb043e4fc898b3d-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx37562cdb043e4fc898b3d-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx37562cdb043e4fc898b3d-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx74609bf9487e4db98fdd6-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx74609bf9487e4db98fdd6-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx74609bf9487e4db98fdd6-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx74609bf9487e4db98fdd6-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx720423fae220462283db5-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx720423fae220462283db5-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx720423fae220462283db5-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx720423fae220462283db5-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx13ce4c0c8840495a84638-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx13ce4c0c8840495a84638-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx13ce4c0c8840495a84638-00665c9cd9) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx13ce4c0c8840495a84638-00665c9cd9) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx1ff76985a4cc4fa7a1ba0-00665c9cdb) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: tx1ff76985a4cc4fa7a1ba0-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx1ff76985a4cc4fa7a1ba0-00665c9cdb) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx1ff76985a4cc4fa7a1ba0-00665c9cdb) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx1ff76985a4cc4fa7a1ba0-00665c9cdb) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx1ff76985a4cc4fa7a1ba0-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7ba8d8e49fbc416d86c81-00665c9cdb) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx7ba8d8e49fbc416d86c81-00665c9cdb) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx7ba8d8e49fbc416d86c81-00665c9cdb) test ERROR: Container GET returning 503 for [503, 503] (txn: tx7ba8d8e49fbc416d86c81-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx914a29ed8caf41699d649-00665c9cdb) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx914a29ed8caf41699d649-00665c9cdb) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx914a29ed8caf41699d649-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx914a29ed8caf41699d649-00665c9cdb) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx914a29ed8caf41699d649-00665c9cdb) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx914a29ed8caf41699d649-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx914a29ed8caf41699d649-00665c9cdb) test ERROR: Container GET returning 503 for [503, 503] (txn: tx914a29ed8caf41699d649-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx6180aa1f1f8a4008a19e9-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx6180aa1f1f8a4008a19e9-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx6180aa1f1f8a4008a19e9-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx6180aa1f1f8a4008a19e9-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx57ab411e47f3434289561-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx57ab411e47f3434289561-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx57ab411e47f3434289561-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx57ab411e47f3434289561-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txf6a75ce3387c4c37b1865-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txf6a75ce3387c4c37b1865-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txf6a75ce3387c4c37b1865-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: txf6a75ce3387c4c37b1865-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa4f83e63b107489292952-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa4f83e63b107489292952-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa4f83e63b107489292952-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: txa4f83e63b107489292952-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx90889282c7be4999bac2e-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx90889282c7be4999bac2e-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx90889282c7be4999bac2e-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx90889282c7be4999bac2e-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txe13c1f4697f0436c8cbd5-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txe13c1f4697f0436c8cbd5-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txe13c1f4697f0436c8cbd5-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: txe13c1f4697f0436c8cbd5-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx441732b8f7ce448db7175-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx441732b8f7ce448db7175-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx441732b8f7ce448db7175-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx441732b8f7ce448db7175-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx1844b19a05254235b3fef-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx1844b19a05254235b3fef-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx1844b19a05254235b3fef-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx1844b19a05254235b3fef-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx91f373a1e7b44f0bba24c-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx91f373a1e7b44f0bba24c-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx91f373a1e7b44f0bba24c-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx91f373a1e7b44f0bba24c-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx74e529edd71c4868ab540-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx74e529edd71c4868ab540-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx74e529edd71c4868ab540-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx74e529edd71c4868ab540-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx8ff90610d844488fb8e39-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx8ff90610d844488fb8e39-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx8ff90610d844488fb8e39-00665c9cdb) test ERROR: Container GET returning 503 for [] (txn: tx8ff90610d844488fb8e39-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx35e7435c838946a9bf6c9-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx35e7435c838946a9bf6c9-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx35e7435c838946a9bf6c9-00665c9cdb) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx35e7435c838946a9bf6c9-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx1e74ca8c74a349e09d905-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx1e74ca8c74a349e09d905-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx1e74ca8c74a349e09d905-00665c9cdb) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx1e74ca8c74a349e09d905-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx790b79975fe44333b8fbe-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx790b79975fe44333b8fbe-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx790b79975fe44333b8fbe-00665c9cdb) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx790b79975fe44333b8fbe-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7a3c4e9fc0bc4de1a039b-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7a3c4e9fc0bc4de1a039b-00665c9cdb) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7a3c4e9fc0bc4de1a039b-00665c9cdb) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx7a3c4e9fc0bc4de1a039b-00665c9cdb) test.unit.proxy.test_server.TestContainerController.test_error_limiting test.unit.proxy.test_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_transfer_headers test.unit.proxy.test_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345494.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok test INFO: Client disconnected on read of '/a/c' (txn: txb46fac585abf481cb5ea8-00665c9cdb) test INFO: Client disconnected on read of '/a/c/' (txn: txb46fac585abf481cb5ea8-00665c9cdb) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txd3898b743ebd415a88b8a-00665c9cdd) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txd3898b743ebd415a88b8a-00665c9cdd) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txd3898b743ebd415a88b8a-00665c9cdd) test ERROR: Account HEAD returning 503 for [] (txn: txd3898b743ebd415a88b8a-00665c9cdd) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx17c55e5f3e7b473c83eb1-00665c9cdd) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx17c55e5f3e7b473c83eb1-00665c9cdd) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx17c55e5f3e7b473c83eb1-00665c9cdd) test ERROR: Account HEAD returning 503 for [] (txn: tx17c55e5f3e7b473c83eb1-00665c9cdd) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx0004109a4a684087a39db-00665c9cde) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx1fafbeb35c94489e8bc17-00665c9cde) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txef4bf0952cf2488ab111b-00665c9cde) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('de664fb4-4f35-442a-b163-c62d6f4ff8d0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'de664fb4-4f35-442a-b163-c62d6f4ff8d0-sda', '1717345488.75132', '1717345488.00000', '0', '', '1717345488.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('de664fb4-4f35-442a-b163-c62d6f4ff8d0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 3.46755 seconds (0.57678/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:4 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpn8skjvfm/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', 'c9276111-d867-4d9c-93f4-fb8d577de90a-sda', '1717345488.76360', '1717345489.00000', '0', '{"X-Container-Sync-To": ["", "1717345492.00000"], "X-Container-Sync-Key": ["secret", "1717345491.00000"]}', '1717345489.00000', 0, 0) test DEBUG: /tmp/tmpn8skjvfm/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', 'c9276111-d867-4d9c-93f4-fb8d577de90a-sda', '1717345488.76360', '1717345489.00000', '0', '{"X-Container-Sync-To": ["", "1717345492.00000"], "X-Container-Sync-Key": ["secret", "1717345491.00000"]}', '1717345489.00000', 0, 0) test DEBUG: /tmp/tmpn8skjvfm/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.2:1002/sdc, nothing to do test DEBUG: Replicating db /tmp/tmpn8skjvfm/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'de664fb4-4f35-442a-b163-c62d6f4ff8d0-sda', '1717345488.75132', '1717345488.00000', '0', '', '1717345488.00000', 0, 0) test DEBUG: /tmp/tmpn8skjvfm/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'de664fb4-4f35-442a-b163-c62d6f4ff8d0-sda', '1717345488.75132', '1717345488.00000', '0', '', '1717345488.00000', 0, 0) test DEBUG: /tmp/tmpn8skjvfm/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.04304 seconds (46.46619/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:4 remote_merge:0 rsync:0 ts_repl:0 test ERROR: Failed to update sync_store /tmp/tmpjsy0qaet/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/replicator.py", line 301, in _post_replicate_hook self.sync_store.update_sync_store(broker) File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_replicator.py", line 1226, in update_sync_store raise OSError(1, '1') PermissionError: [Errno 1] 1 test INFO: Begin container update single threaded sweep test ERROR: Error processing container /tmp/tmpg4mvw_6m/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db: Boom!: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/updater.py", line 225, in container_sweep self.process_container(dbfile) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Boom! test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: ERROR: Failed to get paths to drive partitions: permission_denied test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpyfbbutfh/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpyfbbutfh/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once test.unit.container.test_updater.TestContainerUpdater.test_run_once ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27afc4020c2f44e597ee6-00665c9cdb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpyfbbutfh/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmp_j2casti/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmp_j2casti/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/c /tmp/tmp_j2casti/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test WARNING: /tmp/tmp7a_k53f0/tmp_test_container_updater/devices/sda1 is not mounted test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Failed to get container info (Lock timeout: 10 seconds: /tmp/tmp1hgjlub8/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db); skipping. test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpc87rvgyz/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test.unit.container.test_updater.TestContainerUpdater.test_shard_container test.unit.container.test_updater.TestContainerUpdater.test_shard_container ... ok test.unit.container.test_updater.TestContainerUpdater.test_unicode test.unit.container.test_updater.TestContainerUpdater.test_unicode ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345499.03911', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345499.05967', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpc87rvgyz/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpc87rvgyz/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/Ω /tmp/tmpqbz4ck57/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx3807b8202de6448198db8-00665c9ce1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx40d12bb860014caf890fd-00665c9ce1) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp_hgrxnk4/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp_hgrxnk4/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp_hgrxnk4/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp_hgrxnk4/tmp_test_proxy_server_chunked/container-sync-realms.conf' test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a8c79da835f4e67849d2-00665c9cdb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:25:02 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345502.06055', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:25:02 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345502.07939', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission ... ok cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0126 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0124 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0137 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0123 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe72d242b029d4d18a209a-00665c9ce4" "proxy-server 3542025" 0.0017 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:57637/sdb1/3/a/c" "txe72d242b029d4d18a209a-00665c9ce4" "container-server 3542025" 0.0019 "-" 3542025 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:52179/sda1/3/a/c" "txe72d242b029d4d18a209a-00665c9ce4" "container-server 3542025" 0.0015 "-" 3542025 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txe72d242b029d4d18a209a-00665c9ce4" "proxy-server 3542025" 0.0212 "-" 3542025 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txe72d242b029d4d18a209a-00665c9ce4" "proxy-server 3542025" 0.0338 "-" 3542025 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/08 PUT /v1/a/c HTTP/1.0 201 - - t - - - txe72d242b029d4d18a209a-00665c9ce4 - 0.0441 - - 1717345508.752757072 1717345508.796856403 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txf166c03392104fcea34ba-00665c9ce4" "proxy-server 3542025" 0.0017 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:57637/sdb1/0/a1/c1" "txf166c03392104fcea34ba-00665c9ce4" "container-server 3542025" 0.0015 "-" 3542025 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:52179/sda1/0/a1/c1" "txf166c03392104fcea34ba-00665c9ce4" "container-server 3542025" 0.0014 "-" 3542025 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txf166c03392104fcea34ba-00665c9ce4" "proxy-server 3542025" 0.0401 "-" 3542025 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txf166c03392104fcea34ba-00665c9ce4" "proxy-server 3542025" 0.0199 "-" 3542025 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/08 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txf166c03392104fcea34ba-00665c9ce4 - 0.0519 - - 1717345508.798828125 1717345508.850688219 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txeba2aa80b6be4c07ae9de-00665c9ce4" "proxy-server 3542025" 0.0080 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:52179/sda1/3/a/c1" "txeba2aa80b6be4c07ae9de-00665c9ce4" "container-server 3542025" 0.0016 "-" 3542025 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:57637/sdb1/3/a/c1" "txeba2aa80b6be4c07ae9de-00665c9ce4" "container-server 3542025" 0.0095 "-" 3542025 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txeba2aa80b6be4c07ae9de-00665c9ce4" "proxy-server 3542025" 0.0654 "-" 3542025 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txeba2aa80b6be4c07ae9de-00665c9ce4" "proxy-server 3542025" 0.0534 "-" 3542025 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/08 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txeba2aa80b6be4c07ae9de-00665c9ce4 - 0.0845 - - 1717345508.852923393 1717345508.937425613 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9c98a3f02ed145798e9f9-00665c9ce4" "proxy-server 3542025" 0.0072 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:57637/sdb1/0/a/c2" "tx9c98a3f02ed145798e9f9-00665c9ce4" "container-server 3542025" 0.0015 "-" 3542025 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:52179/sda1/0/a/c2" "tx9c98a3f02ed145798e9f9-00665c9ce4" "container-server 3542025" 0.0111 "-" 3542025 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx9c98a3f02ed145798e9f9-00665c9ce4" "proxy-server 3542025" 0.0316 "-" 3542025 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:08 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx9c98a3f02ed145798e9f9-00665c9ce4" "proxy-server 3542025" 0.0448 "-" 3542025 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/09 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx9c98a3f02ed145798e9f9-00665c9ce4 - 0.0612 - - 1717345508.939881563 1717345509.001073360 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpw830lfa6/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpw830lfa6/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpw830lfa6/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpw830lfa6/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0167 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0112 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0129 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0102 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx03414c4ceef2419b9a691-00665c9ce5" "proxy-server 3542025" 0.0019 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:55571/sdb1/3/a/c" "tx03414c4ceef2419b9a691-00665c9ce5" "container-server 3542025" 0.0037 "-" 3542025 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:59527/sda1/3/a/c" "tx03414c4ceef2419b9a691-00665c9ce5" "container-server 3542025" 0.0015 "-" 3542025 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx03414c4ceef2419b9a691-00665c9ce5" "proxy-server 3542025" 0.0282 "-" 3542025 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx03414c4ceef2419b9a691-00665c9ce5" "proxy-server 3542025" 0.0526 "-" 3542025 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/09 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx03414c4ceef2419b9a691-00665c9ce5 - 0.0634 - - 1717345509.258055925 1717345509.321490526 0 test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82ead802b53a4835b1e99-00665c9cdf) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:09 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx8b002bb9393f409f87ca4-00665c9ce5" "proxy-server 3542025" 0.0021 "-" 3542025 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:55571/sdb1/0/a1/c1" "tx8b002bb9393f409f87ca4-00665c9ce5" "container-server 3542025" 0.0014 "-" 3542025 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:59527/sda1/0/a1/c1" "tx8b002bb9393f409f87ca4-00665c9ce5" "container-server 3542025" 0.0013 "-" 3542025 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx8b002bb9393f409f87ca4-00665c9ce5" "proxy-server 3542025" 0.0292 "-" 3542025 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx8b002bb9393f409f87ca4-00665c9ce5" "proxy-server 3542025" 0.0174 "-" 3542025 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/12 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx8b002bb9393f409f87ca4-00665c9ce5 - 3.0754 - - 1717345509.323414564 1717345512.398861170 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx989e1aec1fec44c2b1299-00665c9ce8" "proxy-server 3542025" 0.0098 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:55571/sdb1/3/a/c1" "tx989e1aec1fec44c2b1299-00665c9ce8" "container-server 3542025" 0.0033 "-" 3542025 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:59527/sda1/3/a/c1" "tx989e1aec1fec44c2b1299-00665c9ce8" "container-server 3542025" 0.0062 "-" 3542025 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx989e1aec1fec44c2b1299-00665c9ce8" "proxy-server 3542025" 0.0358 "-" 3542025 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx989e1aec1fec44c2b1299-00665c9ce8" "proxy-server 3542025" 0.0243 "-" 3542025 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/12 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx989e1aec1fec44c2b1299-00665c9ce8 - 0.0551 - - 1717345512.400686741 1717345512.455808163 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3274c375c11f488aa5d94-00665c9ce8" "proxy-server 3542025" 0.0071 "-" 3542025 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:59527/sda1/0/a/c2" "tx3274c375c11f488aa5d94-00665c9ce8" "container-server 3542025" 0.0015 "-" 3542025 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:55571/sdb1/0/a/c2" "tx3274c375c11f488aa5d94-00665c9ce8" "container-server 3542025" 0.0057 "-" 3542025 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3274c375c11f488aa5d94-00665c9ce8" "proxy-server 3542025" 0.0322 "-" 3542025 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:12 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3274c375c11f488aa5d94-00665c9ce8" "proxy-server 3542025" 0.0213 "-" 3542025 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/12 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx3274c375c11f488aa5d94-00665c9ce8 - 0.0491 - - 1717345512.457698584 1717345512.506766558 2 test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:24:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7e40d0eda7aa4be782c53-00665c9ce3) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.95384', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.95734', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.96024', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.96297', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.96557', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.96945', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.97216', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.97531', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.97801', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.98978', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345505.99245', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded ... ok test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test WARNING: boom sharder-test WARNING: Reclaimable db stuck waiting for shrinking, path: a/c, db: /tmp/tmp2w6smgza/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717345494.00000, (0, 0) as of 1717345494.00000, active as of 1717345494.00000> to ShardRange<'k' to 't' as of 1717345495.00000, (0, 0) as of 1717345495.00000, active as of 1717345495.00000>, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345499.00000, (0, 0) as of 1717345499.00000, shrinking as of 1717345496.00000> to ShardRange<'k' to 't' as of 1717345499.00000, (0, 0) as of 1717345499.00000, sharding as of 1717345500.00000>, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpdzh_200m/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345495.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/339/d4ef3a91a4a611dba8cbee2b07be0339/d4ef3a91a4a611dba8cbee2b07be0339.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345497.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a55/b25b1186a77bc0770fe28c52ca383a55/b25b1186a77bc0770fe28c52ca383a55.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345496.00000, (0, 0) as of 1717345496.00000, found as of 1717345496.00000> to ShardRange<'k' to 't' as of 1717345497.00000, (0, 0) as of 1717345497.00000, created as of 1717345497.00000>, path: .shards_a/shard_c_1717345497.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a55/b25b1186a77bc0770fe28c52ca383a55/b25b1186a77bc0770fe28c52ca383a55.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345499.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f3d/da56fcf8bb044eaa4d7a95aa0627ff3d/da56fcf8bb044eaa4d7a95aa0627ff3d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345498.00000, (0, 0) as of 1717345498.00000, found as of 1717345498.00000> to ShardRange<'k' to 't' as of 1717345499.00000, (0, 0) as of 1717345499.00000, cleaved as of 1717345499.00000>, path: .shards_a/shard_c_1717345499.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f3d/da56fcf8bb044eaa4d7a95aa0627ff3d/da56fcf8bb044eaa4d7a95aa0627ff3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345501.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/87c/d2642570799551004dee1e3417b4887c/d2642570799551004dee1e3417b4887c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345500.00000, (0, 0) as of 1717345500.00000, found as of 1717345500.00000> to ShardRange<'k' to 't' as of 1717345501.00000, (0, 0) as of 1717345501.00000, active as of 1717345501.00000>, path: .shards_a/shard_c_1717345501.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/87c/d2642570799551004dee1e3417b4887c/d2642570799551004dee1e3417b4887c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345503.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3cd/7b0659f0466b3c6ae91d8310f0eb73cd/7b0659f0466b3c6ae91d8310f0eb73cd.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345502.00000, (0, 0) as of 1717345502.00000, found as of 1717345502.00000> to ShardRange<'k' to 't' as of 1717345503.00000, (0, 0) as of 1717345503.00000, shrinking as of 1717345503.00000>, path: .shards_a/shard_c_1717345503.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3cd/7b0659f0466b3c6ae91d8310f0eb73cd/7b0659f0466b3c6ae91d8310f0eb73cd.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345503.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3cd/7b0659f0466b3c6ae91d8310f0eb73cd/7b0659f0466b3c6ae91d8310f0eb73cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345505.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/b9a/b6f85988984b32c9884b1f5bf35b7b9a/b6f85988984b32c9884b1f5bf35b7b9a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345504.00000, (0, 0) as of 1717345504.00000, found as of 1717345504.00000> to ShardRange<'k' to 't' as of 1717345505.00000, (0, 0) as of 1717345505.00000, sharding as of 1717345505.00000>, path: .shards_a/shard_c_1717345505.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/b9a/b6f85988984b32c9884b1f5bf35b7b9a/b6f85988984b32c9884b1f5bf35b7b9a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345505.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/b9a/b6f85988984b32c9884b1f5bf35b7b9a/b6f85988984b32c9884b1f5bf35b7b9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345507.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6b1/59d3625004a3b3926e2707385c7856b1/59d3625004a3b3926e2707385c7856b1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345506.00000, (0, 0) as of 1717345506.00000, found as of 1717345506.00000> to ShardRange<'k' to 't' as of 1717345507.00000, (0, 0) as of 1717345507.00000, sharded as of 1717345507.00000>, path: .shards_a/shard_c_1717345507.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6b1/59d3625004a3b3926e2707385c7856b1/59d3625004a3b3926e2707385c7856b1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345507.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6b1/59d3625004a3b3926e2707385c7856b1/59d3625004a3b3926e2707385c7856b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345509.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0e5/8f940b3d7ced89b4dfb633cb5930b0e5/8f940b3d7ced89b4dfb633cb5930b0e5.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345508.00000, (0, 0) as of 1717345508.00000, found as of 1717345508.00000> to ShardRange<'k' to 't' as of 1717345509.00000, (0, 0) as of 1717345509.00000, shrunk as of 1717345509.00000>, path: .shards_a/shard_c_1717345509.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0e5/8f940b3d7ced89b4dfb633cb5930b0e5/8f940b3d7ced89b4dfb633cb5930b0e5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345509.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0e5/8f940b3d7ced89b4dfb633cb5930b0e5/8f940b3d7ced89b4dfb633cb5930b0e5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345511.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/125/7ffc5edf2147ea9538d08e10de196125/7ffc5edf2147ea9538d08e10de196125.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345510.00000, (0, 0) as of 1717345510.00000, created as of 1717345510.00000> to ShardRange<'k' to 't' as of 1717345511.00000, (0, 0) as of 1717345511.00000, found as of 1717345511.00000>, path: .shards_a/shard_c_1717345511.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/125/7ffc5edf2147ea9538d08e10de196125/7ffc5edf2147ea9538d08e10de196125.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345513.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0c4/ccfb05961025772c318cc00fe9e2f0c4/ccfb05961025772c318cc00fe9e2f0c4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345515.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e34/ce1b6c29eb8b0e9340876dc5f6affe34/ce1b6c29eb8b0e9340876dc5f6affe34.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345514.00000, (0, 0) as of 1717345514.00000, created as of 1717345514.00000> to ShardRange<'k' to 't' as of 1717345515.00000, (0, 0) as of 1717345515.00000, cleaved as of 1717345515.00000>, path: .shards_a/shard_c_1717345515.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e34/ce1b6c29eb8b0e9340876dc5f6affe34/ce1b6c29eb8b0e9340876dc5f6affe34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345517.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7b2/a1af201c6f3ec15cfcfec75e2d2d17b2/a1af201c6f3ec15cfcfec75e2d2d17b2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345516.00000, (0, 0) as of 1717345516.00000, created as of 1717345516.00000> to ShardRange<'k' to 't' as of 1717345517.00000, (0, 0) as of 1717345517.00000, active as of 1717345517.00000>, path: .shards_a/shard_c_1717345517.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7b2/a1af201c6f3ec15cfcfec75e2d2d17b2/a1af201c6f3ec15cfcfec75e2d2d17b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345519.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a8c/4c75ba087b625e381114c35c5d8eea8c/4c75ba087b625e381114c35c5d8eea8c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345518.00000, (0, 0) as of 1717345518.00000, created as of 1717345518.00000> to ShardRange<'k' to 't' as of 1717345519.00000, (0, 0) as of 1717345519.00000, shrinking as of 1717345519.00000>, path: .shards_a/shard_c_1717345519.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a8c/4c75ba087b625e381114c35c5d8eea8c/4c75ba087b625e381114c35c5d8eea8c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345519.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a8c/4c75ba087b625e381114c35c5d8eea8c/4c75ba087b625e381114c35c5d8eea8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345521.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/9f8/5be93aa34c3c3bd4f0a93902a875a9f8/5be93aa34c3c3bd4f0a93902a875a9f8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345520.00000, (0, 0) as of 1717345520.00000, created as of 1717345520.00000> to ShardRange<'k' to 't' as of 1717345521.00000, (0, 0) as of 1717345521.00000, sharding as of 1717345521.00000>, path: .shards_a/shard_c_1717345521.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/9f8/5be93aa34c3c3bd4f0a93902a875a9f8/5be93aa34c3c3bd4f0a93902a875a9f8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345521.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/9f8/5be93aa34c3c3bd4f0a93902a875a9f8/5be93aa34c3c3bd4f0a93902a875a9f8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345523.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4ec/d0033c95762bd214819a33109bf5f4ec/d0033c95762bd214819a33109bf5f4ec.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345522.00000, (0, 0) as of 1717345522.00000, created as of 1717345522.00000> to ShardRange<'k' to 't' as of 1717345523.00000, (0, 0) as of 1717345523.00000, sharded as of 1717345523.00000>, path: .shards_a/shard_c_1717345523.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4ec/d0033c95762bd214819a33109bf5f4ec/d0033c95762bd214819a33109bf5f4ec.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345523.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4ec/d0033c95762bd214819a33109bf5f4ec/d0033c95762bd214819a33109bf5f4ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345525.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6ea/a4c17c0133579c6c455318b02a51a6ea/a4c17c0133579c6c455318b02a51a6ea.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345524.00000, (0, 0) as of 1717345524.00000, created as of 1717345524.00000> to ShardRange<'k' to 't' as of 1717345525.00000, (0, 0) as of 1717345525.00000, shrunk as of 1717345525.00000>, path: .shards_a/shard_c_1717345525.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6ea/a4c17c0133579c6c455318b02a51a6ea/a4c17c0133579c6c455318b02a51a6ea.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345525.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6ea/a4c17c0133579c6c455318b02a51a6ea/a4c17c0133579c6c455318b02a51a6ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345527.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/16e/b795fdadd338f3634cafe8237d7b116e/b795fdadd338f3634cafe8237d7b116e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345526.00000, (0, 0) as of 1717345526.00000, cleaved as of 1717345526.00000> to ShardRange<'k' to 't' as of 1717345527.00000, (0, 0) as of 1717345527.00000, found as of 1717345527.00000>, path: .shards_a/shard_c_1717345527.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/16e/b795fdadd338f3634cafe8237d7b116e/b795fdadd338f3634cafe8237d7b116e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345529.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f23/6ea27c516bbf51b3d8132f23b7f7cf23/6ea27c516bbf51b3d8132f23b7f7cf23.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345528.00000, (0, 0) as of 1717345528.00000, cleaved as of 1717345528.00000> to ShardRange<'k' to 't' as of 1717345529.00000, (0, 0) as of 1717345529.00000, created as of 1717345529.00000>, path: .shards_a/shard_c_1717345529.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f23/6ea27c516bbf51b3d8132f23b7f7cf23/6ea27c516bbf51b3d8132f23b7f7cf23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345531.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d8b/ee832282cac861ae8dc4f9c4bf51fd8b/ee832282cac861ae8dc4f9c4bf51fd8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345533.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d17/79ecc70e5c3c5bbd5583828c56f86d17/79ecc70e5c3c5bbd5583828c56f86d17.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345532.00000, (0, 0) as of 1717345532.00000, cleaved as of 1717345532.00000> to ShardRange<'k' to 't' as of 1717345533.00000, (0, 0) as of 1717345533.00000, active as of 1717345533.00000>, path: .shards_a/shard_c_1717345533.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d17/79ecc70e5c3c5bbd5583828c56f86d17/79ecc70e5c3c5bbd5583828c56f86d17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345535.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/14b/d789d53d37d33a119251826521f3d14b/d789d53d37d33a119251826521f3d14b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345534.00000, (0, 0) as of 1717345534.00000, cleaved as of 1717345534.00000> to ShardRange<'k' to 't' as of 1717345535.00000, (0, 0) as of 1717345535.00000, shrinking as of 1717345535.00000>, path: .shards_a/shard_c_1717345535.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/14b/d789d53d37d33a119251826521f3d14b/d789d53d37d33a119251826521f3d14b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345535.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/14b/d789d53d37d33a119251826521f3d14b/d789d53d37d33a119251826521f3d14b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345537.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/58d/ce559ed708cc19e33754eda9191db58d/ce559ed708cc19e33754eda9191db58d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345536.00000, (0, 0) as of 1717345536.00000, cleaved as of 1717345536.00000> to ShardRange<'k' to 't' as of 1717345537.00000, (0, 0) as of 1717345537.00000, sharding as of 1717345537.00000>, path: .shards_a/shard_c_1717345537.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/58d/ce559ed708cc19e33754eda9191db58d/ce559ed708cc19e33754eda9191db58d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345537.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/58d/ce559ed708cc19e33754eda9191db58d/ce559ed708cc19e33754eda9191db58d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345539.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e4f/f07c22f3c8580c5fb077e952ec085e4f/f07c22f3c8580c5fb077e952ec085e4f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345538.00000, (0, 0) as of 1717345538.00000, cleaved as of 1717345538.00000> to ShardRange<'k' to 't' as of 1717345539.00000, (0, 0) as of 1717345539.00000, sharded as of 1717345539.00000>, path: .shards_a/shard_c_1717345539.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e4f/f07c22f3c8580c5fb077e952ec085e4f/f07c22f3c8580c5fb077e952ec085e4f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345539.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e4f/f07c22f3c8580c5fb077e952ec085e4f/f07c22f3c8580c5fb077e952ec085e4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345541.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d7d/404a3150b54a88e0b4ded16db17b6d7d/404a3150b54a88e0b4ded16db17b6d7d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345540.00000, (0, 0) as of 1717345540.00000, cleaved as of 1717345540.00000> to ShardRange<'k' to 't' as of 1717345541.00000, (0, 0) as of 1717345541.00000, shrunk as of 1717345541.00000>, path: .shards_a/shard_c_1717345541.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d7d/404a3150b54a88e0b4ded16db17b6d7d/404a3150b54a88e0b4ded16db17b6d7d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345541.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d7d/404a3150b54a88e0b4ded16db17b6d7d/404a3150b54a88e0b4ded16db17b6d7d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345543.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d74/aa35b90f46fa35ab9b1ff1badf8c1d74/aa35b90f46fa35ab9b1ff1badf8c1d74.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345542.00000, (0, 0) as of 1717345542.00000, active as of 1717345542.00000> to ShardRange<'k' to 't' as of 1717345543.00000, (0, 0) as of 1717345543.00000, found as of 1717345543.00000>, path: .shards_a/shard_c_1717345543.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d74/aa35b90f46fa35ab9b1ff1badf8c1d74/aa35b90f46fa35ab9b1ff1badf8c1d74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345545.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0e3/2cc3bd6a12d87de1f10f61704c81a0e3/2cc3bd6a12d87de1f10f61704c81a0e3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345544.00000, (0, 0) as of 1717345544.00000, active as of 1717345544.00000> to ShardRange<'k' to 't' as of 1717345545.00000, (0, 0) as of 1717345545.00000, created as of 1717345545.00000>, path: .shards_a/shard_c_1717345545.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0e3/2cc3bd6a12d87de1f10f61704c81a0e3/2cc3bd6a12d87de1f10f61704c81a0e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345547.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7db/d32c3b83e59039cf1b51f0c6228ca7db/d32c3b83e59039cf1b51f0c6228ca7db.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345546.00000, (0, 0) as of 1717345546.00000, active as of 1717345546.00000> to ShardRange<'k' to 't' as of 1717345547.00000, (0, 0) as of 1717345547.00000, cleaved as of 1717345547.00000>, path: .shards_a/shard_c_1717345547.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7db/d32c3b83e59039cf1b51f0c6228ca7db/d32c3b83e59039cf1b51f0c6228ca7db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345549.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/28d/c53d51703f314f6af3439bb2952d128d/c53d51703f314f6af3439bb2952d128d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345551.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/90d/fdeae3878380afbdc57f90085ad1990d/fdeae3878380afbdc57f90085ad1990d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345550.00000, (0, 0) as of 1717345550.00000, active as of 1717345550.00000> to ShardRange<'k' to 't' as of 1717345551.00000, (0, 0) as of 1717345551.00000, shrinking as of 1717345551.00000>, path: .shards_a/shard_c_1717345551.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/90d/fdeae3878380afbdc57f90085ad1990d/fdeae3878380afbdc57f90085ad1990d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345551.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/90d/fdeae3878380afbdc57f90085ad1990d/fdeae3878380afbdc57f90085ad1990d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345553.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/148/4db13ae8a6d340d05f7550b5d4382148/4db13ae8a6d340d05f7550b5d4382148.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345552.00000, (0, 0) as of 1717345552.00000, active as of 1717345552.00000> to ShardRange<'k' to 't' as of 1717345553.00000, (0, 0) as of 1717345553.00000, sharding as of 1717345553.00000>, path: .shards_a/shard_c_1717345553.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/148/4db13ae8a6d340d05f7550b5d4382148/4db13ae8a6d340d05f7550b5d4382148.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345553.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/148/4db13ae8a6d340d05f7550b5d4382148/4db13ae8a6d340d05f7550b5d4382148.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345555.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3eb/cdeee04938fd768d2ca8f2ebed4133eb/cdeee04938fd768d2ca8f2ebed4133eb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345554.00000, (0, 0) as of 1717345554.00000, active as of 1717345554.00000> to ShardRange<'k' to 't' as of 1717345555.00000, (0, 0) as of 1717345555.00000, sharded as of 1717345555.00000>, path: .shards_a/shard_c_1717345555.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3eb/cdeee04938fd768d2ca8f2ebed4133eb/cdeee04938fd768d2ca8f2ebed4133eb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345555.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3eb/cdeee04938fd768d2ca8f2ebed4133eb/cdeee04938fd768d2ca8f2ebed4133eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345557.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/891/79bbebe3e24eccb4596248123eb2c891/79bbebe3e24eccb4596248123eb2c891.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345556.00000, (0, 0) as of 1717345556.00000, active as of 1717345556.00000> to ShardRange<'k' to 't' as of 1717345557.00000, (0, 0) as of 1717345557.00000, shrunk as of 1717345557.00000>, path: .shards_a/shard_c_1717345557.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/891/79bbebe3e24eccb4596248123eb2c891/79bbebe3e24eccb4596248123eb2c891.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345557.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/891/79bbebe3e24eccb4596248123eb2c891/79bbebe3e24eccb4596248123eb2c891.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345559.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/33f/e6bde4368ff9d20ea3a7fc1ba673333f/e6bde4368ff9d20ea3a7fc1ba673333f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345558.00000, (0, 0) as of 1717345558.00000, shrinking as of 1717345558.00000> to ShardRange<'k' to 't' as of 1717345559.00000, (0, 0) as of 1717345559.00000, found as of 1717345559.00000>, path: .shards_a/shard_c_1717345559.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/33f/e6bde4368ff9d20ea3a7fc1ba673333f/e6bde4368ff9d20ea3a7fc1ba673333f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345561.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a02/6eba65cabd929e0543c8cbb418a5ba02/6eba65cabd929e0543c8cbb418a5ba02.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345560.00000, (0, 0) as of 1717345560.00000, shrinking as of 1717345560.00000> to ShardRange<'k' to 't' as of 1717345561.00000, (0, 0) as of 1717345561.00000, created as of 1717345561.00000>, path: .shards_a/shard_c_1717345561.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a02/6eba65cabd929e0543c8cbb418a5ba02/6eba65cabd929e0543c8cbb418a5ba02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345563.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/303/32d5fb13a80bf26fa446fb0078fd3303/32d5fb13a80bf26fa446fb0078fd3303.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345562.00000, (0, 0) as of 1717345562.00000, shrinking as of 1717345562.00000> to ShardRange<'k' to 't' as of 1717345563.00000, (0, 0) as of 1717345563.00000, cleaved as of 1717345563.00000>, path: .shards_a/shard_c_1717345563.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/303/32d5fb13a80bf26fa446fb0078fd3303/32d5fb13a80bf26fa446fb0078fd3303.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345565.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/1f8/4ebdca2454afeac84ccb4f25f78cb1f8/4ebdca2454afeac84ccb4f25f78cb1f8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345564.00000, (0, 0) as of 1717345564.00000, shrinking as of 1717345564.00000> to ShardRange<'k' to 't' as of 1717345565.00000, (0, 0) as of 1717345565.00000, active as of 1717345565.00000>, path: .shards_a/shard_c_1717345565.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/1f8/4ebdca2454afeac84ccb4f25f78cb1f8/4ebdca2454afeac84ccb4f25f78cb1f8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345567.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/b44/c8dc155eefb68ee321c3bf1b3a809b44/c8dc155eefb68ee321c3bf1b3a809b44.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345567.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/b44/c8dc155eefb68ee321c3bf1b3a809b44/c8dc155eefb68ee321c3bf1b3a809b44.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345569.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e05/a032b56226ad885df3a550e822594e05/a032b56226ad885df3a550e822594e05.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345568.00000, (0, 0) as of 1717345568.00000, shrinking as of 1717345568.00000> to ShardRange<'k' to 't' as of 1717345569.00000, (0, 0) as of 1717345569.00000, sharding as of 1717345569.00000>, path: .shards_a/shard_c_1717345569.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e05/a032b56226ad885df3a550e822594e05/a032b56226ad885df3a550e822594e05.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345569.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e05/a032b56226ad885df3a550e822594e05/a032b56226ad885df3a550e822594e05.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345571.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/692/a272be8b1c04dd44c51421fafffcb692/a272be8b1c04dd44c51421fafffcb692.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345570.00000, (0, 0) as of 1717345570.00000, shrinking as of 1717345570.00000> to ShardRange<'k' to 't' as of 1717345571.00000, (0, 0) as of 1717345571.00000, sharded as of 1717345571.00000>, path: .shards_a/shard_c_1717345571.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/692/a272be8b1c04dd44c51421fafffcb692/a272be8b1c04dd44c51421fafffcb692.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345571.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/692/a272be8b1c04dd44c51421fafffcb692/a272be8b1c04dd44c51421fafffcb692.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345573.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d70/68a28a59dcb920825874404837907d70/68a28a59dcb920825874404837907d70.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345572.00000, (0, 0) as of 1717345572.00000, shrinking as of 1717345572.00000> to ShardRange<'k' to 't' as of 1717345573.00000, (0, 0) as of 1717345573.00000, shrunk as of 1717345573.00000>, path: .shards_a/shard_c_1717345573.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d70/68a28a59dcb920825874404837907d70/68a28a59dcb920825874404837907d70.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345573.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d70/68a28a59dcb920825874404837907d70/68a28a59dcb920825874404837907d70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345575.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7a8/19ac830ef862fdc54e692aca1c89f7a8/19ac830ef862fdc54e692aca1c89f7a8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345574.00000, (0, 0) as of 1717345574.00000, sharding as of 1717345574.00000> to ShardRange<'k' to 't' as of 1717345575.00000, (0, 0) as of 1717345575.00000, found as of 1717345575.00000>, path: .shards_a/shard_c_1717345575.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7a8/19ac830ef862fdc54e692aca1c89f7a8/19ac830ef862fdc54e692aca1c89f7a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345577.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4cb/7d09529259c864749f13ef7dc7aad4cb/7d09529259c864749f13ef7dc7aad4cb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345576.00000, (0, 0) as of 1717345576.00000, sharding as of 1717345576.00000> to ShardRange<'k' to 't' as of 1717345577.00000, (0, 0) as of 1717345577.00000, created as of 1717345577.00000>, path: .shards_a/shard_c_1717345577.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4cb/7d09529259c864749f13ef7dc7aad4cb/7d09529259c864749f13ef7dc7aad4cb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345579.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/fbd/61ed5c8594b4e0563ab453dd4daa7fbd/61ed5c8594b4e0563ab453dd4daa7fbd.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345578.00000, (0, 0) as of 1717345578.00000, sharding as of 1717345578.00000> to ShardRange<'k' to 't' as of 1717345579.00000, (0, 0) as of 1717345579.00000, cleaved as of 1717345579.00000>, path: .shards_a/shard_c_1717345579.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/fbd/61ed5c8594b4e0563ab453dd4daa7fbd/61ed5c8594b4e0563ab453dd4daa7fbd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345581.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/004/cedc0a0208c3d3ef2592fe375be9d004/cedc0a0208c3d3ef2592fe375be9d004.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345580.00000, (0, 0) as of 1717345580.00000, sharding as of 1717345580.00000> to ShardRange<'k' to 't' as of 1717345581.00000, (0, 0) as of 1717345581.00000, active as of 1717345581.00000>, path: .shards_a/shard_c_1717345581.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/004/cedc0a0208c3d3ef2592fe375be9d004/cedc0a0208c3d3ef2592fe375be9d004.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345583.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/656/3200dccb0e1a277e19235d9d1156d656/3200dccb0e1a277e19235d9d1156d656.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345582.00000, (0, 0) as of 1717345582.00000, sharding as of 1717345582.00000> to ShardRange<'k' to 't' as of 1717345583.00000, (0, 0) as of 1717345583.00000, shrinking as of 1717345583.00000>, path: .shards_a/shard_c_1717345583.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/656/3200dccb0e1a277e19235d9d1156d656/3200dccb0e1a277e19235d9d1156d656.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345583.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/656/3200dccb0e1a277e19235d9d1156d656/3200dccb0e1a277e19235d9d1156d656.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345585.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/005/4b624ddf5562dc5485b27b72797e7005/4b624ddf5562dc5485b27b72797e7005.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345585.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/005/4b624ddf5562dc5485b27b72797e7005/4b624ddf5562dc5485b27b72797e7005.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345587.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/178/47ffa8a5b29c9eb51dee414cf5f09178/47ffa8a5b29c9eb51dee414cf5f09178.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345586.00000, (0, 0) as of 1717345586.00000, sharding as of 1717345586.00000> to ShardRange<'k' to 't' as of 1717345587.00000, (0, 0) as of 1717345587.00000, sharded as of 1717345587.00000>, path: .shards_a/shard_c_1717345587.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/178/47ffa8a5b29c9eb51dee414cf5f09178/47ffa8a5b29c9eb51dee414cf5f09178.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345587.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/178/47ffa8a5b29c9eb51dee414cf5f09178/47ffa8a5b29c9eb51dee414cf5f09178.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345589.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/358/b56d8eb0d36bbff6b0bbcf153da96358/b56d8eb0d36bbff6b0bbcf153da96358.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345588.00000, (0, 0) as of 1717345588.00000, sharding as of 1717345588.00000> to ShardRange<'k' to 't' as of 1717345589.00000, (0, 0) as of 1717345589.00000, shrunk as of 1717345589.00000>, path: .shards_a/shard_c_1717345589.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/358/b56d8eb0d36bbff6b0bbcf153da96358/b56d8eb0d36bbff6b0bbcf153da96358.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345589.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/358/b56d8eb0d36bbff6b0bbcf153da96358/b56d8eb0d36bbff6b0bbcf153da96358.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345591.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3ac/48aba7d2d0fd0839fa7ae407eee973ac/48aba7d2d0fd0839fa7ae407eee973ac.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345590.00000, (0, 0) as of 1717345590.00000, sharded as of 1717345590.00000> to ShardRange<'k' to 't' as of 1717345591.00000, (0, 0) as of 1717345591.00000, found as of 1717345591.00000>, path: .shards_a/shard_c_1717345591.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3ac/48aba7d2d0fd0839fa7ae407eee973ac/48aba7d2d0fd0839fa7ae407eee973ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345593.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/8a6/def984a1dcaf383f8b1d17a7137f28a6/def984a1dcaf383f8b1d17a7137f28a6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345592.00000, (0, 0) as of 1717345592.00000, sharded as of 1717345592.00000> to ShardRange<'k' to 't' as of 1717345593.00000, (0, 0) as of 1717345593.00000, created as of 1717345593.00000>, path: .shards_a/shard_c_1717345593.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/8a6/def984a1dcaf383f8b1d17a7137f28a6/def984a1dcaf383f8b1d17a7137f28a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345595.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/c77/b3df4f9acb61019c80d5bdd73d35dc77/b3df4f9acb61019c80d5bdd73d35dc77.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345594.00000, (0, 0) as of 1717345594.00000, sharded as of 1717345594.00000> to ShardRange<'k' to 't' as of 1717345595.00000, (0, 0) as of 1717345595.00000, cleaved as of 1717345595.00000>, path: .shards_a/shard_c_1717345595.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/c77/b3df4f9acb61019c80d5bdd73d35dc77/b3df4f9acb61019c80d5bdd73d35dc77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345597.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/50c/9a556d4ffa8304dbd2c3b0e1f66a450c/9a556d4ffa8304dbd2c3b0e1f66a450c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345596.00000, (0, 0) as of 1717345596.00000, sharded as of 1717345596.00000> to ShardRange<'k' to 't' as of 1717345597.00000, (0, 0) as of 1717345597.00000, active as of 1717345597.00000>, path: .shards_a/shard_c_1717345597.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/50c/9a556d4ffa8304dbd2c3b0e1f66a450c/9a556d4ffa8304dbd2c3b0e1f66a450c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345599.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/c9c/33bccbb948338f67857a93031992ec9c/33bccbb948338f67857a93031992ec9c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345598.00000, (0, 0) as of 1717345598.00000, sharded as of 1717345598.00000> to ShardRange<'k' to 't' as of 1717345599.00000, (0, 0) as of 1717345599.00000, shrinking as of 1717345599.00000>, path: .shards_a/shard_c_1717345599.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/c9c/33bccbb948338f67857a93031992ec9c/33bccbb948338f67857a93031992ec9c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345599.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/c9c/33bccbb948338f67857a93031992ec9c/33bccbb948338f67857a93031992ec9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345601.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/beb/f69d7c83533fd56f84a50d84fe1bbbeb/f69d7c83533fd56f84a50d84fe1bbbeb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345600.00000, (0, 0) as of 1717345600.00000, sharded as of 1717345600.00000> to ShardRange<'k' to 't' as of 1717345601.00000, (0, 0) as of 1717345601.00000, sharding as of 1717345601.00000>, path: .shards_a/shard_c_1717345601.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/beb/f69d7c83533fd56f84a50d84fe1bbbeb/f69d7c83533fd56f84a50d84fe1bbbeb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345601.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/beb/f69d7c83533fd56f84a50d84fe1bbbeb/f69d7c83533fd56f84a50d84fe1bbbeb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345603.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e88/6a2f4baf100ed5fb0c5db563cebd9e88/6a2f4baf100ed5fb0c5db563cebd9e88.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345603.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e88/6a2f4baf100ed5fb0c5db563cebd9e88/6a2f4baf100ed5fb0c5db563cebd9e88.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345605.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/cac/23b8ee027297dce03d7d795fcf050cac/23b8ee027297dce03d7d795fcf050cac.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345604.00000, (0, 0) as of 1717345604.00000, sharded as of 1717345604.00000> to ShardRange<'k' to 't' as of 1717345605.00000, (0, 0) as of 1717345605.00000, shrunk as of 1717345605.00000>, path: .shards_a/shard_c_1717345605.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/cac/23b8ee027297dce03d7d795fcf050cac/23b8ee027297dce03d7d795fcf050cac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345605.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/cac/23b8ee027297dce03d7d795fcf050cac/23b8ee027297dce03d7d795fcf050cac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345607.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d9c/2ce720e257c9ff6fa803806283357d9c/2ce720e257c9ff6fa803806283357d9c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345606.00000, (0, 0) as of 1717345606.00000, shrunk as of 1717345606.00000> to ShardRange<'k' to 't' as of 1717345607.00000, (0, 0) as of 1717345607.00000, found as of 1717345607.00000>, path: .shards_a/shard_c_1717345607.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d9c/2ce720e257c9ff6fa803806283357d9c/2ce720e257c9ff6fa803806283357d9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345609.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/23c/4590255edaddefa01e90aa4c4407323c/4590255edaddefa01e90aa4c4407323c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345608.00000, (0, 0) as of 1717345608.00000, shrunk as of 1717345608.00000> to ShardRange<'k' to 't' as of 1717345609.00000, (0, 0) as of 1717345609.00000, created as of 1717345609.00000>, path: .shards_a/shard_c_1717345609.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/23c/4590255edaddefa01e90aa4c4407323c/4590255edaddefa01e90aa4c4407323c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345611.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/75f/d14b0632966b1fdec3366c56c399f75f/d14b0632966b1fdec3366c56c399f75f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345610.00000, (0, 0) as of 1717345610.00000, shrunk as of 1717345610.00000> to ShardRange<'k' to 't' as of 1717345611.00000, (0, 0) as of 1717345611.00000, cleaved as of 1717345611.00000>, path: .shards_a/shard_c_1717345611.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/75f/d14b0632966b1fdec3366c56c399f75f/d14b0632966b1fdec3366c56c399f75f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345613.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/04a/c1d7fe9cedd404848675a1695d6cc04a/c1d7fe9cedd404848675a1695d6cc04a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345612.00000, (0, 0) as of 1717345612.00000, shrunk as of 1717345612.00000> to ShardRange<'k' to 't' as of 1717345613.00000, (0, 0) as of 1717345613.00000, active as of 1717345613.00000>, path: .shards_a/shard_c_1717345613.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/04a/c1d7fe9cedd404848675a1695d6cc04a/c1d7fe9cedd404848675a1695d6cc04a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345615.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/290/1781215481e1697a37c18d728a139290/1781215481e1697a37c18d728a139290.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345614.00000, (0, 0) as of 1717345614.00000, shrunk as of 1717345614.00000> to ShardRange<'k' to 't' as of 1717345615.00000, (0, 0) as of 1717345615.00000, shrinking as of 1717345615.00000>, path: .shards_a/shard_c_1717345615.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/290/1781215481e1697a37c18d728a139290/1781215481e1697a37c18d728a139290.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345615.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/290/1781215481e1697a37c18d728a139290/1781215481e1697a37c18d728a139290.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345617.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f4a/f3077a990489a2afcfa26ce77d9eff4a/f3077a990489a2afcfa26ce77d9eff4a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345616.00000, (0, 0) as of 1717345616.00000, shrunk as of 1717345616.00000> to ShardRange<'k' to 't' as of 1717345617.00000, (0, 0) as of 1717345617.00000, sharding as of 1717345617.00000>, path: .shards_a/shard_c_1717345617.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f4a/f3077a990489a2afcfa26ce77d9eff4a/f3077a990489a2afcfa26ce77d9eff4a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345617.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f4a/f3077a990489a2afcfa26ce77d9eff4a/f3077a990489a2afcfa26ce77d9eff4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345619.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ec1/714a958db6d882a85acfacdf8a15eec1/714a958db6d882a85acfacdf8a15eec1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345618.00000, (0, 0) as of 1717345618.00000, shrunk as of 1717345618.00000> to ShardRange<'k' to 't' as of 1717345619.00000, (0, 0) as of 1717345619.00000, sharded as of 1717345619.00000>, path: .shards_a/shard_c_1717345619.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ec1/714a958db6d882a85acfacdf8a15eec1/714a958db6d882a85acfacdf8a15eec1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345619.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ec1/714a958db6d882a85acfacdf8a15eec1/714a958db6d882a85acfacdf8a15eec1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345621.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0af/1e94497743b0c7125e211c73ac4490af/1e94497743b0c7125e211c73ac4490af.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345621.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/0af/1e94497743b0c7125e211c73ac4490af/1e94497743b0c7125e211c73ac4490af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345622.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f5a/60a516c16e6a70fcbe983a3784c94f5a/60a516c16e6a70fcbe983a3784c94f5a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345624.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/07c/f3717dba590849f36ebe3f67f63b507c/f3717dba590849f36ebe3f67f63b507c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345626.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/8ef/03f6aa6f0056b4ffc1defafc0e2e68ef/03f6aa6f0056b4ffc1defafc0e2e68ef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345628.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e70/d76578fb0bfeccbd7513a6ba90a4be70/d76578fb0bfeccbd7513a6ba90a4be70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345630.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a47/237bab5c84c56203295e9f79cf56aa47/237bab5c84c56203295e9f79cf56aa47.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345632.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6d3/1e7d027038590c3bb14fc8423caaa6d3/1e7d027038590c3bb14fc8423caaa6d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345634.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3ae/5b6d86b2c3ba42a502239149faeac3ae/5b6d86b2c3ba42a502239149faeac3ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345636.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/fc9/0e80ee33ab258f5344ad178c1010ffc9/0e80ee33ab258f5344ad178c1010ffc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345638.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/78a/579073a23dfc8eb16d990b01e395378a/579073a23dfc8eb16d990b01e395378a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345640.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ef0/62fa9abef0e296ee81f8b88f4db27ef0/62fa9abef0e296ee81f8b88f4db27ef0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345642.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/49f/5f1227d399861868209225bec328449f/5f1227d399861868209225bec328449f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345644.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ac9/c357274f69979755f5108e12be52cac9/c357274f69979755f5108e12be52cac9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345646.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/295/6664e437f5933ba04bef5048cca8c295/6664e437f5933ba04bef5048cca8c295.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345648.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d0f/328e4d39225e77365b4e0d844b41bd0f/328e4d39225e77365b4e0d844b41bd0f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345650.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3c8/63b782e7b8e54c876240962701c493c8/63b782e7b8e54c876240962701c493c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345652.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/363/e950c4734adf9da456782e584e01a363/e950c4734adf9da456782e584e01a363.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345654.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e5f/1adde1855ebd3e66cc1c7edff9604e5f/1adde1855ebd3e66cc1c7edff9604e5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345656.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/288/720fb4ea170fe37c34167c90482b9288/720fb4ea170fe37c34167c90482b9288.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345658.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4ca/b604a7e05a55caf52215d2252243b4ca/b604a7e05a55caf52215d2252243b4ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345660.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3d6/10ebb754bde56306fac0335175e063d6/10ebb754bde56306fac0335175e063d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345662.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6d8/acd14b1972607cbb69969a8578a926d8/acd14b1972607cbb69969a8578a926d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345664.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/22e/7f45411f2839db5ac789fd5ea151822e/7f45411f2839db5ac789fd5ea151822e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345666.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6df/b6c16d33e42e25f8dad4f2cfe3f0d6df/b6c16d33e42e25f8dad4f2cfe3f0d6df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345668.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/915/461b12e5c6824e922001a5be89815915/461b12e5c6824e922001a5be89815915.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345670.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7c9/334500d5030ee0998714e69c8f0607c9/334500d5030ee0998714e69c8f0607c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345672.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d75/3f5935f3d72cb7f72a4958f2f321bd75/3f5935f3d72cb7f72a4958f2f321bd75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345674.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/720/396ca3dc69c187f7f443113687a65720/396ca3dc69c187f7f443113687a65720.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345676.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/646/e5696ee97c27dfd1c4341b781d0da646/e5696ee97c27dfd1c4341b781d0da646.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345678.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e35/69618a19e0b43e00ccaa31af34f6ce35/69618a19e0b43e00ccaa31af34f6ce35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345680.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3c3/4de4915ba3b4f2d6b400f199fa6e13c3/4de4915ba3b4f2d6b400f199fa6e13c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345682.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/216/0487ffefd35f27d897fbee2d6c3a3216/0487ffefd35f27d897fbee2d6c3a3216.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345684.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/df7/05c614d5e78a9875839e47efd87fedf7/05c614d5e78a9875839e47efd87fedf7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345686.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3e8/9e213b8c7708aea89e8e061fa2eba3e8/9e213b8c7708aea89e8e061fa2eba3e8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345686.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/3e8/9e213b8c7708aea89e8e061fa2eba3e8/9e213b8c7708aea89e8e061fa2eba3e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345688.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/51b/154c231e7f08158b598cee395750951b/154c231e7f08158b598cee395750951b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345688.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/51b/154c231e7f08158b598cee395750951b/154c231e7f08158b598cee395750951b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345690.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d9f/5f6fc095f8dff98aeb6acea290e7ed9f/5f6fc095f8dff98aeb6acea290e7ed9f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345690.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d9f/5f6fc095f8dff98aeb6acea290e7ed9f/5f6fc095f8dff98aeb6acea290e7ed9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345692.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6a6/ae0858e4ae6f364c934535d9b15ce6a6/ae0858e4ae6f364c934535d9b15ce6a6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345692.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6a6/ae0858e4ae6f364c934535d9b15ce6a6/ae0858e4ae6f364c934535d9b15ce6a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345694.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7f7/e76f8978ef75bf32f0b0918f5c4847f7/e76f8978ef75bf32f0b0918f5c4847f7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345694.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/7f7/e76f8978ef75bf32f0b0918f5c4847f7/e76f8978ef75bf32f0b0918f5c4847f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345696.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/68b/d6687481d89bd8b3afe156800b61968b/d6687481d89bd8b3afe156800b61968b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345696.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/68b/d6687481d89bd8b3afe156800b61968b/d6687481d89bd8b3afe156800b61968b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345698.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/982/fec7098d81838444d77a55b635b85982/fec7098d81838444d77a55b635b85982.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345698.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/982/fec7098d81838444d77a55b635b85982/fec7098d81838444d77a55b635b85982.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345700.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a1c/55c6bfaa7234ee03f37ec9b9c8730a1c/55c6bfaa7234ee03f37ec9b9c8730a1c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345700.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a1c/55c6bfaa7234ee03f37ec9b9c8730a1c/55c6bfaa7234ee03f37ec9b9c8730a1c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345702.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/fff/e083d3c314f11d72439662ea72b66fff/e083d3c314f11d72439662ea72b66fff.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345702.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/fff/e083d3c314f11d72439662ea72b66fff/e083d3c314f11d72439662ea72b66fff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345704.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e84/020d00803203cf0a939f765ab580ee84/020d00803203cf0a939f765ab580ee84.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345704.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/e84/020d00803203cf0a939f765ab580ee84/020d00803203cf0a939f765ab580ee84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345706.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ae3/f7fdd88232a529cc5dd37d99258c3ae3/f7fdd88232a529cc5dd37d99258c3ae3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345706.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ae3/f7fdd88232a529cc5dd37d99258c3ae3/f7fdd88232a529cc5dd37d99258c3ae3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345708.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ddf/e0195430a4e55d810d03fa7918c5bddf/e0195430a4e55d810d03fa7918c5bddf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345708.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ddf/e0195430a4e55d810d03fa7918c5bddf/e0195430a4e55d810d03fa7918c5bddf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345710.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/45f/dd4569e761401ce717d07f430dfb745f/dd4569e761401ce717d07f430dfb745f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345710.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/45f/dd4569e761401ce717d07f430dfb745f/dd4569e761401ce717d07f430dfb745f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345712.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/8ea/66bdabf0462941bf4ed94e3c0c8148ea/66bdabf0462941bf4ed94e3c0c8148ea.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345712.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/8ea/66bdabf0462941bf4ed94e3c0c8148ea/66bdabf0462941bf4ed94e3c0c8148ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345714.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/412/22be60f8da401b6f4e3d8a2a65e0e412/22be60f8da401b6f4e3d8a2a65e0e412.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345714.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/412/22be60f8da401b6f4e3d8a2a65e0e412/22be60f8da401b6f4e3d8a2a65e0e412.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345716.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f66/6d6f93129bfba4116edbafe7ae619f66/6d6f93129bfba4116edbafe7ae619f66.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345716.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/f66/6d6f93129bfba4116edbafe7ae619f66/6d6f93129bfba4116edbafe7ae619f66.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345718.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/097/3b553351ccb787af8cfe7e78ea64c097/3b553351ccb787af8cfe7e78ea64c097.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345718.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/097/3b553351ccb787af8cfe7e78ea64c097/3b553351ccb787af8cfe7e78ea64c097.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345720.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/94d/f762bdaf8d329a2a73901ad5467ef94d/f762bdaf8d329a2a73901ad5467ef94d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345720.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/94d/f762bdaf8d329a2a73901ad5467ef94d/f762bdaf8d329a2a73901ad5467ef94d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345722.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/721/ac84b69e9b7871c806546bd64ae18721/ac84b69e9b7871c806546bd64ae18721.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345722.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/721/ac84b69e9b7871c806546bd64ae18721/ac84b69e9b7871c806546bd64ae18721.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345724.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/121/1f0675f6ee137167a58f2c948dc4f121/1f0675f6ee137167a58f2c948dc4f121.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345724.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/121/1f0675f6ee137167a58f2c948dc4f121/1f0675f6ee137167a58f2c948dc4f121.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345726.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ba7/655308f091b5b69479c84c37dda0fba7/655308f091b5b69479c84c37dda0fba7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345726.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/ba7/655308f091b5b69479c84c37dda0fba7/655308f091b5b69479c84c37dda0fba7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345728.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/87c/b20ae98d47fc327b584b78ad1e38887c/b20ae98d47fc327b584b78ad1e38887c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345728.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/87c/b20ae98d47fc327b584b78ad1e38887c/b20ae98d47fc327b584b78ad1e38887c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345730.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4d0/215b7ee52deb64a5f715d63e18dda4d0/215b7ee52deb64a5f715d63e18dda4d0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345730.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4d0/215b7ee52deb64a5f715d63e18dda4d0/215b7ee52deb64a5f715d63e18dda4d0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345732.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/62b/1e2115d139dd1182ba8ee72d3c7c862b/1e2115d139dd1182ba8ee72d3c7c862b.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345510.53933', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T162510Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345510.55998', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T162512Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345512.93468', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T162512Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345512.95039', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission ... ok Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345515.98092', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345515.99740', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345520.23910', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345520.25220', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:19 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion ... ok test.unit.common.test_constraints.TestConstraints.test_check_account_format test.unit.common.test_constraints.TestConstraints.test_check_account_format ... ok test.unit.common.test_constraints.TestConstraints.test_check_container_format test.unit.common.test_constraints.TestConstraints.test_check_container_format ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount ... ok test.unit.common.test_constraints.TestConstraints.test_check_float test.unit.common.test_constraints.TestConstraints.test_check_float ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_count test.unit.common.test_constraints.TestConstraints.test_check_metadata_count ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_good test.unit.common.test_constraints.TestConstraints.test_check_metadata_good ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_size test.unit.common.test_constraints.TestConstraints.test_check_metadata_size ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8 test.unit.common.test_constraints.TestConstraints.test_check_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical ... ok test.unit.common.test_constraints.TestConstraints.test_valid_api_version test.unit.common.test_constraints.TestConstraints.test_valid_api_version ... ok test.unit.common.test_constraints.TestConstraints.test_valid_timestamp test.unit.common.test_constraints.TestConstraints.test_valid_timestamp ... ok test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta ... ok test.unit.common.test_constraints.TestConstraints.test_validate_constraints test.unit.common.test_constraints.TestConstraints.test_validate_constraints ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get test.unit.common.test_internal_client.TestSimpleClient.test_get ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param ... ok test.unit.common.test_internal_client.TestSimpleClient.test_head test.unit.common.test_internal_client.TestSimpleClient.test_head ... ok test.unit.common.test_internal_client.TestSimpleClient.test_proxy test.unit.common.test_internal_client.TestSimpleClient.test_proxy ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError ... ok test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.9:1009/sdj, error: ERROR Insufficient Storage (txn: tx8e768733c6be4478b9efa-00665c9cef) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test-ic DEBUG: -> 2014-05-27T20:54:11 GET http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - test-ic DEBUG: -> 2014-05-27T20:54:11 HEAD http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10036826133728027 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10183262825012207 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10212826728820801 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10238885879516602 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10262656211853027 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.1028745174407959 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.1031196117401123 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.1033623218536377 test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0002577304840088 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0008225440979004 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0010931491851807 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.001338243484497 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.00162672996521 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.001866102218628 test.unit.common.test_memcached.TestMemcached.test_connection_pooling test.unit.common.test_memcached.TestMemcached.test_connection_pooling ... ok test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect ... ok test.unit.common.test_memcached.TestMemcached.test_decr test.unit.common.test_memcached.TestMemcached.test_decr ... ok test.unit.common.test_memcached.TestMemcached.test_delete test.unit.common.test_memcached.TestMemcached.test_delete ... ok test.unit.common.test_memcached.TestMemcached.test_error_disabled test.unit.common.test_memcached.TestMemcached.test_error_disabled ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission ... ok test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0021450519561768 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0023770332336426 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout connecting to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10327339172363281 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 3.886222839355469e-05: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.4:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test.unit.common.test_memcached.TestMemcached.test_error_limiting test.unit.common.test_memcached.TestMemcached.test_error_limiting ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server ERROR: Object returning 503 for [] (txn: tx026005a5c5464542bec94-00665c9cf2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.8:1008/sdi (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.13:1013/sdn (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.12:1012/sdm (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.5:1005/sdf (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx21aa4399c51741608a989-00665c9cf2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 Trying to GET /a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx7cb25a4241264f1db54a2-00665c9cf2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx756c2db01ea849d4bd0bd-00665c9cf2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Ignoring 412 from primary (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission ... ok test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method get: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config ... ok test.unit.common.test_memcached.TestMemcached.test_error_raising test.unit.common.test_memcached.TestMemcached.test_error_raising ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns test.unit.common.test_memcached.TestMemcached.test_get_conns ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname ... ok test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr test.unit.common.test_memcached.TestMemcached.test_incr ... ok test.unit.common.test_memcached.TestMemcached.test_incr_add_expires test.unit.common.test_memcached.TestMemcached.test_incr_add_expires ... ok test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response ... ok test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold ... ok test.unit.common.test_memcached.TestMemcached.test_logger_kwarg test.unit.common.test_memcached.TestMemcached.test_logger_kwarg ... ok test.unit.common.test_memcached.TestMemcached.test_multi test.unit.common.test_memcached.TestMemcached.test_multi ... ok test.unit.common.test_memcached.TestMemcached.test_multi_delete test.unit.common.test_memcached.TestMemcached.test_multi_delete ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_retry test.unit.common.test_memcached.TestMemcached.test_retry ... ok test.unit.common.test_memcached.TestMemcached.test_serialization test.unit.common.test_memcached.TestMemcached.test_serialization ... ok test.unit.common.test_memcached.TestMemcached.test_set_error test.unit.common.test_memcached.TestMemcached.test_set_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_get_json test.unit.common.test_memcached.TestMemcached.test_set_get_json ... ok test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg ... ok test.unit.common.test_utils.TestParseContentRange.test_bad test.unit.common.test_utils.TestParseContentRange.test_bad ... ok test.unit.common.test_utils.TestParseContentRange.test_good test.unit.common.test_utils.TestParseContentRange.test_good ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding ... ok test.unit.container.test_server.TestContainerController.test_DELETE test.unit.container.test_server.TestContainerController.test_DELETE ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error ... ok test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 3.6716461181640625e-05: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0, expired ttl=1.23 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 493, in _incr_or_decr return int(line[0].strip()) ^^^^^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: b'UNEXPECTED' test WARNING: Item size larger than warning threshold: 80 (80) >= 80 (80) test WARNING: Item size larger than warning threshold: 81 (81) >= 80 (80) test WARNING: Item size larger than warning threshold: 99 (99) >= 80 (80) test WARNING: Item size larger than warning threshold: 100 (100) >= 80 (80) test WARNING: Item size larger than warning threshold: 0 (0) >= 0 (0) test WARNING: Item size larger than warning threshold: 1 (1) >= 0 (0) test WARNING: Item size larger than warning threshold: 50 (50) >= 0 (0) test WARNING: Item size larger than warning threshold: 99 (99) >= 0 (0) test WARNING: Item size larger than warning threshold: 100 (100) >= 0 (0) test WARNING: Item size larger than warning threshold: 2048576 (2Mi) >= 1000000 (977Ki) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix multi_key, method get_multi, time_spent 4.124641418457031e-05, incomplete read test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0, failed to connect test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 460, in get sock.sendall(b'get ' + cmd.hash_key + b'\r\n') File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1282, in handle_get raise Exception('get failed') Exception: get failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method get, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix incr_key, method incr, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 531, in incr new_val = self._add(fp, sock, cmd, add_val, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 496, in _add sock.sendall(b' '.join([ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1224, in handle_add raise Exception('add failed') Exception: add failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix nvratelimit/v2/wf, method incr, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix set_key, method set, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 427, in set sock.sendall(set_msg(cmd.hash_key, flags, timeout, value)) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1252, in handle_set raise Exception('set failed') Exception: set failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0082 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0120 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0076 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0060 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race ... ok test.unit.container.test_server.TestContainerController.test_DELETE_account_update test.unit.container.test_server.TestContainerController.test_DELETE_account_update ... ok test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_DELETE_not_found test.unit.container.test_server.TestContainerController.test_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_object test.unit.container.test_server.TestContainerController.test_DELETE_object ... ok sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345732.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/62b/1e2115d139dd1182ba8ee72d3c7c862b/1e2115d139dd1182ba8ee72d3c7c862b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345734.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4e3/ff852ffc38d391db2ac1e66c9e9524e3/ff852ffc38d391db2ac1e66c9e9524e3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345734.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/4e3/ff852ffc38d391db2ac1e66c9e9524e3/ff852ffc38d391db2ac1e66c9e9524e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345736.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d82/8025f65c0a06a98af8a7c26eba81cd82/8025f65c0a06a98af8a7c26eba81cd82.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345736.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/d82/8025f65c0a06a98af8a7c26eba81cd82/8025f65c0a06a98af8a7c26eba81cd82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345738.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/28e/34299b99230dd730d07d3ec3c4ecd28e/34299b99230dd730d07d3ec3c4ecd28e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345738.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/28e/34299b99230dd730d07d3ec3c4ecd28e/34299b99230dd730d07d3ec3c4ecd28e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345740.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/aee/1e6fa51192e4a6c766ff867cadd29aee/1e6fa51192e4a6c766ff867cadd29aee.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345740.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/aee/1e6fa51192e4a6c766ff867cadd29aee/1e6fa51192e4a6c766ff867cadd29aee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345742.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6f0/78477a336228b5fce8e2b74e215c16f0/78477a336228b5fce8e2b74e215c16f0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345742.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6f0/78477a336228b5fce8e2b74e215c16f0/78477a336228b5fce8e2b74e215c16f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345744.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6dc/4a7adc2312b7f54994937a02099ab6dc/4a7adc2312b7f54994937a02099ab6dc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345744.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/6dc/4a7adc2312b7f54994937a02099ab6dc/4a7adc2312b7f54994937a02099ab6dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345746.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a62/6385097bbaad4b3a2834ba7fdf58ca62/6385097bbaad4b3a2834ba7fdf58ca62.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345746.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/a62/6385097bbaad4b3a2834ba7fdf58ca62/6385097bbaad4b3a2834ba7fdf58ca62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345748.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/bfa/655ae98eb62c58148cc70b26a9e89bfa/655ae98eb62c58148cc70b26a9e89bfa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345748.00000, db: /tmp/tmp_pjbhjwi/sda/containers/0/bfa/655ae98eb62c58148cc70b26a9e89bfa/655ae98eb62c58148cc70b26a9e89bfa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345515.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d03/1fa299199c0a1db9809315924fa20d03/1fa299199c0a1db9809315924fa20d03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345520.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/398/f90c4a0e03752dfd539c6ea61cd5f398/f90c4a0e03752dfd539c6ea61cd5f398.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345525.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/55e/c5f889eef5d345e023376867f838e55e/c5f889eef5d345e023376867f838e55e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345530.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/06b/54a8feedca7fb2b0cd1c3ac81c64206b/54a8feedca7fb2b0cd1c3ac81c64206b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345535.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e5b/bb6c61d8326c3d1a30652f069fd6ae5b/bb6c61d8326c3d1a30652f069fd6ae5b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345540.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/291/94799bcf1741d648fd9987594d3a5291/94799bcf1741d648fd9987594d3a5291.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345545.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/5b1/1e70c031d530942eb3655b1a805105b1/1e70c031d530942eb3655b1a805105b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345550.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2ac/9c10e5d4fc6b0b9340534cf55ba872ac/9c10e5d4fc6b0b9340534cf55ba872ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345555.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1c0/701aacdac53989b0888114bc9fbbd1c0/701aacdac53989b0888114bc9fbbd1c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345560.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2a5/898fdceb97c8f1930c48a2e5713eb2a5/898fdceb97c8f1930c48a2e5713eb2a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345565.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d0e/ea72c188c4224ac849a2d7fe169bad0e/ea72c188c4224ac849a2d7fe169bad0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345570.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/23d/78b1fe2c5d31f09aa58f4cb40ccc923d/78b1fe2c5d31f09aa58f4cb40ccc923d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345575.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/8b0/f8ae096f335e76e81d948650f16eb8b0/f8ae096f335e76e81d948650f16eb8b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345585.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/6ab/d0509d170b0f1e444db30456185ae6ab/d0509d170b0f1e444db30456185ae6ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345590.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/665/d1b77861918be119806951a3d5f2c665/d1b77861918be119806951a3d5f2c665.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345600.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/0ca/032a2dc0bad7bde9e369fb12d5ab80ca/032a2dc0bad7bde9e369fb12d5ab80ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345605.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/9a2/21d48763d166823a83f7bccfddde79a2/21d48763d166823a83f7bccfddde79a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345610.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/a5f/e1cf616facba3f6f034939a512cc0a5f/e1cf616facba3f6f034939a512cc0a5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345615.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f1e/d2df3b46c47d0a6c39116672a5fe8f1e/d2df3b46c47d0a6c39116672a5fe8f1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345620.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/32d/89aa309dc2bc18859f87bfdf186f232d/89aa309dc2bc18859f87bfdf186f232d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345625.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/371/6106ef7c90a2c4416c0b57ddfa173371/6106ef7c90a2c4416c0b57ddfa173371.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345630.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d03/7da7b78b1e899319b94a5348029fad03/7da7b78b1e899319b94a5348029fad03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345640.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/ea7/0e27dbff000bf481fef60d7129b98ea7/0e27dbff000bf481fef60d7129b98ea7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345645.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/606/8b58f7179269eb3df31b718be5406606/8b58f7179269eb3df31b718be5406606.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345650.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d8a/93ac8b776635264cfbf2799e40604d8a/93ac8b776635264cfbf2799e40604d8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345660.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/925/0f78f61f39dc33ae21592eee86828925/0f78f61f39dc33ae21592eee86828925.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345665.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d35/4416bf20dce847d9330869aa30b6cd35/4416bf20dce847d9330869aa30b6cd35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345670.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/02e/f71202e3621e56629d00f8162fcf502e/f71202e3621e56629d00f8162fcf502e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345675.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b39/ade3a336b3114f0025b8f6ca4fa05b39/ade3a336b3114f0025b8f6ca4fa05b39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345680.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/fd4/e112f233af5e69d5723623c9c8973fd4/e112f233af5e69d5723623c9c8973fd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345685.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e2c/beaa7d16cc9ea04b699d6b4dd546be2c/beaa7d16cc9ea04b699d6b4dd546be2c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345690.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/ce1/a2a8ce446089cf34cc8ac42e0e0d4ce1/a2a8ce446089cf34cc8ac42e0e0d4ce1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345700.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/852/d52130748b05ceaf9bd055b98b90a852/d52130748b05ceaf9bd055b98b90a852.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345705.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/99c/deb81236e83353b7dab2e4f0007f499c/deb81236e83353b7dab2e4f0007f499c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345710.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b17/c79d9c4aa512a13a70989135029bbb17/c79d9c4aa512a13a70989135029bbb17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345720.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/eb9/af8080cc8b1fbd081f0184cc64fbaeb9/af8080cc8b1fbd081f0184cc64fbaeb9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345725.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3de/ea4c6ecdba57bedb47dac12c90c5f3de/ea4c6ecdba57bedb47dac12c90c5f3de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345730.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/933/2eded075ed2a94e25c7b5d81d0620933/2eded075ed2a94e25c7b5d81d0620933.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345735.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/85c/6cc1446b3d3747a132dfe6e37382285c/6cc1446b3d3747a132dfe6e37382285c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345740.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/063/2b96708c5e6d033025fbc861d30ee063/2b96708c5e6d033025fbc861d30ee063.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345745.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/caa/22d7c901597da9f00a2561b81e917caa/22d7c901597da9f00a2561b81e917caa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345750.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/879/384dabcc88c9ce2af0efbcd298cb8879/384dabcc88c9ce2af0efbcd298cb8879.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345755.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/db9/d57ac6f55bb096b7b53bce53eb985db9/d57ac6f55bb096b7b53bce53eb985db9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345760.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/173/dc3d779a86fd7a534a8c00cc0c79d173/dc3d779a86fd7a534a8c00cc0c79d173.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345765.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/36e/a1a6f74da88400a38f6908e346dba36e/a1a6f74da88400a38f6908e346dba36e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345770.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/bdd/bb6b72defdb4220435e4ae4055d15bdd/bb6b72defdb4220435e4ae4055d15bdd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345775.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/a51/0357613d8a0b68cbecaaf47a2ea38a51/0357613d8a0b68cbecaaf47a2ea38a51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345780.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/802/bd9676157ee0f6b3ff02b6192e755802/bd9676157ee0f6b3ff02b6192e755802.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345785.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/529/15c7cc78710e0cb83b1ef0701a95f529/15c7cc78710e0cb83b1ef0701a95f529.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345790.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/bbb/119a1a135c2d3cdc3ba8e42653f69bbb/119a1a135c2d3cdc3ba8e42653f69bbb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345795.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/09a/a7d77580e4fd4195087224c157e0a09a/a7d77580e4fd4195087224c157e0a09a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345800.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/387/0b0356e2e43677d5f2e9cfa873bc4387/0b0356e2e43677d5f2e9cfa873bc4387.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345805.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/9a6/573755a8e688ab69a45f7d0efdc3a9a6/573755a8e688ab69a45f7d0efdc3a9a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345810.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d4b/148e5738fbcec2d0097805418abc4d4b/148e5738fbcec2d0097805418abc4d4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345815.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b34/65a2ad93f8f5b2927b3701941925bb34/65a2ad93f8f5b2927b3701941925bb34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345820.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/208/b31bc16f2543d08f4a0670f5937a0208/b31bc16f2543d08f4a0670f5937a0208.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345825.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c2b/c5a98a82cdc33bd2aa5b48e63bddcc2b/c5a98a82cdc33bd2aa5b48e63bddcc2b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345830.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/070/35935edbc3f550ed5c6d474f455e8070/35935edbc3f550ed5c6d474f455e8070.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345835.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/025/600568b26e686e91b350099bbf036025/600568b26e686e91b350099bbf036025.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345840.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b3c/e225b37299a260a8f9b971c38a19eb3c/e225b37299a260a8f9b971c38a19eb3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345845.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/6e4/b5e1168c72bc12cfb8a8bf527e7ad6e4/b5e1168c72bc12cfb8a8bf527e7ad6e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345850.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d1e/db49e2e99f7ef32b2ebe14c697d32d1e/db49e2e99f7ef32b2ebe14c697d32d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345855.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/0b3/7da809de52d6c0590cd8f1957c7d40b3/7da809de52d6c0590cd8f1957c7d40b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345860.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/88b/ad0749a6f0ec31d14698e3cf6c59788b/ad0749a6f0ec31d14698e3cf6c59788b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345865.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/4b4/46f19aaff56ae4ac57fce63d6eed04b4/46f19aaff56ae4ac57fce63d6eed04b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345870.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/187/72e939c66eafad040162994b596eb187/72e939c66eafad040162994b596eb187.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345875.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345880.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/15e/61aaf46c17b1e494fbb883f2cb48215e/61aaf46c17b1e494fbb883f2cb48215e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345885.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/32d/8d4130bf153f46c4c156851c7528532d/8d4130bf153f46c4c156851c7528532d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345890.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/55c/39f6658130a353f14e8a6c37ad95c55c/39f6658130a353f14e8a6c37ad95c55c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345895.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345900.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/80d/1f907861c089d71a94b0d728e278180d/1f907861c089d71a94b0d728e278180d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345905.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/0a3/059f7cd8d224c5897d1e7acf09ada0a3/059f7cd8d224c5897d1e7acf09ada0a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345915.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d41/740cb5d220733796133a1347f1910d41/740cb5d220733796133a1347f1910d41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345920.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/deb/12ecad9a67c2a93914ed09a180d18deb/12ecad9a67c2a93914ed09a180d18deb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345930.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/dcb/9ce78ad7519f77b225ce82fec74badcb/9ce78ad7519f77b225ce82fec74badcb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345940.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/622/d60128a68626c160d5792a9a75f6f622/d60128a68626c160d5792a9a75f6f622.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345945.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/bb3/0e13ba1894fd7686880ce2e061079bb3/0e13ba1894fd7686880ce2e061079bb3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345950.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b69/b7866328e9ff41c34dfaa43b12317b69/b7866328e9ff41c34dfaa43b12317b69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345960.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/fe0/5061282030530cfe9d99503dbc39afe0/5061282030530cfe9d99503dbc39afe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345965.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/32c/67c64dda2b60a66458c8b842222eb32c/67c64dda2b60a66458c8b842222eb32c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345970.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345975.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1c6/06727a10627bb589627c0835996451c6/06727a10627bb589627c0835996451c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345980.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/ed5/2df33c1469286e6799495fb91d7a9ed5/2df33c1469286e6799495fb91d7a9ed5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345990.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/fe6/fb2e86a1afea266cd93a4b1c0dc36fe6/fb2e86a1afea266cd93a4b1c0dc36fe6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345995.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346000.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/6fe/497c628f729b1bbb60870990f0bad6fe/497c628f729b1bbb60870990f0bad6fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346005.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/592/c0d44b0dbbc428d2de62edfbe32ed592/c0d44b0dbbc428d2de62edfbe32ed592.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346010.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/fc6/1398ecba34e0d89c184c83f06fa4cfc6/1398ecba34e0d89c184c83f06fa4cfc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346020.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d62/b36c92577a99e4d7a5d8a17beab60d62/b36c92577a99e4d7a5d8a17beab60d62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346025.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b4a/e62d484238b79671e549c4f98df15b4a/e62d484238b79671e549c4f98df15b4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346030.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346035.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/037/21fe1f4898825a6a856e1f376b464037/21fe1f4898825a6a856e1f376b464037.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346040.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/254/58275ab922b2d9c974fed445488ff254/58275ab922b2d9c974fed445488ff254.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346045.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/094/970bcee06085d9e70038e2bd329fd094/970bcee06085d9e70038e2bd329fd094.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346050.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/808/ee2bfada3eb7a1f85192f40d7761d808/ee2bfada3eb7a1f85192f40d7761d808.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346055.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/76b/8517e9828eb7eca20964951da6d8176b/8517e9828eb7eca20964951da6d8176b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346060.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/505/7886bb9d108c06b6395eca78f6ea1505/7886bb9d108c06b6395eca78f6ea1505.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346065.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/23f/f58cebc8e9f80e0c3d59f7a22a4af23f/f58cebc8e9f80e0c3d59f7a22a4af23f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346070.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/964/ad49306915196ae164a2ce9596ae5964/ad49306915196ae164a2ce9596ae5964.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346075.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/256/8c28cad3e3414d4c0358c374c3b65256/8c28cad3e3414d4c0358c374c3b65256.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346080.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e60/36024d247878f3cdc40e175bd0cfee60/36024d247878f3cdc40e175bd0cfee60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346085.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/5bd/27af413e145c29aaaedf36d3109815bd/27af413e145c29aaaedf36d3109815bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346090.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2d6/2babc59ee60f61da2b53859777c9d2d6/2babc59ee60f61da2b53859777c9d2d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346095.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/391/8e831b661fc3b45c8576186e7931e391/8e831b661fc3b45c8576186e7931e391.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346100.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/180/5bfa3d6c61aa73e1514230b344845180/5bfa3d6c61aa73e1514230b344845180.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346105.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/cb4/2cd53168c6425d20a26c0be9ff29ecb4/2cd53168c6425d20a26c0be9ff29ecb4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346110.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/52c/2f8e06f657ff80348fb8b63ce18b952c/2f8e06f657ff80348fb8b63ce18b952c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346115.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/9d2/a40147a7d26491322ccdad84eee269d2/a40147a7d26491322ccdad84eee269d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346120.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c8d/f381380ef6ec974b467e8ad8527abc8d/f381380ef6ec974b467e8ad8527abc8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346125.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/53e/fb7dc45d745f775d136697656f41f53e/fb7dc45d745f775d136697656f41f53e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346130.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/94f/18c0253baf08d423e0ff1fe1d7b7b94f/18c0253baf08d423e0ff1fe1d7b7b94f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346135.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/46a/563bf149d2efaedf49bff236b486a46a/563bf149d2efaedf49bff236b486a46a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346140.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/05c/c5b8c00ea9842c74b6ef881f1075e05c/c5b8c00ea9842c74b6ef881f1075e05c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346145.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/34a/ecd6e608e127dc77236c344f308c634a/ecd6e608e127dc77236c344f308c634a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346150.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/966/f38081baf352eef21faa24dc3430d966/f38081baf352eef21faa24dc3430d966.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346155.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/dc9/1eed52dc5f057b26476cc823ddeb4dc9/1eed52dc5f057b26476cc823ddeb4dc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346160.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/687/f4615af5ca9d77add348feef3f721687/f4615af5ca9d77add348feef3f721687.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346165.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/543/f47122c735da4e232b927946e476c543/f47122c735da4e232b927946e476c543.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346170.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1b4/3d34f444098dfb7fc5ba03a75e7b11b4/3d34f444098dfb7fc5ba03a75e7b11b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346175.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e3d/14d062c63df20c011263f7150f018e3d/14d062c63df20c011263f7150f018e3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346180.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/35d/8894f66c15fcce2f404f6df6f63ee35d/8894f66c15fcce2f404f6df6f63ee35d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346185.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/220/6503be00ce37609b706a5e4243fee220/6503be00ce37609b706a5e4243fee220.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346190.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/664/7bb972b7aa8f6b89ebe77bce1da1d664/7bb972b7aa8f6b89ebe77bce1da1d664.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346195.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/42e/c05cabe948b3cb0061f0e3d393d9e42e/c05cabe948b3cb0061f0e3d393d9e42e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346200.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/146/73afdd3dccc176083a5c7411c5311146/73afdd3dccc176083a5c7411c5311146.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346205.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e34/772b5d3c171d51616980e1bc52381e34/772b5d3c171d51616980e1bc52381e34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346210.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/702/47002958df5e7c1c0bc3887477304702/47002958df5e7c1c0bc3887477304702.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346215.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/5c1/233d754a88059a5c8eb5dcc0896045c1/233d754a88059a5c8eb5dcc0896045c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346220.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/916/44855d3e38960b54330ac0a63b0ca916/44855d3e38960b54330ac0a63b0ca916.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346225.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f44/d3622d14259c48b2c1c314c318624f44/d3622d14259c48b2c1c314c318624f44.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346230.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/94e/5e9ba0d5fb502ea4f09b37cd076b594e/5e9ba0d5fb502ea4f09b37cd076b594e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346235.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/dc8/1f37e5bc1ea5989578422387201c4dc8/1f37e5bc1ea5989578422387201c4dc8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346240.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e5c/016bca03e5fa2e456747f288d0897e5c/016bca03e5fa2e456747f288d0897e5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346245.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/fcb/ccfa6d20bfe520b2c51599ea05672fcb/ccfa6d20bfe520b2c51599ea05672fcb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346250.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/20c/91b038233a330dcf4f1522200f3db20c/91b038233a330dcf4f1522200f3db20c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346255.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/52c/d1d1a811935c2533bd66bb167fbf852c/d1d1a811935c2533bd66bb167fbf852c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346260.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/cc8/c12b1f4482e5d78ecd121194f0017cc8/c12b1f4482e5d78ecd121194f0017cc8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346265.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/551/6cfb38664f5d0a463fbdcfb4f1dff551/6cfb38664f5d0a463fbdcfb4f1dff551.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346270.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f51/b7c39c43afdf43f85956fe0fffe15f51/b7c39c43afdf43f85956fe0fffe15f51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346275.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c33/323fbd3d1cc2d27bef21530ab3ab2c33/323fbd3d1cc2d27bef21530ab3ab2c33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346280.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/6da/88fe3e9a3f73de8fc0f40dddb57836da/88fe3e9a3f73de8fc0f40dddb57836da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346285.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/cf5/3deaaf7d7e5802b0cfa19dcd87bfdcf5/3deaaf7d7e5802b0cfa19dcd87bfdcf5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346290.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/8fd/1a78061b3d4578dc9775fc95cb3e78fd/1a78061b3d4578dc9775fc95cb3e78fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346295.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1c6/fe4ed434546ebda9d2613687075b01c6/fe4ed434546ebda9d2613687075b01c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346300.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1a3/49ac5fa9cb4c5b37fc61623ae25ab1a3/49ac5fa9cb4c5b37fc61623ae25ab1a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346305.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/8c1/c9a3a7e204d5dc4d373bee2e511b48c1/c9a3a7e204d5dc4d373bee2e511b48c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346310.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/665/c337fa4b276e71ddf99d5aae7675a665/c337fa4b276e71ddf99d5aae7675a665.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346315.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/e07/1e4ba19e7e558551eb7a80da03062e07/1e4ba19e7e558551eb7a80da03062e07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346320.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/04e/5582725c1eb7492d90750837787fe04e/5582725c1eb7492d90750837787fe04e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346325.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f45/514358fe16514124eefe2c07d3e97f45/514358fe16514124eefe2c07d3e97f45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346330.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/432/67bbd46eb70c3baacddf990dff41c432/67bbd46eb70c3baacddf990dff41c432.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346335.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/865/157e42795d4a10e4f65bdff41236d865/157e42795d4a10e4f65bdff41236d865.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346340.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f50/42fda0278d5ff801e5e0150cd82c5f50/42fda0278d5ff801e5e0150cd82c5f50.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346345.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f8a/4232bd96abf6e77ca5cc426f6ba9df8a/4232bd96abf6e77ca5cc426f6ba9df8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346350.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3e8/fb00ad2a5c768c6f36dbbf64a94e03e8/fb00ad2a5c768c6f36dbbf64a94e03e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346355.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/9e8/5dda8766c0ca9c325477f4778e54f9e8/5dda8766c0ca9c325477f4778e54f9e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346360.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/580/b21ef98e8e16a69bf3b8cf41025a7580/b21ef98e8e16a69bf3b8cf41025a7580.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346365.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c71/29a89e6682c53871d64c5f763ce73c71/29a89e6682c53871d64c5f763ce73c71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346370.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/28c/7455cccd7e89b7b673f6e3a86f2b628c/7455cccd7e89b7b673f6e3a86f2b628c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346375.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/47b/358d449cef1f9109e904d6108e20347b/358d449cef1f9109e904d6108e20347b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346380.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/15d/1683069baac03b63b09e40a9bc48a15d/1683069baac03b63b09e40a9bc48a15d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346385.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/da1/4489492c5d80fda73a91884e9dcbfda1/4489492c5d80fda73a91884e9dcbfda1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346390.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1be/a02b8014a5c6e0407678fa2d72fce1be/a02b8014a5c6e0407678fa2d72fce1be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346395.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/93b/d81280f4ede41ff87ab17c3ae998c93b/d81280f4ede41ff87ab17c3ae998c93b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346400.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2f9/8f7ff9ff6c9c0d2ee9dfdc1545d932f9/8f7ff9ff6c9c0d2ee9dfdc1545d932f9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346405.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/9a3/7544c8fe6e9f44a5b27db1f9fb89a9a3/7544c8fe6e9f44a5b27db1f9fb89a9a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346410.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/725/7d5e4bdec53acfc1e665b16d21021725/7d5e4bdec53acfc1e665b16d21021725.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346415.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/6ae/d89cfd114a388c987fffab33805496ae/d89cfd114a388c987fffab33805496ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346420.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/68d/00c80d5e5d6a641246b72417597c768d/00c80d5e5d6a641246b72417597c768d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346425.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c73/0048121de793b9f3365fa6f15d57bc73/0048121de793b9f3365fa6f15d57bc73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346430.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2db/c4439c3e2ffb056be4a15d7cf0ce32db/c4439c3e2ffb056be4a15d7cf0ce32db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346435.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/46f/0c916984f76f712d05b93474413ba46f/0c916984f76f712d05b93474413ba46f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346440.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/6f6/8f6dd9ca1f4230ff43c5a8a4a1a136f6/8f6dd9ca1f4230ff43c5a8a4a1a136f6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346445.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/465/688085c7c8899734500b4c6cd76da465/688085c7c8899734500b4c6cd76da465.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346450.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d84/bafc3160e0ffed5e0345af59c1793d84/bafc3160e0ffed5e0345af59c1793d84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346455.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c25/4e0cca6814b6fd743a744dc3380ffc25/4e0cca6814b6fd743a744dc3380ffc25.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346460.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f69/5568cb1664f64d122aa1f2e6dfeeaf69/5568cb1664f64d122aa1f2e6dfeeaf69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346465.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3dc/6a402d09355f378ddf28a3f6388313dc/6a402d09355f378ddf28a3f6388313dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346470.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/12b/5ea7950fa2fbd7a3207ed561c917e12b/5ea7950fa2fbd7a3207ed561c917e12b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346475.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/65a/ff77509392a6956e0133fdd90eb9165a/ff77509392a6956e0133fdd90eb9165a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346480.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/a4e/ed3600dd2bf5c46c529d8a39f84fea4e/ed3600dd2bf5c46c529d8a39f84fea4e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346485.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/aa4/aeebb0dd40dabe526bba05de0f869aa4/aeebb0dd40dabe526bba05de0f869aa4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346490.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/513/27a3274369c029208ae2457d49531513/27a3274369c029208ae2457d49531513.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346495.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b7f/453eedd915af4bb9d7c03b13b6c71b7f/453eedd915af4bb9d7c03b13b6c71b7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346500.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/1e7/0f4ca82ea1e9767a93ec90a9ba0f21e7/0f4ca82ea1e9767a93ec90a9ba0f21e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346505.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/fd1/d627b35fa9caa8df8c308607fb3ecfd1/d627b35fa9caa8df8c308607fb3ecfd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346510.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/5b0/5ffa87dcfa8ed2912af8c36d7a10b5b0/5ffa87dcfa8ed2912af8c36d7a10b5b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346515.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/576/809fa52f00a81e47e9744cd859f27576/809fa52f00a81e47e9744cd859f27576.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346520.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d06/55dcfd8b9022587a10769e2a201aed06/55dcfd8b9022587a10769e2a201aed06.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346525.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/12f/def8a81a12b291c702fd08a9e8a0512f/def8a81a12b291c702fd08a9e8a0512f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346530.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/b2f/c46051f25d1a63018b5a4a1bc6457b2f/c46051f25d1a63018b5a4a1bc6457b2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346535.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/bca/fb0325615263ee49beb787fe3d10fbca/fb0325615263ee49beb787fe3d10fbca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346540.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3e0/0b7f21fe5ec053f78cf33edd367773e0/0b7f21fe5ec053f78cf33edd367773e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346545.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/76b/9e8d0777d188283e683f18e92b26576b/9e8d0777d188283e683f18e92b26576b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346550.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/da2/978a84673caf17f6e238e1f7c69f5da2/978a84673caf17f6e238e1f7c69f5da2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346555.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/4ab/650468ecea3faccb01313058b06414ab/650468ecea3faccb01313058b06414ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346560.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/55c/7cd31e3ffb1e4055b8d2c3460452355c/7cd31e3ffb1e4055b8d2c3460452355c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346565.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/628/9663e12e4cb538b401f8318f6effe628/9663e12e4cb538b401f8318f6effe628.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346570.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/a37/75498a60f92af9f4065cf5c1e96c4a37/75498a60f92af9f4065cf5c1e96c4a37.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346575.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/918/0ce659162d53e3402bb4f2e5fb69e918/0ce659162d53e3402bb4f2e5fb69e918.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346580.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/8e0/b1f71f0a1cd65de97b8d7813065008e0/b1f71f0a1cd65de97b8d7813065008e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346585.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/624/b263f2bb1ee7589aa6f3f027ea6e8624/b263f2bb1ee7589aa6f3f027ea6e8624.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346590.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d6f/e04456b94d81ffde2f4c1bf86c55ad6f/e04456b94d81ffde2f4c1bf86c55ad6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346595.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/808/0704f115ad1aa517367e9cc077a5f808/0704f115ad1aa517367e9cc077a5f808.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346600.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/386/8f60a24b0763e83306f053f190130386/8f60a24b0763e83306f053f190130386.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346605.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/506/5200f0415b583f199212b0fe457fc506/5200f0415b583f199212b0fe457fc506.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346610.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/113/53515dd3edcd6c5668a1288a56ad0113/53515dd3edcd6c5668a1288a56ad0113.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346615.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/ff1/c5b31e8f5421c2147da80a7ee4b59ff1/c5b31e8f5421c2147da80a7ee4b59ff1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346620.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/088/863216b28b8d9de1f0d01c4d97f91088/863216b28b8d9de1f0d01c4d97f91088.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346625.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/af7/41ac7f38b3c272cb72afe84a34302af7/41ac7f38b3c272cb72afe84a34302af7.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving ... ok test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0166 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0072 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0003 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0162 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0087 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0062 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0085 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0087 "-" 3542017 0 test ERROR: ERROR Account update failed with 127.0.0.1:38945/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0099 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0114 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0070 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0044 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0185 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0171 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0137 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0026 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:31 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0032 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0059 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0014 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0124 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0019 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0019 "-" 3542017 - test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346630.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/664/93af3c7276f1915e0e533d3575ac2664/93af3c7276f1915e0e533d3575ac2664.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346635.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c98/c31b1ecfb59ba5d375c46a576ff53c98/c31b1ecfb59ba5d375c46a576ff53c98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346640.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/4a9/ebed0601ef2a99d416bd5021f8c144a9/ebed0601ef2a99d416bd5021f8c144a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346645.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/869/dccdd5d98be2faf4ca1e893d8f4c0869/dccdd5d98be2faf4ca1e893d8f4c0869.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346650.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/332/c9ee3f73ccffcca714ba7782a706e332/c9ee3f73ccffcca714ba7782a706e332.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346655.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/a77/b693b938bcee65738743624d738c9a77/b693b938bcee65738743624d738c9a77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346660.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/134/d5b3b459e35a145366538bc17402d134/d5b3b459e35a145366538bc17402d134.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346665.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/d5d/6722cb9f4972ee211d32a07a131e8d5d/6722cb9f4972ee211d32a07a131e8d5d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346670.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/bf3/2a7cc29edec53ded7ae07f08b604ebf3/2a7cc29edec53ded7ae07f08b604ebf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346675.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/153/7ad482048ebd1919bf6812ad4e7b1153/7ad482048ebd1919bf6812ad4e7b1153.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346680.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c43/a1bf1adb8d21e7c9632d65d63f93ec43/a1bf1adb8d21e7c9632d65d63f93ec43.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346685.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/7c5/c0e2e021474efd5016dd026a237077c5/c0e2e021474efd5016dd026a237077c5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346690.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/9c9/eae9c991d8964952b15708e00ee219c9/eae9c991d8964952b15708e00ee219c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346695.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/5f4/86b1d447ceeae65e842f8cf1c99e75f4/86b1d447ceeae65e842f8cf1c99e75f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346700.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/44a/d6a3c3755f2bc016d6191ae48a92544a/d6a3c3755f2bc016d6191ae48a92544a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346705.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/196/b6c52425e5974a4b94148a113a936196/b6c52425e5974a4b94148a113a936196.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346710.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/c71/d7537b0c55509c16244d3066a06cbc71/d7537b0c55509c16244d3066a06cbc71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346715.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/8b4/e498086c99c95cade73a3d27964f88b4/e498086c99c95cade73a3d27964f88b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346720.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2ca/61d009ebed6af20fc6d8c0abf86d52ca/61d009ebed6af20fc6d8c0abf86d52ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346725.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3df/1552232e7b6bbee4be40a383112573df/1552232e7b6bbee4be40a383112573df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346730.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/317/506dce6def235306bebe4a2efd91c317/506dce6def235306bebe4a2efd91c317.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346735.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/159/1fa14251720d06d96848c7a709457159/1fa14251720d06d96848c7a709457159.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346740.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3bb/795c25f8e347e013031f800b22df03bb/795c25f8e347e013031f800b22df03bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346745.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/3dc/9f59a29d945011e59f83056cbb0243dc/9f59a29d945011e59f83056cbb0243dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346750.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/f39/4e3f5a96c5470d1e63bb5b18abb25f39/4e3f5a96c5470d1e63bb5b18abb25f39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346755.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/2b1/9435dda45d016bd74bcd2de46da8c2b1/9435dda45d016bd74bcd2de46da8c2b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346760.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/4ca/250c29654176c25b909eb918f99ae4ca/250c29654176c25b909eb918f99ae4ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346765.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/ecf/387fe83472f8fcec0d71c76740455ecf/387fe83472f8fcec0d71c76740455ecf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346770.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/510/6dfcf0d048986627b7baff3f9f9e4510/6dfcf0d048986627b7baff3f9f9e4510.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346775.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/df1/f265290c40b25c6ef47c41feab525df1/f265290c40b25c6ef47c41feab525df1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346780.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/823/839f9468909c31df34c0c99a9a819823/839f9468909c31df34c0c99a9a819823.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346785.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/04b/de9ea7d8be203ec55b83f2ed38f2d04b/de9ea7d8be203ec55b83f2ed38f2d04b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346790.00000-0, db: /tmp/tmpn8ikaitg/sda/containers/0/661/037dd5d366ffc6de17275699a134a661/037dd5d366ffc6de17275699a134a661.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'found': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_audit_root_container test.unit.container.test_sharder.TestSharder.test_audit_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container test.unit.container.test_sharder.TestSharder.test_audit_shard_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding ... ok test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0024 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0019 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0055 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0014 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0013 "-" 3542017 - test DEBUG: Unlinked retiring db '/tmp/tmp541cow1b/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0051 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0058 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0058 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0019 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0111 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0110 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:32 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0003 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_GET_auto_record_type test.unit.container.test_server.TestContainerController.test_GET_auto_record_type ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter test.unit.container.test_server.TestContainerController.test_GET_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestContainerController.test_GET_invalid_accept test.unit.container.test_server.TestContainerController.test_GET_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_GET_json test.unit.container.test_server.TestContainerController.test_GET_json ... ok sharder-test WARNING: Audit failed for root: overlapping ranges in state 'created': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'sharding': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmp9psap1en/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: own_shard_range reset to None should be 1717345528.00000, path: a/c, db: /tmp/tmp8tm7qdr6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345528.00000.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': a-c a-f, a-f c-f, path: a/c, db: /tmp/tmpl4aijkcx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345529.00000-0, db: /tmp/tmp6wohmc0u/sda/containers/0/48b/394acb01aaf36d7fbcd017b1969aa48b/394acb01aaf36d7fbcd017b1969aa48b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345529.00000-0, db: /tmp/tmp6wohmc0u/sda/containers/0/48b/394acb01aaf36d7fbcd017b1969aa48b/394acb01aaf36d7fbcd017b1969aa48b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345532.00000-0, db: /tmp/tmp6wohmc0u/sda/containers/0/5c9/10229255f75f5a8091d0610af77f95c9/10229255f75f5a8091d0610af77f95c9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345532.00000-0, db: /tmp/tmp6wohmc0u/sda/containers/0/5c9/10229255f75f5a8091d0610af77f95c9/10229255f75f5a8091d0610af77f95c9.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717345529.00000, (0, 0) as of 1717345529.00000, active as of 1717345529.00000> to ShardRange<'k' to 't' as of 1717345530.00000, (0, 0) as of 1717345530.00000, active as of 1717345530.00000>, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345534.00000, (0, 0) as of 1717345534.00000, shrinking as of 1717345531.00000> to ShardRange<'k' to 't' as of 1717345534.00000, (0, 0) as of 1717345534.00000, sharding as of 1717345535.00000>, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmp5mx_9roz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-548e70ddafed3d2f8f2520fafc3b2fc4-1717345533.00000-2, db: /tmp/tmp1udqrzx2/sda/containers/0/1f2/e12d399db3a4a7150feccb01c600c1f2/e12d399db3a4a7150feccb01c600c1f2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345529.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/bad/342c1043f61ff13cdc3a8aaf7c8e0bad/342c1043f61ff13cdc3a8aaf7c8e0bad.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345529.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/bad/342c1043f61ff13cdc3a8aaf7c8e0bad/342c1043f61ff13cdc3a8aaf7c8e0bad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345538.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/1a4/9f132b43210d768fd98805df99bdc1a4/9f132b43210d768fd98805df99bdc1a4.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345538.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/1a4/9f132b43210d768fd98805df99bdc1a4/9f132b43210d768fd98805df99bdc1a4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345547.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/a0c/cec208cafbbd059d2b2a75b199170a0c/cec208cafbbd059d2b2a75b199170a0c.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345547.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/a0c/cec208cafbbd059d2b2a75b199170a0c/cec208cafbbd059d2b2a75b199170a0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345556.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/29f/4fbace147488bbf32b3043135131a29f/4fbace147488bbf32b3043135131a29f.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345556.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/29f/4fbace147488bbf32b3043135131a29f/4fbace147488bbf32b3043135131a29f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345565.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/39b/a2f7c2bdaff50dd343df475b97fdf39b/a2f7c2bdaff50dd343df475b97fdf39b.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345565.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/39b/a2f7c2bdaff50dd343df475b97fdf39b/a2f7c2bdaff50dd343df475b97fdf39b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345574.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/06a/4637d1c8275dace29d2ecf581011a06a/4637d1c8275dace29d2ecf581011a06a.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345574.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/06a/4637d1c8275dace29d2ecf581011a06a/4637d1c8275dace29d2ecf581011a06a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345583.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/ab6/00197976c8402e687e8748e982a40ab6/00197976c8402e687e8748e982a40ab6.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345583.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/ab6/00197976c8402e687e8748e982a40ab6/00197976c8402e687e8748e982a40ab6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345592.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/98a/8e3faabbe8a1ee0068bce9eb1096b98a/8e3faabbe8a1ee0068bce9eb1096b98a.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345592.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/98a/8e3faabbe8a1ee0068bce9eb1096b98a/8e3faabbe8a1ee0068bce9eb1096b98a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345601.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/d36/3b310a6d232cb5a2337dbcfdc2620d36/3b310a6d232cb5a2337dbcfdc2620d36.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345601.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/d36/3b310a6d232cb5a2337dbcfdc2620d36/3b310a6d232cb5a2337dbcfdc2620d36.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345601.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/d36/3b310a6d232cb5a2337dbcfdc2620d36/3b310a6d232cb5a2337dbcfdc2620d36.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345610.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/623/bf7915bde3fe1281506520e5b8956623/bf7915bde3fe1281506520e5b8956623.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345610.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/623/bf7915bde3fe1281506520e5b8956623/bf7915bde3fe1281506520e5b8956623.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345610.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/623/bf7915bde3fe1281506520e5b8956623/bf7915bde3fe1281506520e5b8956623.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345619.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/f8e/27c1719f46bf420cef4d6832cd021f8e/27c1719f46bf420cef4d6832cd021f8e.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345619.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/f8e/27c1719f46bf420cef4d6832cd021f8e/27c1719f46bf420cef4d6832cd021f8e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345619.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/f8e/27c1719f46bf420cef4d6832cd021f8e/27c1719f46bf420cef4d6832cd021f8e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345628.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/a17/62abc89332e95bef8668db1aa84c4a17/62abc89332e95bef8668db1aa84c4a17.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345628.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/a17/62abc89332e95bef8668db1aa84c4a17/62abc89332e95bef8668db1aa84c4a17.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345628.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/a17/62abc89332e95bef8668db1aa84c4a17/62abc89332e95bef8668db1aa84c4a17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345637.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/b76/f5df16929cbc44dd988ffac7473a2b76/f5df16929cbc44dd988ffac7473a2b76.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345637.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/b76/f5df16929cbc44dd988ffac7473a2b76/f5df16929cbc44dd988ffac7473a2b76.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345637.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/b76/f5df16929cbc44dd988ffac7473a2b76/f5df16929cbc44dd988ffac7473a2b76.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345646.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/6b0/1fc1ac38ee6783f04ff1142ceab646b0/1fc1ac38ee6783f04ff1142ceab646b0.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345646.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/6b0/1fc1ac38ee6783f04ff1142ceab646b0/1fc1ac38ee6783f04ff1142ceab646b0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345646.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/6b0/1fc1ac38ee6783f04ff1142ceab646b0/1fc1ac38ee6783f04ff1142ceab646b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345655.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/898/18949fd53c20ceb9283973e42332c898/18949fd53c20ceb9283973e42332c898.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345655.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/898/18949fd53c20ceb9283973e42332c898/18949fd53c20ceb9283973e42332c898.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345655.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/898/18949fd53c20ceb9283973e42332c898/18949fd53c20ceb9283973e42332c898.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345664.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/843/a912caff95cc19375e9a1804de6f6843/a912caff95cc19375e9a1804de6f6843.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345664.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/843/a912caff95cc19375e9a1804de6f6843/a912caff95cc19375e9a1804de6f6843.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded ... ok test INFO: - - - [02/Jun/2024:16:25:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0107 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:33 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:33 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0034 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0038 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0034 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0042 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0080 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0036 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0035 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0042 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0044 "-" 3542017 0 test DEBUG: Unlinked retiring db '/tmp/tmp8tqen9a1/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0039 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0115 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0121 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0124 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0035 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0112 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0110 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0157 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0088 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0120 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0082 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0020 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0056 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0035 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0111 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:34 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test.unit.container.test_server.TestContainerController.test_GET_json_last_modified test.unit.container.test_server.TestContainerController.test_GET_json_last_modified ... ok test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_limit test.unit.container.test_server.TestContainerController.test_GET_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_false test.unit.container.test_server.TestContainerController.test_GET_log_requests_false ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_true test.unit.container.test_server.TestContainerController.test_GET_log_requests_true ... ok test.unit.container.test_server.TestContainerController.test_GET_marker test.unit.container.test_server.TestContainerController.test_GET_marker ... ok test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET ... ok test INFO: - - - [02/Jun/2024:16:25:34 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0108 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0137 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0095 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0128 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0096 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0131 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0099 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0042 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0041 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0039 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:35 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0044 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0043 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0046 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0083 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0045 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0042 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0041 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0044 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0021 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:36 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0086 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0021 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0051 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0050 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0024 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0021 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0052 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0029 "-" 3542017 0 test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_non_ascii test.unit.container.test_server.TestContainerController.test_GET_non_ascii ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource ... ok test INFO: - - - [02/Jun/2024:16:25:36 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0022 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0195 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0146 "-" 3542017 0 test DEBUG: Unlinked retiring db '/tmp/tmp7ukiqe6m/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0029 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0159 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0181 "-" 3542017 0 test DEBUG: Unlinked retiring db '/tmp/tmplw2j7y7n/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0059 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0031 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0131 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0028 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0120 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0165 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0120 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0116 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0212 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0137 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0032 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0147 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0098 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0031 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0005 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0005 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestContainerController.test_GET_over_limit test.unit.container.test_server.TestContainerController.test_GET_over_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_path test.unit.container.test_server.TestContainerController.test_GET_path ... ok test.unit.container.test_server.TestContainerController.test_GET_plain test.unit.container.test_server.TestContainerController.test_GET_plain ... ok test.unit.container.test_server.TestContainerController.test_GET_prefix test.unit.container.test_server.TestContainerController.test_GET_prefix ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors ... ok test INFO: - - - [02/Jun/2024:16:25:37 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0004 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0112 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0034 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0014 "-" 3542017 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0009 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0010 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0120 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0129 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0136 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0082 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0075 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0039 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0035 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0105 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0093 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0113 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0058 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0021 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0110 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0019 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0074 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0131 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0111 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0043 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:38 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0025 "-" 3542017 0 test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket ... ok test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0024 "-" 3542017 0 test DEBUG: Unlinked retiring db '/tmp/tmppa69xqir/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0062 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0027 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:39 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:39 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0049 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0060 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0027 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0101 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0094 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0315 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0108 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0027 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0020 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0019 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0072 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0072 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0047 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0106 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0077 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0107 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0088 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0113 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0097 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0033 "-" 3542017 0test.unit.container.test_server.TestContainerController.test_GET_xml test.unit.container.test_server.TestContainerController.test_GET_xml ... ok test.unit.container.test_server.TestContainerController.test_HEAD test.unit.container.test_server.TestContainerController.test_HEAD ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_HEAD_not_found test.unit.container.test_server.TestContainerController.test_HEAD_not_found ... ok test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestContainerController.test_OPTIONS test.unit.container.test_server.TestContainerController.test_OPTIONS ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestContainerController.test_POST_insufficient_space test.unit.container.test_server.TestContainerController.test_POST_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_partition test.unit.container.test_server.TestContainerController.test_POST_invalid_partition ... ok proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4dc81a7825d84b96b6327-00665c9cf5) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txcd00955a8f7b42b784eec-00665c9cf6) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d673277681549108cc29-00665c9cf7) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfb350e08e0e8412b82809-00665c9cf8) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_default ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers ... ok test.unit.common.test_db.TestChexor.test_chexor test.unit.common.test_db.TestChexor.test_chexor ... ok test.unit.common.test_db.TestChexor.test_invalid_old_hash test.unit.common.test_db.TestChexor.test_invalid_old_hash ... ok test.unit.common.test_db.TestChexor.test_no_name test.unit.common.test_db.TestChexor.test_no_name ... ok test.unit.common.test_db.TestChexor.test_normal_case test.unit.common.test_db.TestChexor.test_normal_case ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same ... ok test.unit.common.test_utils.TestFallocate.test_arg_bounds test.unit.common.test_utils.TestFallocate.test_arg_bounds ... ok test.unit.common.test_utils.TestFallocate.test_fallocate test.unit.common.test_utils.TestFallocate.test_fallocate ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_offset test.unit.common.test_utils.TestFallocate.test_fallocate_offset ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors ... ok test.unit.common.test_utils.TestFallocate.test_no_fallocates_available test.unit.common.test_utils.TestFallocate.test_no_fallocates_available ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works ... ok test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle ... ok test INFO: - - - [02/Jun/2024:16:25:40 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0038 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0020 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0035 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:40 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0038 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0110 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0012 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0197 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0003 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0004 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0003 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0002 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0131 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0094 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0077 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0089 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0105 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0177 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0120 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0122 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0081 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0125 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0079 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0137 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0015 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0127 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0058 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0207 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0019 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0183 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0071 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:41 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3542017 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:41 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0004 "-" 3542017 - test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT test.unit.container.test_server.TestContainerController.test_PUT ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0241 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:42 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0101 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0201 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0209 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0074 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0156 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3542017 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3542017 - test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0168 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0040 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0068 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0073 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0054 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0024 "-" 3542017 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345664.00000-2, db: /tmp/tmpm0rv3s71/sda/containers/0/843/a912caff95cc19375e9a1804de6f6843/a912caff95cc19375e9a1804de6f6843.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717345532.00000, (0, 0) as of 1717345532.00000, active as of 1717345532.00000> to ShardRange<'a' to 'b' as of 1717345532.00000, (0, 0) as of 1717345532.00000, sharding as of 1717345537.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f_1717345532.00000.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f_1717345532.00000.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f_1717345532.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f_1717345532.00000.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717345532.00000, (0, 0) as of 1717345532.00000, sharding as of 1717345538.00000> to ShardRange<'a' to 'b' as of 1717345532.00000, (0, 0) as of 1717345532.00000, sharded as of 1717345541.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345532.00000-2, db: /tmp/tmpxqcejjvp/sda/containers/0/91f/42c8fe264c247a0d05e9be3cc00a091f/42c8fe264c247a0d05e9be3cc00a091f_1717345532.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345533.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d17/79ecc70e5c3c5bbd5583828c56f86d17/79ecc70e5c3c5bbd5583828c56f86d17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345535.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/14b/d789d53d37d33a119251826521f3d14b/d789d53d37d33a119251826521f3d14b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345534.00000, (0, 0) as of 1717345534.00000, found as of 1717345534.00000> to ShardRange<'k' to 't' as of 1717345535.00000, (0, 0) as of 1717345535.00000, created as of 1717345535.00000>, path: .shards_a/shard_c_1717345535.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/14b/d789d53d37d33a119251826521f3d14b/d789d53d37d33a119251826521f3d14b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345537.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/58d/ce559ed708cc19e33754eda9191db58d/ce559ed708cc19e33754eda9191db58d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345536.00000, (0, 0) as of 1717345536.00000, found as of 1717345536.00000> to ShardRange<'k' to 't' as of 1717345537.00000, (0, 0) as of 1717345537.00000, cleaved as of 1717345537.00000>, path: .shards_a/shard_c_1717345537.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/58d/ce559ed708cc19e33754eda9191db58d/ce559ed708cc19e33754eda9191db58d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345539.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e4f/f07c22f3c8580c5fb077e952ec085e4f/f07c22f3c8580c5fb077e952ec085e4f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345538.00000, (0, 0) as of 1717345538.00000, found as of 1717345538.00000> to ShardRange<'k' to 't' as of 1717345539.00000, (0, 0) as of 1717345539.00000, active as of 1717345539.00000>, path: .shards_a/shard_c_1717345539.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e4f/f07c22f3c8580c5fb077e952ec085e4f/f07c22f3c8580c5fb077e952ec085e4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345541.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d7d/404a3150b54a88e0b4ded16db17b6d7d/404a3150b54a88e0b4ded16db17b6d7d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345540.00000, (0, 0) as of 1717345540.00000, found as of 1717345540.00000> to ShardRange<'k' to 't' as of 1717345541.00000, (0, 0) as of 1717345541.00000, shrinking as of 1717345541.00000>, path: .shards_a/shard_c_1717345541.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d7d/404a3150b54a88e0b4ded16db17b6d7d/404a3150b54a88e0b4ded16db17b6d7d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345541.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d7d/404a3150b54a88e0b4ded16db17b6d7d/404a3150b54a88e0b4ded16db17b6d7d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345543.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d74/aa35b90f46fa35ab9b1ff1badf8c1d74/aa35b90f46fa35ab9b1ff1badf8c1d74.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345542.00000, (0, 0) as of 1717345542.00000, found as of 1717345542.00000> to ShardRange<'k' to 't' as of 1717345543.00000, (0, 0) as of 1717345543.00000, sharding as of 1717345543.00000>, path: .shards_a/shard_c_1717345543.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d74/aa35b90f46fa35ab9b1ff1badf8c1d74/aa35b90f46fa35ab9b1ff1badf8c1d74.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345543.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d74/aa35b90f46fa35ab9b1ff1badf8c1d74/aa35b90f46fa35ab9b1ff1badf8c1d74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345545.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0e3/2cc3bd6a12d87de1f10f61704c81a0e3/2cc3bd6a12d87de1f10f61704c81a0e3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345544.00000, (0, 0) as of 1717345544.00000, found as of 1717345544.00000> to ShardRange<'k' to 't' as of 1717345545.00000, (0, 0) as of 1717345545.00000, sharded as of 1717345545.00000>, path: .shards_a/shard_c_1717345545.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0e3/2cc3bd6a12d87de1f10f61704c81a0e3/2cc3bd6a12d87de1f10f61704c81a0e3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345545.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0e3/2cc3bd6a12d87de1f10f61704c81a0e3/2cc3bd6a12d87de1f10f61704c81a0e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345547.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7db/d32c3b83e59039cf1b51f0c6228ca7db/d32c3b83e59039cf1b51f0c6228ca7db.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345546.00000, (0, 0) as of 1717345546.00000, found as of 1717345546.00000> to ShardRange<'k' to 't' as of 1717345547.00000, (0, 0) as of 1717345547.00000, shrunk as of 1717345547.00000>, path: .shards_a/shard_c_1717345547.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7db/d32c3b83e59039cf1b51f0c6228ca7db/d32c3b83e59039cf1b51f0c6228ca7db.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345547.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7db/d32c3b83e59039cf1b51f0c6228ca7db/d32c3b83e59039cf1b51f0c6228ca7db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345549.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/28d/c53d51703f314f6af3439bb2952d128d/c53d51703f314f6af3439bb2952d128d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345548.00000, (0, 0) as of 1717345548.00000, created as of 1717345548.00000> to ShardRange<'k' to 't' as of 1717345549.00000, (0, 0) as of 1717345549.00000, found as of 1717345549.00000>, path: .shards_a/shard_c_1717345549.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/28d/c53d51703f314f6af3439bb2952d128d/c53d51703f314f6af3439bb2952d128d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345551.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/90d/fdeae3878380afbdc57f90085ad1990d/fdeae3878380afbdc57f90085ad1990d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345553.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/148/4db13ae8a6d340d05f7550b5d4382148/4db13ae8a6d340d05f7550b5d4382148.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345552.00000, (0, 0) as of 1717345552.00000, created as of 1717345552.00000> to ShardRange<'k' to 't' as of 1717345553.00000, (0, 0) as of 1717345553.00000, cleaved as of 1717345553.00000>, path: .shards_a/shard_c_1717345553.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/148/4db13ae8a6d340d05f7550b5d4382148/4db13ae8a6d340d05f7550b5d4382148.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345555.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3eb/cdeee04938fd768d2ca8f2ebed4133eb/cdeee04938fd768d2ca8f2ebed4133eb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345554.00000, (0, 0) as of 1717345554.00000, created as of 1717345554.00000> to ShardRange<'k' to 't' as of 1717345555.00000, (0, 0) as of 1717345555.00000, active as of 1717345555.00000>, path: .shards_a/shard_c_1717345555.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3eb/cdeee04938fd768d2ca8f2ebed4133eb/cdeee04938fd768d2ca8f2ebed4133eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345557.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/891/79bbebe3e24eccb4596248123eb2c891/79bbebe3e24eccb4596248123eb2c891.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345556.00000, (0, 0) as of 1717345556.00000, created as of 1717345556.00000> to ShardRange<'k' to 't' as of 1717345557.00000, (0, 0) as of 1717345557.00000, shrinking as of 1717345557.00000>, path: .shards_a/shard_c_1717345557.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/891/79bbebe3e24eccb4596248123eb2c891/79bbebe3e24eccb4596248123eb2c891.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345557.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/891/79bbebe3e24eccb4596248123eb2c891/79bbebe3e24eccb4596248123eb2c891.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345559.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/33f/e6bde4368ff9d20ea3a7fc1ba673333f/e6bde4368ff9d20ea3a7fc1ba673333f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345558.00000, (0, 0) as of 1717345558.00000, created as of 1717345558.00000> to ShardRange<'k' to 't' as of 1717345559.00000, (0, 0) as of 1717345559.00000, sharding as of 1717345559.00000>, path: .shards_a/shard_c_1717345559.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/33f/e6bde4368ff9d20ea3a7fc1ba673333f/e6bde4368ff9d20ea3a7fc1ba673333f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345559.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/33f/e6bde4368ff9d20ea3a7fc1ba673333f/e6bde4368ff9d20ea3a7fc1ba673333f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345561.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/a02/6eba65cabd929e0543c8cbb418a5ba02/6eba65cabd929e0543c8cbb418a5ba02.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345560.00000, (0, 0) as of 1717345560.00000, created as of 1717345560.00000> to ShardRange<'k' to 't' as of 1717345561.00000, (0, 0) as of 1717345561.00000, sharded as of 1717345561.00000>, path: .shards_a/shard_c_1717345561.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/a02/6eba65cabd929e0543c8cbb418a5ba02/6eba65cabd929e0543c8cbb418a5ba02.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345561.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/a02/6eba65cabd929e0543c8cbb418a5ba02/6eba65cabd929e0543c8cbb418a5ba02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345563.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/303/32d5fb13a80bf26fa446fb0078fd3303/32d5fb13a80bf26fa446fb0078fd3303.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345562.00000, (0, 0) as of 1717345562.00000, created as of 1717345562.00000> to ShardRange<'k' to 't' as of 1717345563.00000, (0, 0) as of 1717345563.00000, shrunk as of 1717345563.00000>, path: .shards_a/shard_c_1717345563.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/303/32d5fb13a80bf26fa446fb0078fd3303/32d5fb13a80bf26fa446fb0078fd3303.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345563.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/303/32d5fb13a80bf26fa446fb0078fd3303/32d5fb13a80bf26fa446fb0078fd3303.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345565.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/1f8/4ebdca2454afeac84ccb4f25f78cb1f8/4ebdca2454afeac84ccb4f25f78cb1f8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345564.00000, (0, 0) as of 1717345564.00000, cleaved as of 1717345564.00000> to ShardRange<'k' to 't' as of 1717345565.00000, (0, 0) as of 1717345565.00000, found as of 1717345565.00000>, path: .shards_a/shard_c_1717345565.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/1f8/4ebdca2454afeac84ccb4f25f78cb1f8/4ebdca2454afeac84ccb4f25f78cb1f8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345567.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b44/c8dc155eefb68ee321c3bf1b3a809b44/c8dc155eefb68ee321c3bf1b3a809b44.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345566.00000, (0, 0) as of 1717345566.00000, cleaved as of 1717345566.00000> to ShardRange<'k' to 't' as of 1717345567.00000, (0, 0) as of 1717345567.00000, created as of 1717345567.00000>, path: .shards_a/shard_c_1717345567.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b44/c8dc155eefb68ee321c3bf1b3a809b44/c8dc155eefb68ee321c3bf1b3a809b44.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345569.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e05/a032b56226ad885df3a550e822594e05/a032b56226ad885df3a550e822594e05.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345571.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/692/a272be8b1c04dd44c51421fafffcb692/a272be8b1c04dd44c51421fafffcb692.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345570.00000, (0, 0) as of 1717345570.00000, cleaved as of 1717345570.00000> to ShardRange<'k' to 't' as of 1717345571.00000, (0, 0) as of 1717345571.00000, active as of 1717345571.00000>, path: .shards_a/shard_c_1717345571.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/692/a272be8b1c04dd44c51421fafffcb692/a272be8b1c04dd44c51421fafffcb692.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345573.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d70/68a28a59dcb920825874404837907d70/68a28a59dcb920825874404837907d70.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345572.00000, (0, 0) as of 1717345572.00000, cleaved as of 1717345572.00000> to ShardRange<'k' to 't' as of 1717345573.00000, (0, 0) as of 1717345573.00000, shrinking as of 1717345573.00000>, path: .shards_a/shard_c_1717345573.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d70/68a28a59dcb920825874404837907d70/68a28a59dcb920825874404837907d70.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345573.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d70/68a28a59dcb920825874404837907d70/68a28a59dcb920825874404837907d70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345575.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7a8/19ac830ef862fdc54e692aca1c89f7a8/19ac830ef862fdc54e692aca1c89f7a8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345574.00000, (0, 0) as of 1717345574.00000, cleaved as of 1717345574.00000> to ShardRange<'k' to 't' as of 1717345575.00000, (0, 0) as of 1717345575.00000, sharding as of 1717345575.00000>, path: .shards_a/shard_c_1717345575.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7a8/19ac830ef862fdc54e692aca1c89f7a8/19ac830ef862fdc54e692aca1c89f7a8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345575.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7a8/19ac830ef862fdc54e692aca1c89f7a8/19ac830ef862fdc54e692aca1c89f7a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345577.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4cb/7d09529259c864749f13ef7dc7aad4cb/7d09529259c864749f13ef7dc7aad4cb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345576.00000, (0, 0) as of 1717345576.00000, cleaved as of 1717345576.00000> to ShardRange<'k' to 't' as of 1717345577.00000, (0, 0) as of 1717345577.00000, sharded as of 1717345577.00000>, path: .shards_a/shard_c_1717345577.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4cb/7d09529259c864749f13ef7dc7aad4cb/7d09529259c864749f13ef7dc7aad4cb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345577.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4cb/7d09529259c864749f13ef7dc7aad4cb/7d09529259c864749f13ef7dc7aad4cb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345579.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/fbd/61ed5c8594b4e0563ab453dd4daa7fbd/61ed5c8594b4e0563ab453dd4daa7fbd.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345578.00000, (0, 0) as of 1717345578.00000, cleaved as of 1717345578.00000> to ShardRange<'k' to 't' as of 1717345579.00000, (0, 0) as of 1717345579.00000, shrunk as of 1717345579.00000>, path: .shards_a/shard_c_1717345579.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/fbd/61ed5c8594b4e0563ab453dd4daa7fbd/61ed5c8594b4e0563ab453dd4daa7fbd.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345579.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/fbd/61ed5c8594b4e0563ab453dd4daa7fbd/61ed5c8594b4e0563ab453dd4daa7fbd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345581.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/004/cedc0a0208c3d3ef2592fe375be9d004/cedc0a0208c3d3ef2592fe375be9d004.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345580.00000, (0, 0) as of 1717345580.00000, active as of 1717345580.00000> to ShardRange<'k' to 't' as of 1717345581.00000, (0, 0) as of 1717345581.00000, found as of 1717345581.00000>, path: .shards_a/shard_c_1717345581.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/004/cedc0a0208c3d3ef2592fe375be9d004/cedc0a0208c3d3ef2592fe375be9d004.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345583.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/656/3200dccb0e1a277e19235d9d1156d656/3200dccb0e1a277e19235d9d1156d656.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345582.00000, (0, 0) as of 1717345582.00000, active as of 1717345582.00000> to ShardRange<'k' to 't' as of 1717345583.00000, (0, 0) as of 1717345583.00000, created as of 1717345583.00000>, path: .shards_a/shard_c_1717345583.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/656/3200dccb0e1a277e19235d9d1156d656/3200dccb0e1a277e19235d9d1156d656.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345585.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/005/4b624ddf5562dc5485b27b72797e7005/4b624ddf5562dc5485b27b72797e7005.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345584.00000, (0, 0) as of 1717345584.00000, active as of 1717345584.00000> to ShardRange<'k' to 't' as of 1717345585.00000, (0, 0) as of 1717345585.00000, cleaved as of 1717345585.00000>, path: .shards_a/shard_c_1717345585.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/005/4b624ddf5562dc5485b27b72797e7005/4b624ddf5562dc5485b27b72797e7005.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345587.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/178/47ffa8a5b29c9eb51dee414cf5f09178/47ffa8a5b29c9eb51dee414cf5f09178.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345589.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/358/b56d8eb0d36bbff6b0bbcf153da96358/b56d8eb0d36bbff6b0bbcf153da96358.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345588.00000, (0, 0) as of 1717345588.00000, active as of 1717345588.00000> to ShardRange<'k' to 't' as of 1717345589.00000, (0, 0) as of 1717345589.00000, shrinking as of 1717345589.00000>, path: .shards_a/shard_c_1717345589.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/358/b56d8eb0d36bbff6b0bbcf153da96358/b56d8eb0d36bbff6b0bbcf153da96358.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345589.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/358/b56d8eb0d36bbff6b0bbcf153da96358/b56d8eb0d36bbff6b0bbcf153da96358.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345591.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3ac/48aba7d2d0fd0839fa7ae407eee973ac/48aba7d2d0fd0839fa7ae407eee973ac.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345590.00000, (0, 0) as of 1717345590.00000, active as of 1717345590.00000> to ShardRange<'k' to 't' as of 1717345591.00000, (0, 0) as of 1717345591.00000, sharding as of 1717345591.00000>, path: .shards_a/shard_c_1717345591.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3ac/48aba7d2d0fd0839fa7ae407eee973ac/48aba7d2d0fd0839fa7ae407eee973ac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345591.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3ac/48aba7d2d0fd0839fa7ae407eee973ac/48aba7d2d0fd0839fa7ae407eee973ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345593.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/8a6/def984a1dcaf383f8b1d17a7137f28a6/def984a1dcaf383f8b1d17a7137f28a6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345592.00000, (0, 0) as of 1717345592.00000, active as of 1717345592.00000> to ShardRange<'k' to 't' as of 1717345593.00000, (0, 0) as of 1717345593.00000, sharded as of 1717345593.00000>, path: .shards_a/shard_c_1717345593.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/8a6/def984a1dcaf383f8b1d17a7137f28a6/def984a1dcaf383f8b1d17a7137f28a6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345593.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/8a6/def984a1dcaf383f8b1d17a7137f28a6/def984a1dcaf383f8b1d17a7137f28a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345595.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c77/b3df4f9acb61019c80d5bdd73d35dc77/b3df4f9acb61019c80d5bdd73d35dc77.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345594.00000, (0, 0) as of 1717345594.00000, active as of 1717345594.00000> to ShardRange<'k' to 't' as of 1717345595.00000, (0, 0) as of 1717345595.00000, shrunk as of 1717345595.00000>, path: .shards_a/shard_c_1717345595.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c77/b3df4f9acb61019c80d5bdd73d35dc77/b3df4f9acb61019c80d5bdd73d35dc77.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345595.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c77/b3df4f9acb61019c80d5bdd73d35dc77/b3df4f9acb61019c80d5bdd73d35dc77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345597.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/50c/9a556d4ffa8304dbd2c3b0e1f66a450c/9a556d4ffa8304dbd2c3b0e1f66a450c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345596.00000, (0, 0) as of 1717345596.00000, shrinking as of 1717345596.00000> to ShardRange<'k' to 't' as of 1717345597.00000, (0, 0) as of 1717345597.00000, found as of 1717345597.00000>, path: .shards_a/shard_c_1717345597.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/50c/9a556d4ffa8304dbd2c3b0e1f66a450c/9a556d4ffa8304dbd2c3b0e1f66a450c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345599.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c9c/33bccbb948338f67857a93031992ec9c/33bccbb948338f67857a93031992ec9c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345598.00000, (0, 0) as of 1717345598.00000, shrinking as of 1717345598.00000> to ShardRange<'k' to 't' as of 1717345599.00000, (0, 0) as of 1717345599.00000, created as of 1717345599.00000>, path: .shards_a/shard_c_1717345599.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c9c/33bccbb948338f67857a93031992ec9c/33bccbb948338f67857a93031992ec9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345601.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/beb/f69d7c83533fd56f84a50d84fe1bbbeb/f69d7c83533fd56f84a50d84fe1bbbeb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345600.00000, (0, 0) as of 1717345600.00000, shrinking as of 1717345600.00000> to ShardRange<'k' to 't' as of 1717345601.00000, (0, 0) as of 1717345601.00000, cleaved as of 1717345601.00000>, path: .shards_a/shard_c_1717345601.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/beb/f69d7c83533fd56f84a50d84fe1bbbeb/f69d7c83533fd56f84a50d84fe1bbbeb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345603.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e88/6a2f4baf100ed5fb0c5db563cebd9e88/6a2f4baf100ed5fb0c5db563cebd9e88.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345602.00000, (0, 0) as of 1717345602.00000, shrinking as of 1717345602.00000> to ShardRange<'k' to 't' as of 1717345603.00000, (0, 0) as of 1717345603.00000, active as of 1717345603.00000>, path: .shards_a/shard_c_1717345603.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e88/6a2f4baf100ed5fb0c5db563cebd9e88/6a2f4baf100ed5fb0c5db563cebd9e88.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345605.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/cac/23b8ee027297dce03d7d795fcf050cac/23b8ee027297dce03d7d795fcf050cac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345605.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/cac/23b8ee027297dce03d7d795fcf050cac/23b8ee027297dce03d7d795fcf050cac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345607.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d9c/2ce720e257c9ff6fa803806283357d9c/2ce720e257c9ff6fa803806283357d9c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345606.00000, (0, 0) as of 1717345606.00000, shrinking as of 1717345606.00000> to ShardRange<'k' to 't' as of 1717345607.00000, (0, 0) as of 1717345607.00000, sharding as of 1717345607.00000>, path: .shards_a/shard_c_1717345607.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d9c/2ce720e257c9ff6fa803806283357d9c/2ce720e257c9ff6fa803806283357d9c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345607.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d9c/2ce720e257c9ff6fa803806283357d9c/2ce720e257c9ff6fa803806283357d9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345609.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/23c/4590255edaddefa01e90aa4c4407323c/4590255edaddefa01e90aa4c4407323c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345608.00000, (0, 0) as of 1717345608.00000, shrinking as of 1717345608.00000> to ShardRange<'k' to 't' as of 1717345609.00000, (0, 0) as of 1717345609.00000, sharded as of 1717345609.00000>, path: .shards_a/shard_c_1717345609.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/23c/4590255edaddefa01e90aa4c4407323c/4590255edaddefa01e90aa4c4407323c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345609.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/23c/4590255edaddefa01e90aa4c4407323c/4590255edaddefa01e90aa4c4407323c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345611.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/75f/d14b0632966b1fdec3366c56c399f75f/d14b0632966b1fdec3366c56c399f75f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345610.00000, (0, 0) as of 1717345610.00000, shrinking as of 1717345610.00000> to ShardRange<'k' to 't' as of 1717345611.00000, (0, 0) as of 1717345611.00000, shrunk as of 1717345611.00000>, path: .shards_a/shard_c_1717345611.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/75f/d14b0632966b1fdec3366c56c399f75f/d14b0632966b1fdec3366c56c399f75f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345611.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/75f/d14b0632966b1fdec3366c56c399f75f/d14b0632966b1fdec3366c56c399f75f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345613.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/04a/c1d7fe9cedd404848675a1695d6cc04a/c1d7fe9cedd404848675a1695d6cc04a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345612.00000, (0, 0) as of 1717345612.00000, sharding as of 1717345612.00000> to ShardRange<'k' to 't' as of 1717345613.00000, (0, 0) as of 1717345613.00000, found as of 1717345613.00000>, path: .shards_a/shard_c_1717345613.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/04a/c1d7fe9cedd404848675a1695d6cc04a/c1d7fe9cedd404848675a1695d6cc04a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345615.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/290/1781215481e1697a37c18d728a139290/1781215481e1697a37c18d728a139290.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345614.00000, (0, 0) as of 1717345614.00000, sharding as of 1717345614.00000> to ShardRange<'k' to 't' as of 1717345615.00000, (0, 0) as of 1717345615.00000, created as of 1717345615.00000>, path: .shards_a/shard_c_1717345615.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/290/1781215481e1697a37c18d728a139290/1781215481e1697a37c18d728a139290.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345617.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f4a/f3077a990489a2afcfa26ce77d9eff4a/f3077a990489a2afcfa26ce77d9eff4a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345616.00000, (0, 0) as of 1717345616.00000, sharding as of 1717345616.00000> to ShardRange<'k' to 't' as of 1717345617.00000, (0, 0) as of 1717345617.00000, cleaved as of 1717345617.00000>, path: .shards_a/shard_c_1717345617.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f4a/f3077a990489a2afcfa26ce77d9eff4a/f3077a990489a2afcfa26ce77d9eff4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345619.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ec1/714a958db6d882a85acfacdf8a15eec1/714a958db6d882a85acfacdf8a15eec1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345618.00000, (0, 0) as of 1717345618.00000, sharding as of 1717345618.00000> to ShardRange<'k' to 't' as of 1717345619.00000, (0, 0) as of 1717345619.00000, active as of 1717345619.00000>, path: .shards_a/shard_c_1717345619.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ec1/714a958db6d882a85acfacdf8a15eec1/714a958db6d882a85acfacdf8a15eec1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345621.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0af/1e94497743b0c7125e211c73ac4490af/1e94497743b0c7125e211c73ac4490af.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345620.00000, (0, 0) as of 1717345620.00000, sharding as of 1717345620.00000> to ShardRange<'k' to 't' as of 1717345621.00000, (0, 0) as of 1717345621.00000, shrinking as of 1717345621.00000>, path: .shards_a/shard_c_1717345621.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0af/1e94497743b0c7125e211c73ac4490af/1e94497743b0c7125e211c73ac4490af.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345621.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0af/1e94497743b0c7125e211c73ac4490af/1e94497743b0c7125e211c73ac4490af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345623.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f18/48470eac65a425fcbd18fcf7db3aff18/48470eac65a425fcbd18fcf7db3aff18.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345623.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f18/48470eac65a425fcbd18fcf7db3aff18/48470eac65a425fcbd18fcf7db3aff18.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345625.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9c0/75ecb7e07eb8123e9f62f010980919c0/75ecb7e07eb8123e9f62f010980919c0.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345624.00000, (0, 0) as of 1717345624.00000, sharding as of 1717345624.00000> to ShardRange<'k' to 't' as of 1717345625.00000, (0, 0) as of 1717345625.00000, sharded as of 1717345625.00000>, path: .shards_a/shard_c_1717345625.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9c0/75ecb7e07eb8123e9f62f010980919c0/75ecb7e07eb8123e9f62f010980919c0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345625.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9c0/75ecb7e07eb8123e9f62f010980919c0/75ecb7e07eb8123e9f62f010980919c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345627.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/748/f39af25d390351f7ec09d1e27037a748/f39af25d390351f7ec09d1e27037a748.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345626.00000, (0, 0) as of 1717345626.00000, sharding as of 1717345626.00000> to ShardRange<'k' to 't' as of 1717345627.00000, (0, 0) as of 1717345627.00000, shrunk as of 1717345627.00000>, path: .shards_a/shard_c_1717345627.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/748/f39af25d390351f7ec09d1e27037a748/f39af25d390351f7ec09d1e27037a748.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345627.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/748/f39af25d390351f7ec09d1e27037a748/f39af25d390351f7ec09d1e27037a748.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345629.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/217/2a2fbcdf6765a934863cad7f08381217/2a2fbcdf6765a934863cad7f08381217.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345628.00000, (0, 0) as of 1717345628.00000, sharded as of 1717345628.00000> to ShardRange<'k' to 't' as of 1717345629.00000, (0, 0) as of 1717345629.00000, found as of 1717345629.00000>, path: .shards_a/shard_c_1717345629.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/217/2a2fbcdf6765a934863cad7f08381217/2a2fbcdf6765a934863cad7f08381217.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345631.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/1fc/dc695622d17a238388ade848667531fc/dc695622d17a238388ade848667531fc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345630.00000, (0, 0) as of 1717345630.00000, sharded as of 1717345630.00000> to ShardRange<'k' to 't' as of 1717345631.00000, (0, 0) as of 1717345631.00000, created as of 1717345631.00000>, path: .shards_a/shard_c_1717345631.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/1fc/dc695622d17a238388ade848667531fc/dc695622d17a238388ade848667531fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345633.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/07a/503c19552acf3388a5b50f944029707a/503c19552acf3388a5b50f944029707a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345632.00000, (0, 0) as of 1717345632.00000, sharded as of 1717345632.00000> to ShardRange<'k' to 't' as of 1717345633.00000, (0, 0) as of 1717345633.00000, cleaved as of 1717345633.00000>, path: .shards_a/shard_c_1717345633.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/07a/503c19552acf3388a5b50f944029707a/503c19552acf3388a5b50f944029707a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345635.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/bc4/6f7b292b0aad0196e4db6df0f7be5bc4/6f7b292b0aad0196e4db6df0f7be5bc4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345634.00000, (0, 0) as of 1717345634.00000, sharded as of 1717345634.00000> to ShardRange<'k' to 't' as of 1717345635.00000, (0, 0) as of 1717345635.00000, active as of 1717345635.00000>, path: .shards_a/shard_c_1717345635.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/bc4/6f7b292b0aad0196e4db6df0f7be5bc4/6f7b292b0aad0196e4db6df0f7be5bc4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345637.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0ab/be188f6ec6cb355771ef11163a89b0ab/be188f6ec6cb355771ef11163a89b0ab.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345636.00000, (0, 0) as of 1717345636.00000, sharded as of 1717345636.00000> to ShardRange<'k' to 't' as of 1717345637.00000, (0, 0) as of 1717345637.00000, shrinking as of 1717345637.00000>, path: .shards_a/shard_c_1717345637.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0ab/be188f6ec6cb355771ef11163a89b0ab/be188f6ec6cb355771ef11163a89b0ab.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345637.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0ab/be188f6ec6cb355771ef11163a89b0ab/be188f6ec6cb355771ef11163a89b0ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345639.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ffe/e4ce19290d4e69f67e126d875b346ffe/e4ce19290d4e69f67e126d875b346ffe.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345638.00000, (0, 0) as of 1717345638.00000, sharded as of 1717345638.00000> to ShardRange<'k' to 't' as of 1717345639.00000, (0, 0) as of 1717345639.00000, sharding as of 1717345639.00000>, path: .shards_a/shard_c_1717345639.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ffe/e4ce19290d4e69f67e126d875b346ffe/e4ce19290d4e69f67e126d875b346ffe.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345639.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ffe/e4ce19290d4e69f67e126d875b346ffe/e4ce19290d4e69f67e126d875b346ffe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345641.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/be4/0aeb605ba5838dc5405dc2711d722be4/0aeb605ba5838dc5405dc2711d722be4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345641.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/be4/0aeb605ba5838dc5405dc2711d722be4/0aeb605ba5838dc5405dc2711d722be4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345643.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c2f/0cd97ef024087e6913faf00fd28a2c2f/0cd97ef024087e6913faf00fd28a2c2f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345642.00000, (0, 0) as of 1717345642.00000, sharded as of 1717345642.00000> to ShardRange<'k' to 't' as of 1717345643.00000, (0, 0) as of 1717345643.00000, shrunk as of 1717345643.00000>, path: .shards_a/shard_c_1717345643.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c2f/0cd97ef024087e6913faf00fd28a2c2f/0cd97ef024087e6913faf00fd28a2c2f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345643.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/c2f/0cd97ef024087e6913faf00fd28a2c2f/0cd97ef024087e6913faf00fd28a2c2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345645.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/423/2e0f0ee77c65efaa985181e10be00423/2e0f0ee77c65efaa985181e10be00423.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345644.00000, (0, 0) as of 1717345644.00000, shrunk as of 1717345644.00000> to ShardRange<'k' to 't' as of 1717345645.00000, (0, 0) as of 1717345645.00000, found as of 1717345645.00000>, path: .shards_a/shard_c_1717345645.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/423/2e0f0ee77c65efaa985181e10be00423/2e0f0ee77c65efaa985181e10be00423.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345647.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/66f/7adda34f1cdb93bfc5c8e3987e3e166f/7adda34f1cdb93bfc5c8e3987e3e166f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345646.00000, (0, 0) as of 1717345646.00000, shrunk as of 1717345646.00000> to ShardRange<'k' to 't' as of 1717345647.00000, (0, 0) as of 1717345647.00000, created as of 1717345647.00000>, path: .shards_a/shard_c_1717345647.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/66f/7adda34f1cdb93bfc5c8e3987e3e166f/7adda34f1cdb93bfc5c8e3987e3e166f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345649.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/522/9023e4dff4e49c45a4257d7101e58522/9023e4dff4e49c45a4257d7101e58522.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345648.00000, (0, 0) as of 1717345648.00000, shrunk as of 1717345648.00000> to ShardRange<'k' to 't' as of 1717345649.00000, (0, 0) as of 1717345649.00000, cleaved as of 1717345649.00000>, path: .shards_a/shard_c_1717345649.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/522/9023e4dff4e49c45a4257d7101e58522/9023e4dff4e49c45a4257d7101e58522.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345651.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/88a/4c23003c9bec75070c8fcc99133f088a/4c23003c9bec75070c8fcc99133f088a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345650.00000, (0, 0) as of 1717345650.00000, shrunk as of 1717345650.00000> to ShardRange<'k' to 't' as of 1717345651.00000, (0, 0) as of 1717345651.00000, active as of 1717345651.00000>, path: .shards_a/shard_c_1717345651.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/88a/4c23003c9bec75070c8fcc99133f088a/4c23003c9bec75070c8fcc99133f088a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345653.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9df/19a23e93e73f42b2610160b2c4f1e9df/19a23e93e73f42b2610160b2c4f1e9df.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345652.00000, (0, 0) as of 1717345652.00000, shrunk as of 1717345652.00000> to ShardRange<'k' to 't' as of 1717345653.00000, (0, 0) as of 1717345653.00000, shrinking as of 1717345653.00000>, path: .shards_a/shard_c_1717345653.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9df/19a23e93e73f42b2610160b2c4f1e9df/19a23e93e73f42b2610160b2c4f1e9df.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345653.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9df/19a23e93e73f42b2610160b2c4f1e9df/19a23e93e73f42b2610160b2c4f1e9df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345655.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/582/d86e3ee95d16e06ad78e9d5145d76582/d86e3ee95d16e06ad78e9d5145d76582.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345654.00000, (0, 0) as of 1717345654.00000, shrunk as of 1717345654.00000> to ShardRange<'k' to 't' as of 1717345655.00000, (0, 0) as of 1717345655.00000, sharding as of 1717345655.00000>, path: .shards_a/shard_c_1717345655.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/582/d86e3ee95d16e06ad78e9d5145d76582/d86e3ee95d16e06ad78e9d5145d76582.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345655.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/582/d86e3ee95d16e06ad78e9d5145d76582/d86e3ee95d16e06ad78e9d5145d76582.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345657.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/aaa/73da3437ac173d70fe6535eeaa96eaaa/73da3437ac173d70fe6535eeaa96eaaa.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345656.00000, (0, 0) as of 1717345656.00000, shrunk as of 1717345656.00000> to ShardRange<'k' to 't' as of 1717345657.00000, (0, 0) as of 1717345657.00000, sharded as of 1717345657.00000>, path: .shards_a/shard_c_1717345657.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/aaa/73da3437ac173d70fe6535eeaa96eaaa/73da3437ac173d70fe6535eeaa96eaaa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345657.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/aaa/73da3437ac173d70fe6535eeaa96eaaa/73da3437ac173d70fe6535eeaa96eaaa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345659.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/01c/fc0bd267b93904f0672acbdd1f82301c/fc0bd267b93904f0672acbdd1f82301c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345659.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/01c/fc0bd267b93904f0672acbdd1f82301c/fc0bd267b93904f0672acbdd1f82301c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345660.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3d6/10ebb754bde56306fac0335175e063d6/10ebb754bde56306fac0335175e063d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345662.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6d8/acd14b1972607cbb69969a8578a926d8/acd14b1972607cbb69969a8578a926d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345664.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/22e/7f45411f2839db5ac789fd5ea151822e/7f45411f2839db5ac789fd5ea151822e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345666.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6df/b6c16d33e42e25f8dad4f2cfe3f0d6df/b6c16d33e42e25f8dad4f2cfe3f0d6df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345668.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/915/461b12e5c6824e922001a5be89815915/461b12e5c6824e922001a5be89815915.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345670.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7c9/334500d5030ee0998714e69c8f0607c9/334500d5030ee0998714e69c8f0607c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345672.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d75/3f5935f3d72cb7f72a4958f2f321bd75/3f5935f3d72cb7f72a4958f2f321bd75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345674.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/720/396ca3dc69c187f7f443113687a65720/396ca3dc69c187f7f443113687a65720.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345676.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/646/e5696ee97c27dfd1c4341b781d0da646/e5696ee97c27dfd1c4341b781d0da646.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345678.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e35/69618a19e0b43e00ccaa31af34f6ce35/69618a19e0b43e00ccaa31af34f6ce35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345680.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3c3/4de4915ba3b4f2d6b400f199fa6e13c3/4de4915ba3b4f2d6b400f199fa6e13c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345682.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/216/0487ffefd35f27d897fbee2d6c3a3216/0487ffefd35f27d897fbee2d6c3a3216.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345684.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/df7/05c614d5e78a9875839e47efd87fedf7/05c614d5e78a9875839e47efd87fedf7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345686.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/3e8/9e213b8c7708aea89e8e061fa2eba3e8/9e213b8c7708aea89e8e061fa2eba3e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345688.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/51b/154c231e7f08158b598cee395750951b/154c231e7f08158b598cee395750951b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345690.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d9f/5f6fc095f8dff98aeb6acea290e7ed9f/5f6fc095f8dff98aeb6acea290e7ed9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345692.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6a6/ae0858e4ae6f364c934535d9b15ce6a6/ae0858e4ae6f364c934535d9b15ce6a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345694.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/7f7/e76f8978ef75bf32f0b0918f5c4847f7/e76f8978ef75bf32f0b0918f5c4847f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345696.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/68b/d6687481d89bd8b3afe156800b61968b/d6687481d89bd8b3afe156800b61968b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345698.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/982/fec7098d81838444d77a55b635b85982/fec7098d81838444d77a55b635b85982.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345700.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/a1c/55c6bfaa7234ee03f37ec9b9c8730a1c/55c6bfaa7234ee03f37ec9b9c8730a1c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345702.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/fff/e083d3c314f11d72439662ea72b66fff/e083d3c314f11d72439662ea72b66fff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345704.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/e84/020d00803203cf0a939f765ab580ee84/020d00803203cf0a939f765ab580ee84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345706.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ae3/f7fdd88232a529cc5dd37d99258c3ae3/f7fdd88232a529cc5dd37d99258c3ae3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345708.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ddf/e0195430a4e55d810d03fa7918c5bddf/e0195430a4e55d810d03fa7918c5bddf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345710.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/45f/dd4569e761401ce717d07f430dfb745f/dd4569e761401ce717d07f430dfb745f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345712.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/8ea/66bdabf0462941bf4ed94e3c0c8148ea/66bdabf0462941bf4ed94e3c0c8148ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345714.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/412/22be60f8da401b6f4e3d8a2a65e0e412/22be60f8da401b6f4e3d8a2a65e0e412.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345716.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f66/6d6f93129bfba4116edbafe7ae619f66/6d6f93129bfba4116edbafe7ae619f66.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345718.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/097/3b553351ccb787af8cfe7e78ea64c097/3b553351ccb787af8cfe7e78ea64c097.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345720.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/94d/f762bdaf8d329a2a73901ad5467ef94d/f762bdaf8d329a2a73901ad5467ef94d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345722.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/721/ac84b69e9b7871c806546bd64ae18721/ac84b69e9b7871c806546bd64ae18721.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345724.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/121/1f0675f6ee137167a58f2c948dc4f121/1f0675f6ee137167a58f2c948dc4f121.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345724.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/121/1f0675f6ee137167a58f2c948dc4f121/1f0675f6ee137167a58f2c948dc4f121.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345726.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ba7/655308f091b5b69479c84c37dda0fba7/655308f091b5b69479c84c37dda0fba7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345726.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ba7/655308f091b5b69479c84c37dda0fba7/655308f091b5b69479c84c37dda0fba7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345728.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/87c/b20ae98d47fc327b584b78ad1e38887c/b20ae98d47fc327b584b78ad1e38887c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345728.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/87c/b20ae98d47fc327b584b78ad1e38887c/b20ae98d47fc327b584b78ad1e38887c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345730.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4d0/215b7ee52deb64a5f715d63e18dda4d0/215b7ee52deb64a5f715d63e18dda4d0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345730.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4d0/215b7ee52deb64a5f715d63e18dda4d0/215b7ee52deb64a5f715d63e18dda4d0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345732.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/62b/1e2115d139dd1182ba8ee72d3c7c862b/1e2115d139dd1182ba8ee72d3c7c862b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345732.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/62b/1e2115d139dd1182ba8ee72d3c7c862b/1e2115d139dd1182ba8ee72d3c7c862b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345734.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4e3/ff852ffc38d391db2ac1e66c9e9524e3/ff852ffc38d391db2ac1e66c9e9524e3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345734.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/4e3/ff852ffc38d391db2ac1e66c9e9524e3/ff852ffc38d391db2ac1e66c9e9524e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345736.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d82/8025f65c0a06a98af8a7c26eba81cd82/8025f65c0a06a98af8a7c26eba81cd82.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345736.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d82/8025f65c0a06a98af8a7c26eba81cd82/8025f65c0a06a98af8a7c26eba81cd82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345738.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/28e/34299b99230dd730d07d3ec3c4ecd28e/34299b99230dd730d07d3ec3c4ecd28e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345738.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/28e/34299b99230dd730d07d3ec3c4ecd28e/34299b99230dd730d07d3ec3c4ecd28e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345740.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/aee/1e6fa51192e4a6c766ff867cadd29aee/1e6fa51192e4a6c766ff867cadd29aee.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345740.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/aee/1e6fa51192e4a6c766ff867cadd29aee/1e6fa51192e4a6c766ff867cadd29aee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345742.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6f0/78477a336228b5fce8e2b74e215c16f0/78477a336228b5fce8e2b74e215c16f0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345742.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6f0/78477a336228b5fce8e2b74e215c16f0/78477a336228b5fce8e2b74e215c16f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345744.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6dc/4a7adc2312b7f54994937a02099ab6dc/4a7adc2312b7f54994937a02099ab6dc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345744.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/6dc/4a7adc2312b7f54994937a02099ab6dc/4a7adc2312b7f54994937a02099ab6dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345746.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/a62/6385097bbaad4b3a2834ba7fdf58ca62/6385097bbaad4b3a2834ba7fdf58ca62.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345746.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/a62/6385097bbaad4b3a2834ba7fdf58ca62/6385097bbaad4b3a2834ba7fdf58ca62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345748.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/bfa/655ae98eb62c58148cc70b26a9e89bfa/655ae98eb62c58148cc70b26a9e89bfa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345748.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/bfa/655ae98eb62c58148cc70b26a9e89bfa/655ae98eb62c58148cc70b26a9e89bfa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345750.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b5b/fed9b60e7c9507062a2dfb81c938bb5b/fed9b60e7c9507062a2dfb81c938bb5b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345750.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b5b/fed9b60e7c9507062a2dfb81c938bb5b/fed9b60e7c9507062a2dfb81c938bb5b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345752.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f46/1600daaa091b911d97610369e70a4f46/1600daaa091b911d97610369e70a4f46.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345752.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f46/1600daaa091b911d97610369e70a4f46/1600daaa091b911d97610369e70a4f46.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345754.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/759/4ee057c57ae7ee957eae86aeaa92c759/4ee057c57ae7ee957eae86aeaa92c759.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345754.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/759/4ee057c57ae7ee957eae86aeaa92c759/4ee057c57ae7ee957eae86aeaa92c759.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345756.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b0b/9ca6a4c944d0f6a68533d0d0c9808b0b/9ca6a4c944d0f6a68533d0d0c9808b0b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345756.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b0b/9ca6a4c944d0f6a68533d0d0c9808b0b/9ca6a4c944d0f6a68533d0d0c9808b0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345758.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f67/33d10509818f4283878bdee7dc2e5f67/33d10509818f4283878bdee7dc2e5f67.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345758.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/f67/33d10509818f4283878bdee7dc2e5f67/33d10509818f4283878bdee7dc2e5f67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345760.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/540/bfc22a75e695352dd3599f067385f540/bfc22a75e695352dd3599f067385f540.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345760.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/540/bfc22a75e695352dd3599f067385f540/bfc22a75e695352dd3599f067385f540.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345762.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9ac/600d6c714c9e584722b33b362ff0a9ac/600d6c714c9e584722b33b362ff0a9ac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345762.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/9ac/600d6c714c9e584722b33b362ff0a9ac/600d6c714c9e584722b33b362ff0a9ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345764.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/019/46a1df36bf5a4163fe499b5de2437019/46a1df36bf5a4163fe499b5de2437019.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345764.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/019/46a1df36bf5a4163fe499b5de2437019/46a1df36bf5a4163fe499b5de2437019.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345766.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/8b4/cf8c153e613499825f02af468b8c98b4/cf8c153e613499825f02af468b8c98b4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345766.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/8b4/cf8c153e613499825f02af468b8c98b4/cf8c153e613499825f02af468b8c98b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345768.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b34/3f68e8365595a9a99d99225a3f4c7b34/3f68e8365595a9a99d99225a3f4c7b34.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345768.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/b34/3f68e8365595a9a99d99225a3f4c7b34/3f68e8365595a9a99d99225a3f4c7b34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345770.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/46e/858790e26d033c70a1bb52e0f35ad46e/858790e26d033c70a1bb52e0f35ad46e.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking ... ok proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1524a9bf03ed4952bfeca-00665c9cfb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 3, Lengths: [458, 458, 490, 458, 458, 458, 458, 458, 458, 458]: Invalid fragment payload in ECPyECLibDriver.decode (txn: txa6ed8feb47e0433aba98e-00665c9cfb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0945ee1e0fea4997bbf9b-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0945ee1e0fea4997bbf9b-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0945ee1e0fea4997bbf9b-00665c9cfc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 412 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from primary (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txcbd6c12f7fe3446cb5255-00665c9cfc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes ... ok test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0053 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0024 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0094 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0042 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0038 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0037 "-" 3542017 0 test DEBUG: Unlinked retiring db '/tmp/tmp9__o0qkt/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.1422 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0051 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0033 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0030 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:43 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0030 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0129 "-" 3542017 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3542017 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0175 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0074 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0174 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0023 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0221 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0013 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0116 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0111 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0060 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0115 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0099 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0144 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3542017 0 test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestContainerController.test_PUT_account_update test.unit.container.test_server.TestContainerController.test_PUT_account_update ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found ... ok sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345770.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/46e/858790e26d033c70a1bb52e0f35ad46e/858790e26d033c70a1bb52e0f35ad46e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345772.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/806/6ba96bba895bf6c6570eda779e0e8806/6ba96bba895bf6c6570eda779e0e8806.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345772.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/806/6ba96bba895bf6c6570eda779e0e8806/6ba96bba895bf6c6570eda779e0e8806.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345774.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ae2/46cca3350b89ffeb1750893c8e7a0ae2/46cca3350b89ffeb1750893c8e7a0ae2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345774.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/ae2/46cca3350b89ffeb1750893c8e7a0ae2/46cca3350b89ffeb1750893c8e7a0ae2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345776.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/324/e35e4354a632944446afc78c6fe53324/e35e4354a632944446afc78c6fe53324.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345776.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/324/e35e4354a632944446afc78c6fe53324/e35e4354a632944446afc78c6fe53324.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345778.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/dc6/833e76fe3a0855b1732f368a1989edc6/833e76fe3a0855b1732f368a1989edc6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345778.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/dc6/833e76fe3a0855b1732f368a1989edc6/833e76fe3a0855b1732f368a1989edc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345780.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/690/eb78fff1916ed0f4b7c04e920a440690/eb78fff1916ed0f4b7c04e920a440690.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345780.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/690/eb78fff1916ed0f4b7c04e920a440690/eb78fff1916ed0f4b7c04e920a440690.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345782.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0aa/d0dfea9038036e628c801ffaf6a1a0aa/d0dfea9038036e628c801ffaf6a1a0aa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345782.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/0aa/d0dfea9038036e628c801ffaf6a1a0aa/d0dfea9038036e628c801ffaf6a1a0aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345784.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d2a/e6f7095dd2a3d6a35272d4be3ff95d2a/e6f7095dd2a3d6a35272d4be3ff95d2a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345784.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d2a/e6f7095dd2a3d6a35272d4be3ff95d2a/e6f7095dd2a3d6a35272d4be3ff95d2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345786.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d10/539d6f80a2f83d7b43717d99a41ebd10/539d6f80a2f83d7b43717d99a41ebd10.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345786.00000, db: /tmp/tmpu3oaakgx/sda/containers/0/d10/539d6f80a2f83d7b43717d99a41ebd10/539d6f80a2f83d7b43717d99a41ebd10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmprlq7obf0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345542.00000, (0, 0) as of 1717345542.00000, active as of 1717345542.00000> to ShardRange<'k' to 't' as of 1717345543.00000, (0, 0) as of 1717345543.00000, sharded as of 1717345544.00000>, path: .shards_a/shard_c, db: /tmp/tmprlq7obf0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmprlq7obf0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpy_1odd_x/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345544.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/63d/739f6fc4fb33509b97fd8f94941c263d/739f6fc4fb33509b97fd8f94941c263d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345547.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/f1e/25bbbd2a1f0a52b5f67a9320f8572f1e/25bbbd2a1f0a52b5f67a9320f8572f1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345550.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/2ac/9c10e5d4fc6b0b9340534cf55ba872ac/9c10e5d4fc6b0b9340534cf55ba872ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345553.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/4a8/c8c467b1f4e66cf2f5f109d44a4b14a8/c8c467b1f4e66cf2f5f109d44a4b14a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345556.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/314/5f8f5297512020b3d0bd26138e691314/5f8f5297512020b3d0bd26138e691314.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345562.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/cc9/61c04ca51366ec21ff7343440ecaecc9/61c04ca51366ec21ff7343440ecaecc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345565.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/d0e/ea72c188c4224ac849a2d7fe169bad0e/ea72c188c4224ac849a2d7fe169bad0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345568.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/fe2/656d1904a6f8446eda6710621abcbfe2/656d1904a6f8446eda6710621abcbfe2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345574.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/05d/2dcd6cd427dae4950102d46f401fd05d/2dcd6cd427dae4950102d46f401fd05d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345577.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/e52/e4a7114b1d757b32a790161a9a483e52/e4a7114b1d757b32a790161a9a483e52.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp70pal12c/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345544.00000-0, db: /tmp/tmpm2ex75gg/sda/containers/0/63d/739f6fc4fb33509b97fd8f94941c263d/739f6fc4fb33509b97fd8f94941c263d.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking ... ok proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server ERROR: Object returning 503 for [] (txn: tx44633a4213f54945bae71-00665c9cfc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sda (txn: tx7f98abf6951f4c6b87f26-00665c9cff) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container ... ok test INFO: - - - [02/Jun/2024:16:25:44 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:44 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:44 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0095 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0072 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0184 "-" 3542017 - test ERROR: ERROR Account update failed with 127.0.0.1:35409/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0016 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3542017 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0005 "-" 3542017 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3542017 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0002 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0166 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0104 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0130 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3542017 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0065 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0146 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0022 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0020 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0021 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0131 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0010 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0214 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:45 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0678 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0028 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0021 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0010 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0050 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0024 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0050 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0014 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0023 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0019 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0022 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:46 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0050 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0015 "-" 3542017 - test DEBUG: Unlinked retiring db '/tmp/tmpdbrn6_ot/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0018 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0016 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0017 "-" 3542017 - test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT_utf8 test.unit.container.test_server.TestContainerController.test_PUT_utf8 ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345548.00000-0, db: /tmp/tmpm2ex75gg/sda/containers/0/2f1/ea45301e68053bf32c867148560df2f1/ea45301e68053bf32c867148560df2f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345547.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f1e/25bbbd2a1f0a52b5f67a9320f8572f1e/25bbbd2a1f0a52b5f67a9320f8572f1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345551.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b9a/a70380023fe2c4e7c94e4e497dff3b9a/a70380023fe2c4e7c94e4e497dff3b9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345555.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/1c0/701aacdac53989b0888114bc9fbbd1c0/701aacdac53989b0888114bc9fbbd1c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345563.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f72/254526a252cfc49e949d260c3605bf72/254526a252cfc49e949d260c3605bf72.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345567.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/45c/a214d05181bf3a58939eab98aaf2d45c/a214d05181bf3a58939eab98aaf2d45c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345575.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/8b0/f8ae096f335e76e81d948650f16eb8b0/f8ae096f335e76e81d948650f16eb8b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345579.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345587.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345591.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/494/736bc72b1b2f43d54260fe0ac1ced494/736bc72b1b2f43d54260fe0ac1ced494.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345591.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/494/736bc72b1b2f43d54260fe0ac1ced494/736bc72b1b2f43d54260fe0ac1ced494.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345603.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a45/98fc46327bc3d858f7b8d63d7ab08a45/98fc46327bc3d858f7b8d63d7ab08a45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345611.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345615.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f1e/d2df3b46c47d0a6c39116672a5fe8f1e/d2df3b46c47d0a6c39116672a5fe8f1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345627.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/142/427fb0be2423394f2109eb3f824ed142/427fb0be2423394f2109eb3f824ed142.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345639.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f62/8474b27a3f16620032180bcd8cceff62/8474b27a3f16620032180bcd8cceff62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345647.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345651.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/648/fbfc5e169925794f41d80b045b5bb648/fbfc5e169925794f41d80b045b5bb648.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345663.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a56/86591c8dc136221e54b9f1ec5334ea56/86591c8dc136221e54b9f1ec5334ea56.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345671.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345675.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b39/ade3a336b3114f0025b8f6ca4fa05b39/ade3a336b3114f0025b8f6ca4fa05b39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345683.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345687.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f81/75edaf0cd030162232217d2fef599f81/75edaf0cd030162232217d2fef599f81.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345687.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f81/75edaf0cd030162232217d2fef599f81/75edaf0cd030162232217d2fef599f81.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345699.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/1be/701a49feead126ddccd7e937c1acd1be/701a49feead126ddccd7e937c1acd1be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345707.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345711.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/657/3e8ef64022f6dfedbe89412c42002657/3e8ef64022f6dfedbe89412c42002657.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345723.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/07c/2fde1ec26ef1340c593460c15d84707c/2fde1ec26ef1340c593460c15d84707c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345735.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/85c/6cc1446b3d3747a132dfe6e37382285c/6cc1446b3d3747a132dfe6e37382285c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345743.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345747.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/bf8/64e7e0002088f0e061b43e4dd89b3bf8/64e7e0002088f0e061b43e4dd89b3bf8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345751.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345751.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345755.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/db9/d57ac6f55bb096b7b53bce53eb985db9/d57ac6f55bb096b7b53bce53eb985db9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345759.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/341/d9772fe63a4c24f892e39f1a7c29a341/d9772fe63a4c24f892e39f1a7c29a341.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345763.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/eba/c8656bf9dd67ae39a5ff0e8b0ed5beba/c8656bf9dd67ae39a5ff0e8b0ed5beba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345767.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/82a/af7a7d95593bebe38159c18cd239482a/af7a7d95593bebe38159c18cd239482a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345771.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f13/13dcd0d823693a4a26ba602b28f31f13/13dcd0d823693a4a26ba602b28f31f13.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345775.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a51/0357613d8a0b68cbecaaf47a2ea38a51/0357613d8a0b68cbecaaf47a2ea38a51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345779.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/3fd/7cde512a498cc4f6e7e54bdd9fce83fd/7cde512a498cc4f6e7e54bdd9fce83fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345783.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/ebd/6b695f37531b4d4f75d96dac331f6ebd/6b695f37531b4d4f75d96dac331f6ebd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345783.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/ebd/6b695f37531b4d4f75d96dac331f6ebd/6b695f37531b4d4f75d96dac331f6ebd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345787.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/ee6/ae9d9a47225312c25d2fa1f7d8b2eee6/ae9d9a47225312c25d2fa1f7d8b2eee6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345791.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a04/4e8dd23d553a52a0e9b40fb0b2444a04/4e8dd23d553a52a0e9b40fb0b2444a04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345795.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/09a/a7d77580e4fd4195087224c157e0a09a/a7d77580e4fd4195087224c157e0a09a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345799.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/788/e233b98157246fb2f16763643621d788/e233b98157246fb2f16763643621d788.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345803.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/22f/14ec906a79bed1ec1a02553c81f6022f/14ec906a79bed1ec1a02553c81f6022f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345807.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/01b/d96b6495f96d1d875f1a325b4345701b/d96b6495f96d1d875f1a325b4345701b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345811.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b5d/a0ca57477be8e33821cf5fd06a32cb5d/a0ca57477be8e33821cf5fd06a32cb5d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345815.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/b34/65a2ad93f8f5b2927b3701941925bb34/65a2ad93f8f5b2927b3701941925bb34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345819.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f70/0acab4cfaeb4df90edc3dbff69a60f70/0acab4cfaeb4df90edc3dbff69a60f70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345823.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/e03/e8af4b8e29298723c43e8a2b6decce03/e8af4b8e29298723c43e8a2b6decce03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345827.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/fdf/bc59eaa8477b3dc14f979f6d49281fdf/bc59eaa8477b3dc14f979f6d49281fdf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345831.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/648/5d054da4c811982b1ec46474cc885648/5d054da4c811982b1ec46474cc885648.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345835.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/025/600568b26e686e91b350099bbf036025/600568b26e686e91b350099bbf036025.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345839.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/5e3/aa48eb23cb4abb86c0917980853135e3/aa48eb23cb4abb86c0917980853135e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345843.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/cc5/133b0ea6c0251a0aceb196fb46903cc5/133b0ea6c0251a0aceb196fb46903cc5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345847.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/68e/02c60b29fedc06909056c2660953068e/02c60b29fedc06909056c2660953068e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345851.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345855.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/0b3/7da809de52d6c0590cd8f1957c7d40b3/7da809de52d6c0590cd8f1957c7d40b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345859.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/82a/5c7d54d86457214ab2b267672a27a82a/5c7d54d86457214ab2b267672a27a82a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345863.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f35/b7205c0fa015d4356e1a4e6c19164f35/b7205c0fa015d4356e1a4e6c19164f35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345867.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/10a/103cd16479c54c9e582618292a3c110a/103cd16479c54c9e582618292a3c110a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345871.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/577/ff75c4e36d977172a3d89f3beab16577/ff75c4e36d977172a3d89f3beab16577.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345875.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345879.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/277/b45fb34ccf3414c4389ce686a2cba277/b45fb34ccf3414c4389ce686a2cba277.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345883.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a86/9f96247331cfe4b0e61fad020caada86/9f96247331cfe4b0e61fad020caada86.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345887.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/e71/1484e75f1d95065b67c23a8eff750e71/1484e75f1d95065b67c23a8eff750e71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345891.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/20d/4eb4de89a5b06c3f96b6b1e27ba4320d/4eb4de89a5b06c3f96b6b1e27ba4320d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345895.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345903.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/892/b50ba0993aeed6c54f87327f9a014892/b50ba0993aeed6c54f87327f9a014892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345907.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345915.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d41/740cb5d220733796133a1347f1910d41/740cb5d220733796133a1347f1910d41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345923.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/0bd/4cb334d4f01ecd9abfdf3bdca8df30bd/4cb334d4f01ecd9abfdf3bdca8df30bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345927.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/000/9cc39cf5e202188f3cb365ab1f84b000/9cc39cf5e202188f3cb365ab1f84b000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345939.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d69/82df5f6d5f930f5efcd5c422ac1bcd69/82df5f6d5f930f5efcd5c422ac1bcd69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345947.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a17/d7ac936fa77aa2f44d71ebdc60230a17/d7ac936fa77aa2f44d71ebdc60230a17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345951.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/6f3/5145c6253305303db4393a56882626f3/5145c6253305303db4393a56882626f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345963.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/1c6/923db0dcb5f23439340d0336b4df81c6/923db0dcb5f23439340d0336b4df81c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345971.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345975.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/1c6/06727a10627bb589627c0835996451c6/06727a10627bb589627c0835996451c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345983.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/2fe/14e8e0360f977223aacbcdb1876582fe/14e8e0360f977223aacbcdb1876582fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345987.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/a2f/0976486a84408c4d0a9612b9c99f0a2f/0976486a84408c4d0a9612b9c99f0a2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345995.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345999.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346007.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/d9a/cd426bceff16dee8a0035688912e7d9a/cd426bceff16dee8a0035688912e7d9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346011.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/9b1/cecec856fd6c654aee8d170536e519b1/cecec856fd6c654aee8d170536e519b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346023.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/f7e/c76a6f341fe0b5daa394696434d01f7e/c76a6f341fe0b5daa394696434d01f7e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346035.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/037/21fe1f4898825a6a856e1f376b464037/21fe1f4898825a6a856e1f376b464037.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346043.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/34e/65d7cd5609f1cd05bcfa5f850928b34e/65d7cd5609f1cd05bcfa5f850928b34e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346047.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/e10/0c21378f348cc36f4019f220b9ec9e10/0c21378f348cc36f4019f220b9ec9e10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346051.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0020 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0035 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0017 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0023 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0013 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0050 "-" 3542017 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0207 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0038 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0274 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0090 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0075 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0147 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0176 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0127 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3542017 0 test DEBUG: Unlinked retiring db '/tmp/tmpxa7pbmo7/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:25:48 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0129 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0026 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0022 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0138 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0176 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:25:48 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:25:48 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0003 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:25:48 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestContainerController.test_UPDATE test.unit.container.test_server.TestContainerController.test_UPDATE ... ok test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestContainerController.test__call__raise_timeout test.unit.container.test_server.TestContainerController.test__call__raise_timeout ... ok test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device ... ok test.unit.container.test_server.TestContainerController.test_acl_container test.unit.container.test_server.TestContainerController.test_acl_container ... ok test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestContainerController.test_correct_allowed_method test.unit.container.test_server.TestContainerController.test_correct_allowed_method ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestContainerController.test_creation test.unit.container.test_server.TestContainerController.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim ... ok test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:25:48 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.3500 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:25:48 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0006 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:48 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:48 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:48 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0038 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0006 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0171 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0032 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0111 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0110 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0091 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0150 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0203 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0120 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0092 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0168 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0084 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0121 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0019 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0124 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0113 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0143 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:49 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0117 "-" 3542017 - test.unit.container.test_server.TestContainerController.test_delete_auto_create test.unit.container.test_server.TestContainerController.test_delete_auto_create ... ok test.unit.container.test_server.TestContainerController.test_deleted_headers test.unit.container.test_server.TestContainerController.test_deleted_headers ... ok test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestContainerController.test_list_allowed_methods test.unit.container.test_server.TestContainerController.test_list_allowed_methods ... ok test.unit.container.test_server.TestContainerController.test_listing_with_reserved test.unit.container.test_server.TestContainerController.test_listing_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_log_line_format test.unit.container.test_server.TestContainerController.test_log_line_format ... ok test.unit.container.test_server.TestContainerController.test_markers_with_reserved test.unit.container.test_server.TestContainerController.test_markers_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_not_allowed_method test.unit.container.test_server.TestContainerController.test_not_allowed_method ... ok test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0298 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:49 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0205 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0106 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0023 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0195 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0182 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0040 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0035 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0167 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0020 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0002 "-" 3542017 nada test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0002 "-" 3542017 999 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0168 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "HEAD /sda1/p/a/c_nulo" 204 - "-" "-" "-" 0.0025 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0133 "-" 3542017 1 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "HEAD /sda1/p/a/c_unu" 204 - "-" "-" "-" 0.0019 "-" 3542017 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0002 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0156 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0119 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0034 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0142 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0151 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0038 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0039 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0089 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0147 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0163 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0014 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0156 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0113 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0050 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0105 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0112 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0012 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0135 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0120 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3542017 0 test.unit.container.test_server.TestContainerController.test_object_update_with_offset test.unit.container.test_server.TestContainerController.test_object_update_with_offset ... ok test.unit.container.test_server.TestContainerController.test_params_format test.unit.container.test_server.TestContainerController.test_params_format ... ok test.unit.container.test_server.TestContainerController.test_params_utf8 test.unit.container.test_server.TestContainerController.test_params_utf8 ... ok test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_prefix_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_put_auto_create test.unit.container.test_server.TestContainerController.test_put_auto_create ... ok test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestContainerController.test_serv_reserv test.unit.container.test_server.TestContainerController.test_serv_reserv ... ok test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestContainerController.test_through_call test.unit.container.test_server.TestContainerController.test_through_call ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path test.unit.container.test_server.TestContainerController.test_through_call_invalid_path ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346055.00000-0, db: /tmp/tmp7gkazh9e/sda/containers/0/76b/8517e9828eb7eca20964951da6d8176b/8517e9828eb7eca20964951da6d8176b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmprvls7o4c/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345550.00000.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmprvls7o4c/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345550.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345553.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/4a8/c8c467b1f4e66cf2f5f109d44a4b14a8/c8c467b1f4e66cf2f5f109d44a4b14a8_1717345553.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345553.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/4a8/c8c467b1f4e66cf2f5f109d44a4b14a8/c8c467b1f4e66cf2f5f109d44a4b14a8_1717345553.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345556.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/314/5f8f5297512020b3d0bd26138e691314/5f8f5297512020b3d0bd26138e691314_1717345556.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345556.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/314/5f8f5297512020b3d0bd26138e691314/5f8f5297512020b3d0bd26138e691314_1717345556.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7_1717345559.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7_1717345559.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345562.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cc9/61c04ca51366ec21ff7343440ecaecc9/61c04ca51366ec21ff7343440ecaecc9_1717345562.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345562.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cc9/61c04ca51366ec21ff7343440ecaecc9/61c04ca51366ec21ff7343440ecaecc9_1717345562.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345565.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d0e/ea72c188c4224ac849a2d7fe169bad0e/ea72c188c4224ac849a2d7fe169bad0e_1717345565.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345565.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d0e/ea72c188c4224ac849a2d7fe169bad0e/ea72c188c4224ac849a2d7fe169bad0e_1717345565.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345568.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/fe2/656d1904a6f8446eda6710621abcbfe2/656d1904a6f8446eda6710621abcbfe2_1717345568.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345568.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/fe2/656d1904a6f8446eda6710621abcbfe2/656d1904a6f8446eda6710621abcbfe2_1717345568.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f_1717345571.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f_1717345571.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345574.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/05d/2dcd6cd427dae4950102d46f401fd05d/2dcd6cd427dae4950102d46f401fd05d_1717345574.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345574.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/05d/2dcd6cd427dae4950102d46f401fd05d/2dcd6cd427dae4950102d46f401fd05d_1717345574.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345577.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e52/e4a7114b1d757b32a790161a9a483e52/e4a7114b1d757b32a790161a9a483e52_1717345577.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345577.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e52/e4a7114b1d757b32a790161a9a483e52/e4a7114b1d757b32a790161a9a483e52_1717345577.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da_1717345580.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da_1717345580.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f_1717345583.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f_1717345583.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1_1717345586.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1_1717345586.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f_1717345589.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f_1717345589.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae_1717345592.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae_1717345592.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e_1717345595.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e_1717345595.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345598.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ab4/88fc9d55e59e3659795aa905ca0d5ab4/88fc9d55e59e3659795aa905ca0d5ab4_1717345598.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345598.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ab4/88fc9d55e59e3659795aa905ca0d5ab4/88fc9d55e59e3659795aa905ca0d5ab4_1717345598.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345601.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/3f6/cce3c78ea41024a78014a657c5ad13f6/cce3c78ea41024a78014a657c5ad13f6_1717345601.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345601.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/3f6/cce3c78ea41024a78014a657c5ad13f6/cce3c78ea41024a78014a657c5ad13f6_1717345601.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345604.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e80/977857deb4d7009d0e50754496596e80/977857deb4d7009d0e50754496596e80_1717345604.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345604.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e80/977857deb4d7009d0e50754496596e80/977857deb4d7009d0e50754496596e80_1717345604.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9_1717345607.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9_1717345607.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345610.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/a5f/e1cf616facba3f6f034939a512cc0a5f/e1cf616facba3f6f034939a512cc0a5f_1717345610.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345610.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/a5f/e1cf616facba3f6f034939a512cc0a5f/e1cf616facba3f6f034939a512cc0a5f_1717345610.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345613.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/247/214ea89a94368e76f96d8ffab7653247/214ea89a94368e76f96d8ffab7653247_1717345613.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345613.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/247/214ea89a94368e76f96d8ffab7653247/214ea89a94368e76f96d8ffab7653247_1717345613.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345616.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/c69/e485e5b3ca2786d2eb162a460c4dac69/e485e5b3ca2786d2eb162a460c4dac69_1717345616.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345616.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/c69/e485e5b3ca2786d2eb162a460c4dac69/e485e5b3ca2786d2eb162a460c4dac69_1717345616.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4_1717345619.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4_1717345619.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345622.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/be9/b3037b17f4b2fccbf125c19db43eebe9/b3037b17f4b2fccbf125c19db43eebe9_1717345622.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345622.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/be9/b3037b17f4b2fccbf125c19db43eebe9/b3037b17f4b2fccbf125c19db43eebe9_1717345622.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345625.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/371/6106ef7c90a2c4416c0b57ddfa173371/6106ef7c90a2c4416c0b57ddfa173371_1717345625.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345625.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/371/6106ef7c90a2c4416c0b57ddfa173371/6106ef7c90a2c4416c0b57ddfa173371_1717345625.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345628.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/34a/c3f72458cb53f1dfc3892aaefe7b934a/c3f72458cb53f1dfc3892aaefe7b934a_1717345628.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345628.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/34a/c3f72458cb53f1dfc3892aaefe7b934a/c3f72458cb53f1dfc3892aaefe7b934a_1717345628.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5_1717345631.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5_1717345631.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345634.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/612/f7cc4ec9d01b70c34157de2ebbb12612/f7cc4ec9d01b70c34157de2ebbb12612_1717345634.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345634.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/612/f7cc4ec9d01b70c34157de2ebbb12612/f7cc4ec9d01b70c34157de2ebbb12612_1717345634.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345637.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d0d/d32d7037cf1168c2cf9efa389f38dd0d/d32d7037cf1168c2cf9efa389f38dd0d_1717345637.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345637.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d0d/d32d7037cf1168c2cf9efa389f38dd0d/d32d7037cf1168c2cf9efa389f38dd0d_1717345637.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345640.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ea7/0e27dbff000bf481fef60d7129b98ea7/0e27dbff000bf481fef60d7129b98ea7_1717345640.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345640.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ea7/0e27dbff000bf481fef60d7129b98ea7/0e27dbff000bf481fef60d7129b98ea7_1717345640.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1_1717345643.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1_1717345643.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345646.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/5ec/8758d8619071fee95eaaac4ef274e5ec/8758d8619071fee95eaaac4ef274e5ec_1717345646.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345646.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/5ec/8758d8619071fee95eaaac4ef274e5ec/8758d8619071fee95eaaac4ef274e5ec_1717345646.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345649.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/0ec/952bc6b3e46f35d1bd309ab1be8110ec/952bc6b3e46f35d1bd309ab1be8110ec_1717345649.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345649.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/0ec/952bc6b3e46f35d1bd309ab1be8110ec/952bc6b3e46f35d1bd309ab1be8110ec_1717345649.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345652.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/85f/97bcc36d5980788ad59b7a128977685f/97bcc36d5980788ad59b7a128977685f_1717345652.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345652.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/85f/97bcc36d5980788ad59b7a128977685f/97bcc36d5980788ad59b7a128977685f_1717345652.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75_1717345655.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75_1717345655.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345658.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e_1717345658.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345658.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e_1717345658.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345661.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/2f2/19f0e6c5a633313514d8f875a316f2f2/19f0e6c5a633313514d8f875a316f2f2_1717345661.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345661.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/2f2/19f0e6c5a633313514d8f875a316f2f2/19f0e6c5a633313514d8f875a316f2f2_1717345661.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345664.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/f05/d249758a15fbe2e8fd594cf7e9bdbf05/d249758a15fbe2e8fd594cf7e9bdbf05_1717345664.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345664.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/f05/d249758a15fbe2e8fd594cf7e9bdbf05/d249758a15fbe2e8fd594cf7e9bdbf05_1717345664.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44_1717345667.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44_1717345667.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345670.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/02e/f71202e3621e56629d00f8162fcf502e/f71202e3621e56629d00f8162fcf502e_1717345670.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345670.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/02e/f71202e3621e56629d00f8162fcf502e/f71202e3621e56629d00f8162fcf502e_1717345670.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345673.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/856/c575241d8bae68c9ee8bb18941216856/c575241d8bae68c9ee8bb18941216856_1717345673.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345673.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/856/c575241d8bae68c9ee8bb18941216856/c575241d8bae68c9ee8bb18941216856_1717345673.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345676.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/049/874b04a3c3da3338299b4b4c488db049/874b04a3c3da3338299b4b4c488db049_1717345676.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345676.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/049/874b04a3c3da3338299b4b4c488db049/874b04a3c3da3338299b4b4c488db049_1717345676.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c_1717345679.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c_1717345679.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345682.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d38/bbfff7c5348cce1d1da80cd1e7f78d38/bbfff7c5348cce1d1da80cd1e7f78d38_1717345682.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345682.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d38/bbfff7c5348cce1d1da80cd1e7f78d38/bbfff7c5348cce1d1da80cd1e7f78d38_1717345682.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345685.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e2c/beaa7d16cc9ea04b699d6b4dd546be2c/beaa7d16cc9ea04b699d6b4dd546be2c_1717345685.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345685.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e2c/beaa7d16cc9ea04b699d6b4dd546be2c/beaa7d16cc9ea04b699d6b4dd546be2c_1717345685.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345688.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/76b/9eb70852c7694efd8dadaeb2f927076b/9eb70852c7694efd8dadaeb2f927076b_1717345688.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345688.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/76b/9eb70852c7694efd8dadaeb2f927076b/9eb70852c7694efd8dadaeb2f927076b_1717345688.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7_1717345691.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7_1717345691.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345694.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/3bd/1d1c1cefb960133235568057d5fd83bd/1d1c1cefb960133235568057d5fd83bd_1717345694.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345694.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/3bd/1d1c1cefb960133235568057d5fd83bd/1d1c1cefb960133235568057d5fd83bd_1717345694.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345697.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e71/adb7ffe7789169f6548f3c45ada89e71/adb7ffe7789169f6548f3c45ada89e71_1717345697.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345697.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e71/adb7ffe7789169f6548f3c45ada89e71/adb7ffe7789169f6548f3c45ada89e71_1717345697.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345700.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/852/d52130748b05ceaf9bd055b98b90a852/d52130748b05ceaf9bd055b98b90a852_1717345700.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345700.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/852/d52130748b05ceaf9bd055b98b90a852/d52130748b05ceaf9bd055b98b90a852_1717345700.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84_1717345703.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84_1717345703.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345706.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/0cf/4c0a0860a244b1f8047c3e72eba490cf/4c0a0860a244b1f8047c3e72eba490cf_1717345706.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345706.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/0cf/4c0a0860a244b1f8047c3e72eba490cf/4c0a0860a244b1f8047c3e72eba490cf_1717345706.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345709.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/4c6/84f81fdf3d261d8966658c6ffcac64c6/84f81fdf3d261d8966658c6ffcac64c6_1717345709.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345709.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/4c6/84f81fdf3d261d8966658c6ffcac64c6/84f81fdf3d261d8966658c6ffcac64c6_1717345709.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345712.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/87f/eb8f63b625d69ffd40bd998015cc887f/eb8f63b625d69ffd40bd998015cc887f_1717345712.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345712.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/87f/eb8f63b625d69ffd40bd998015cc887f/eb8f63b625d69ffd40bd998015cc887f_1717345712.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3_1717345715.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3_1717345715.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345718.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/944/569fe0393572811b20d45274ba51f944/569fe0393572811b20d45274ba51f944_1717345718.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345718.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/944/569fe0393572811b20d45274ba51f944/569fe0393572811b20d45274ba51f944_1717345718.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345721.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ec7/42d50d280fbd92825b081058e6f33ec7/42d50d280fbd92825b081058e6f33ec7_1717345721.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345721.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ec7/42d50d280fbd92825b081058e6f33ec7/42d50d280fbd92825b081058e6f33ec7_1717345721.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345724.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d74/9f8654f5660a088797f89446bd1ded74/9f8654f5660a088797f89446bd1ded74_1717345724.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345724.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/d74/9f8654f5660a088797f89446bd1ded74/9f8654f5660a088797f89446bd1ded74_1717345724.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df_1717345727.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df_1717345727.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345730.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/933/2eded075ed2a94e25c7b5d81d0620933/2eded075ed2a94e25c7b5d81d0620933_1717345730.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345730.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/933/2eded075ed2a94e25c7b5d81d0620933/2eded075ed2a94e25c7b5d81d0620933_1717345730.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345733.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/c55/556bad5bedc9f0705c223a74aef3fc55/556bad5bedc9f0705c223a74aef3fc55_1717345733.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345733.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/c55/556bad5bedc9f0705c223a74aef3fc55/556bad5bedc9f0705c223a74aef3fc55_1717345733.00000.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding ... ok test INFO: - - - [02/Jun/2024:16:25:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0138 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0015 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0130 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0125 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0122 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:50 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0012 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:50 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0132 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0144 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0063 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0129 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0108 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0031 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0126 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0171 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0035 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0133 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0002 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0113 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0002 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0137 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0008 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0006 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0128 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:51 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestContainerController.test_weird_content_types test.unit.container.test_server.TestContainerController.test_weird_content_types ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5201c595248d4ad1b7cd7-00665c9d02) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx2dd705f06bff4246af600-00665c9d04) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx8f9554319bf244e8a883c-00665c9d04) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx92626ada40a848c3b2c34-00665c9d04) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txc3adf3658302489194d86-00665c9d04) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout ... ok test INFO: - - - [02/Jun/2024:16:25:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0253 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:51 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0198 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0112 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:52 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0148 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:52 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0125 "-" 3542017 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0230 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0340 "-" 3542017 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3542017 0 test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:52 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0009 "-" 3542017 - test INFO: - - - [02/Jun/2024:16:25:52 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0126 "-" 3542017 0 proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx80e0fea8c8d8480c9b084-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: ValueError("upper ('a') must be greater than or equal to lower ('z')") (txn: tx5f0bf86fbaa04a58a6783-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: KeyError('name') (txn: txb32af46f99b84af0bc504-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type None (txn: tx669baba7f78b42c09c767-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: ValueError('not a list') (txn: txa935aca4ca6f438bbd076-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Failed to get container listing from /v1/a/c/o: 404 (txn: txfed2e2a88c1d4e29b6717-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type 'object' (txn: tx620a0aa9f36340d989fb2-00665c9d14) test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_server.TestProxyServer.test_check_response_503 ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from handoff (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: Object returning 503 for [] (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Ignoring 404 from primary (txn: txfe8338461b514630b1cbf-00665c9d04) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: txf366c04723d8463c9e107-00665c9d0f) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: txf366c04723d8463c9e107-00665c9d0f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 0, Lengths: [490, 490, 490, 490, 490, 490, 490, 490, 490, 490]: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx7ed014cf90274ccab8c46-00665c9d0f) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpr7e6caxm/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpr7e6caxm/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpr7e6caxm/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpr7e6caxm/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0124 "-" 3542017 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0088 "-" 3542017 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0116 "-" 3542017 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0113 "-" 3542017 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx35327d86f7874d2693f2d-00665c9d15" "proxy-server 3542017" 0.0017 "-" 3542017 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:46321/sdb1/1/a/c" "tx35327d86f7874d2693f2d-00665c9d15" "container-server 3542017" 0.0020 "-" 3542017 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:58423/sda1/1/a/c" "tx35327d86f7874d2693f2d-00665c9d15" "container-server 3542017" 0.0015 "-" 3542017 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx35327d86f7874d2693f2d-00665c9d15" "proxy-server 3542017" 0.0202 "-" 3542017 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx35327d86f7874d2693f2d-00665c9d15" "proxy-server 3542017" 0.0347 "-" 3542017 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/57 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx35327d86f7874d2693f2d-00665c9d15 - 0.0454 - - 1717345557.155294418 1717345557.200698614 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txe12eb31a91214ec09dbbc-00665c9d15" "proxy-server 3542017" 0.0020 "-" 3542017 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:58423/sda1/1/a1/c1" "txe12eb31a91214ec09dbbc-00665c9d15" "container-server 3542017" 0.0014 "-" 3542017 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:46321/sdb1/1/a1/c1" "txe12eb31a91214ec09dbbc-00665c9d15" "container-server 3542017" 0.0016 "-" 3542017 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe12eb31a91214ec09dbbc-00665c9d15" "proxy-server 3542017" 0.0222 "-" 3542017 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe12eb31a91214ec09dbbc-00665c9d15" "proxy-server 3542017" 0.0358 "-" 3542017 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/57 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txe12eb31a91214ec09dbbc-00665c9d15 - 0.0457 - - 1717345557.202416658 1717345557.248098850 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3df21dd13c45439ea4454-00665c9d15" "proxy-server 3542017" 0.0080 "-" 3542017 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:58423/sda1/2/a/c1" "tx3df21dd13c45439ea4454-00665c9d15" "container-server 3542017" 0.0016 "-" 3542017 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:46321/sdb1/2/a/c1" "tx3df21dd13c45439ea4454-00665c9d15" "container-server 3542017" 0.0125 "-" 3542017 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx3df21dd13c45439ea4454-00665c9d15" "proxy-server 3542017" 0.0310 "-" 3542017 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx3df21dd13c45439ea4454-00665c9d15" "proxy-server 3542017" 0.0434 "-" 3542017 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/57 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx3df21dd13c45439ea4454-00665c9d15 - 0.0602 - - 1717345557.249816179 1717345557.310048580 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx56d11293d3c8409d85bf1-00665c9d15" "proxy-server 3542017" 0.0099 "-" 3542017 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:46321/sdb1/1/a/c2" "tx56d11293d3c8409d85bf1-00665c9d15" "container-server 3542017" 0.0096 "-" 3542017 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:58423/sda1/1/a/c2" "tx56d11293d3c8409d85bf1-00665c9d15" "container-server 3542017" 0.0015 "-" 3542017 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx56d11293d3c8409d85bf1-00665c9d15" "proxy-server 3542017" 0.0277 "-" 3542017 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:25:57 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx56d11293d3c8409d85bf1-00665c9d15" "proxy-server 3542017" 0.0406 "-" 3542017 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/25/57 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx56d11293d3c8409d85bf1-00665c9d15 - 0.0594 - - 1717345557.312199354 1717345557.371604204 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test.unit.proxy.test_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_server.TestProxyServer.test_denied_host_header ... ok test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_server.TestProxyServer.test_error_occurred test.unit.proxy.test_server.TestProxyServer.test_error_occurred ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345736.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ecf/f54f2025ea7d4dd2fc3e41c7acbabecf/f54f2025ea7d4dd2fc3e41c7acbabecf_1717345736.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345736.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/ecf/f54f2025ea7d4dd2fc3e41c7acbabecf/f54f2025ea7d4dd2fc3e41c7acbabecf_1717345736.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12_1717345739.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12_1717345739.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345742.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/7e4/6f73bd8e4a1bc60fd19b8871188957e4/6f73bd8e4a1bc60fd19b8871188957e4_1717345742.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345742.00000-0, db: /tmp/tmplcksnx4d/sda/containers/0/7e4/6f73bd8e4a1bc60fd19b8871188957e4/6f73bd8e4a1bc60fd19b8871188957e4_1717345742.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345559.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/0d7/e1c824d602f4787335495058f2e670d7/e1c824d602f4787335495058f2e670d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345563.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f72/254526a252cfc49e949d260c3605bf72/254526a252cfc49e949d260c3605bf72.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345563.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f72/254526a252cfc49e949d260c3605bf72/254526a252cfc49e949d260c3605bf72.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345567.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/45c/a214d05181bf3a58939eab98aaf2d45c/a214d05181bf3a58939eab98aaf2d45c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345567.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/45c/a214d05181bf3a58939eab98aaf2d45c/a214d05181bf3a58939eab98aaf2d45c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345575.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/8b0/f8ae096f335e76e81d948650f16eb8b0/f8ae096f335e76e81d948650f16eb8b0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345575.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/8b0/f8ae096f335e76e81d948650f16eb8b0/f8ae096f335e76e81d948650f16eb8b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345579.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345579.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345587.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345587.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345591.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/494/736bc72b1b2f43d54260fe0ac1ced494/736bc72b1b2f43d54260fe0ac1ced494.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345591.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/494/736bc72b1b2f43d54260fe0ac1ced494/736bc72b1b2f43d54260fe0ac1ced494.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345603.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a45/98fc46327bc3d858f7b8d63d7ab08a45/98fc46327bc3d858f7b8d63d7ab08a45.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345603.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a45/98fc46327bc3d858f7b8d63d7ab08a45/98fc46327bc3d858f7b8d63d7ab08a45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345611.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345611.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345615.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f1e/d2df3b46c47d0a6c39116672a5fe8f1e/d2df3b46c47d0a6c39116672a5fe8f1e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345615.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f1e/d2df3b46c47d0a6c39116672a5fe8f1e/d2df3b46c47d0a6c39116672a5fe8f1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345627.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/142/427fb0be2423394f2109eb3f824ed142/427fb0be2423394f2109eb3f824ed142.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345627.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/142/427fb0be2423394f2109eb3f824ed142/427fb0be2423394f2109eb3f824ed142.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345639.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f62/8474b27a3f16620032180bcd8cceff62/8474b27a3f16620032180bcd8cceff62.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345639.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f62/8474b27a3f16620032180bcd8cceff62/8474b27a3f16620032180bcd8cceff62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345647.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345647.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345651.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/648/fbfc5e169925794f41d80b045b5bb648/fbfc5e169925794f41d80b045b5bb648.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345651.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/648/fbfc5e169925794f41d80b045b5bb648/fbfc5e169925794f41d80b045b5bb648.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345663.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a56/86591c8dc136221e54b9f1ec5334ea56/86591c8dc136221e54b9f1ec5334ea56.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345663.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a56/86591c8dc136221e54b9f1ec5334ea56/86591c8dc136221e54b9f1ec5334ea56.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345671.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345671.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345675.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b39/ade3a336b3114f0025b8f6ca4fa05b39/ade3a336b3114f0025b8f6ca4fa05b39.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345675.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b39/ade3a336b3114f0025b8f6ca4fa05b39/ade3a336b3114f0025b8f6ca4fa05b39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345683.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345683.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345687.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f81/75edaf0cd030162232217d2fef599f81/75edaf0cd030162232217d2fef599f81.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345687.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f81/75edaf0cd030162232217d2fef599f81/75edaf0cd030162232217d2fef599f81.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345699.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/1be/701a49feead126ddccd7e937c1acd1be/701a49feead126ddccd7e937c1acd1be.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345699.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/1be/701a49feead126ddccd7e937c1acd1be/701a49feead126ddccd7e937c1acd1be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345707.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345707.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345711.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/657/3e8ef64022f6dfedbe89412c42002657/3e8ef64022f6dfedbe89412c42002657.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345711.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/657/3e8ef64022f6dfedbe89412c42002657/3e8ef64022f6dfedbe89412c42002657.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345723.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/07c/2fde1ec26ef1340c593460c15d84707c/2fde1ec26ef1340c593460c15d84707c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345723.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/07c/2fde1ec26ef1340c593460c15d84707c/2fde1ec26ef1340c593460c15d84707c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345735.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/85c/6cc1446b3d3747a132dfe6e37382285c/6cc1446b3d3747a132dfe6e37382285c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345735.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/85c/6cc1446b3d3747a132dfe6e37382285c/6cc1446b3d3747a132dfe6e37382285c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345743.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345743.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345747.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/bf8/64e7e0002088f0e061b43e4dd89b3bf8/64e7e0002088f0e061b43e4dd89b3bf8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345747.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/bf8/64e7e0002088f0e061b43e4dd89b3bf8/64e7e0002088f0e061b43e4dd89b3bf8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345751.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345751.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345755.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/db9/d57ac6f55bb096b7b53bce53eb985db9/d57ac6f55bb096b7b53bce53eb985db9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345755.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/db9/d57ac6f55bb096b7b53bce53eb985db9/d57ac6f55bb096b7b53bce53eb985db9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345759.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/341/d9772fe63a4c24f892e39f1a7c29a341/d9772fe63a4c24f892e39f1a7c29a341.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345759.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/341/d9772fe63a4c24f892e39f1a7c29a341/d9772fe63a4c24f892e39f1a7c29a341.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345763.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/eba/c8656bf9dd67ae39a5ff0e8b0ed5beba/c8656bf9dd67ae39a5ff0e8b0ed5beba.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345763.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/eba/c8656bf9dd67ae39a5ff0e8b0ed5beba/c8656bf9dd67ae39a5ff0e8b0ed5beba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345767.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/82a/af7a7d95593bebe38159c18cd239482a/af7a7d95593bebe38159c18cd239482a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345767.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/82a/af7a7d95593bebe38159c18cd239482a/af7a7d95593bebe38159c18cd239482a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345771.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f13/13dcd0d823693a4a26ba602b28f31f13/13dcd0d823693a4a26ba602b28f31f13.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345771.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/f13/13dcd0d823693a4a26ba602b28f31f13/13dcd0d823693a4a26ba602b28f31f13.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345775.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a51/0357613d8a0b68cbecaaf47a2ea38a51/0357613d8a0b68cbecaaf47a2ea38a51.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345775.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a51/0357613d8a0b68cbecaaf47a2ea38a51/0357613d8a0b68cbecaaf47a2ea38a51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345779.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/3fd/7cde512a498cc4f6e7e54bdd9fce83fd/7cde512a498cc4f6e7e54bdd9fce83fd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345779.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/3fd/7cde512a498cc4f6e7e54bdd9fce83fd/7cde512a498cc4f6e7e54bdd9fce83fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345783.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ebd/6b695f37531b4d4f75d96dac331f6ebd/6b695f37531b4d4f75d96dac331f6ebd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345783.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ebd/6b695f37531b4d4f75d96dac331f6ebd/6b695f37531b4d4f75d96dac331f6ebd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345787.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ee6/ae9d9a47225312c25d2fa1f7d8b2eee6/ae9d9a47225312c25d2fa1f7d8b2eee6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345787.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/ee6/ae9d9a47225312c25d2fa1f7d8b2eee6/ae9d9a47225312c25d2fa1f7d8b2eee6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345791.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a04/4e8dd23d553a52a0e9b40fb0b2444a04/4e8dd23d553a52a0e9b40fb0b2444a04.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345791.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/a04/4e8dd23d553a52a0e9b40fb0b2444a04/4e8dd23d553a52a0e9b40fb0b2444a04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345795.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/09a/a7d77580e4fd4195087224c157e0a09a/a7d77580e4fd4195087224c157e0a09a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345795.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/09a/a7d77580e4fd4195087224c157e0a09a/a7d77580e4fd4195087224c157e0a09a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345799.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/788/e233b98157246fb2f16763643621d788/e233b98157246fb2f16763643621d788.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345799.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/788/e233b98157246fb2f16763643621d788/e233b98157246fb2f16763643621d788.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345803.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/22f/14ec906a79bed1ec1a02553c81f6022f/14ec906a79bed1ec1a02553c81f6022f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345803.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/22f/14ec906a79bed1ec1a02553c81f6022f/14ec906a79bed1ec1a02553c81f6022f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345807.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/01b/d96b6495f96d1d875f1a325b4345701b/d96b6495f96d1d875f1a325b4345701b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345807.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/01b/d96b6495f96d1d875f1a325b4345701b/d96b6495f96d1d875f1a325b4345701b.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync ... ok proxy-server ERROR: ERROR Unhandled exception in request: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/server.py", line 598, in handle_request return handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 375, in wrapped return func(*a, **kw) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 275, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 263, in GETorHEAD resp = self._get_or_head_response(req, node_iter, partition, policy) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2930, in _get_or_head_response app_iter.kickoff(req, resp) File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1184, in kickoff self.stashed_iter = reiterate(self._real_iter(req, resp.headers)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3749, in reiterate chunk = next(iterator) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4452, in _get_next_item return super(StringAlong, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1470, in _iter_one_range for i, next_seg in enumerate(segment_iter): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1549, in _decode_segments_from_fragments segment = self.policy.pyeclib_driver.decode(fragments) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/ec_iface.py", line 300, in decode return self.ec_lib_reference.decode(fragment_payloads, ranges, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/core.py", line 91, in decode return pyeclib_c.decode(self.handle, _fragment_payloads, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pyeclib.ec_iface.ECInvalidFragmentMetadata: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx7ed014cf90274ccab8c46-00665c9d0f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server ERROR: Object returning 503 for [] (txn: tx67c65e80fb904d6584e75-00665c9d10) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test.unit.proxy.test_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_server.TestProxyServer.test_exception_occurred ... ok test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_info_defaults test.unit.proxy.test_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_server.TestProxyServer.test_node_timing test.unit.proxy.test_server.TestProxyServer.test_node_timing ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx9c0eb3b0d50e4c39b208b-00665c9d16) test INFO: Client disconnected on read of '/a/c' (txn: tx9c0eb3b0d50e4c39b208b-00665c9d16) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: txfd6e13792d02407cb10a3-00665c9d17) test INFO: Client disconnected on read of '/a/c' (txn: txfd6e13792d02407cb10a3-00665c9d17) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: txd01205b39c7544aa93371-00665c9d18) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: txd01205b39c7544aa93371-00665c9d18) test.unit.proxy.test_server.TestProxyServer.test_private_method_request test.unit.proxy.test_server.TestProxyServer.test_private_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Ignoring 404 from primary (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server ERROR: Object returning 503 for [] (txn: txaa471493b4b24fcd8c026-00665c9d10) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server ERROR: Object returning 503 for [] (txn: tx8a31fe9e195345568d160-00665c9d11) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx55874ebf78a841f8bf2ca-00665c9d11) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from primary (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx267764fdc4e9451b81df7-00665c9d11) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx0ab3e10d3d864a24894d1-00665c9d12) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx9bf2f14871c141eba04ca-00665c9d12) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body ... ok test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: txd01205b39c7544aa93371-00665c9d18) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: txd01205b39c7544aa93371-00665c9d18) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix ... ok test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_server.TestProxyServer.test_valid_api_version ... ok test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:25:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded test ERROR: Problem finding shard upper in '/tmp/tmp1k1cqd4y/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp1k1cqd4y/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmphia32zhc/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmphia32zhc/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add test.unit.obj.test_replicator.TestReplicatorStats.test_add ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers ... ok test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb819ef176361440195844-00665c9d13) proxy-server ERROR: Object returning 503 for [] (txn: txb819ef176361440195844-00665c9d13) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: tx46246682a611427987a9f-00665c9d13) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.11:1011/sdl (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46246682a611427987a9f-00665c9d13) proxy-server ERROR: Exception fetching fragments for '/a/c/o': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1496, in put_fragments_in_queue for fragment in frag_iter: File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2518, in _iter_bytes_from_response_part six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2490, in _iter_bytes_from_response_part chunk = part_file.read(self.app.object_chunk_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1133, in read raise ShortReadError( swift.common.exceptions.ShortReadError: Too few bytes; read 415, expecting 490 (txn: tx46246682a611427987a9f-00665c9d13) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx46246682a611427987a9f-00665c9d13) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.4:1004/sde (txn: txe5af0707f62e43f9938bf-00665c9d14) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345811.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b5d/a0ca57477be8e33821cf5fd06a32cb5d/a0ca57477be8e33821cf5fd06a32cb5d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345811.00000-0, db: /tmp/tmp9cl9o070/sda/containers/0/b5d/a0ca57477be8e33821cf5fd06a32cb5d/a0ca57477be8e33821cf5fd06a32cb5d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345562.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cc9/61c04ca51366ec21ff7343440ecaecc9/61c04ca51366ec21ff7343440ecaecc9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345562.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cc9/61c04ca51366ec21ff7343440ecaecc9/61c04ca51366ec21ff7343440ecaecc9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345565.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d0e/ea72c188c4224ac849a2d7fe169bad0e/ea72c188c4224ac849a2d7fe169bad0e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345565.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d0e/ea72c188c4224ac849a2d7fe169bad0e/ea72c188c4224ac849a2d7fe169bad0e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345568.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/fe2/656d1904a6f8446eda6710621abcbfe2/656d1904a6f8446eda6710621abcbfe2.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345568.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/fe2/656d1904a6f8446eda6710621abcbfe2/656d1904a6f8446eda6710621abcbfe2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345574.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/05d/2dcd6cd427dae4950102d46f401fd05d/2dcd6cd427dae4950102d46f401fd05d.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345574.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/05d/2dcd6cd427dae4950102d46f401fd05d/2dcd6cd427dae4950102d46f401fd05d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345577.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e52/e4a7114b1d757b32a790161a9a483e52/e4a7114b1d757b32a790161a9a483e52.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345577.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e52/e4a7114b1d757b32a790161a9a483e52/e4a7114b1d757b32a790161a9a483e52.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345598.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ab4/88fc9d55e59e3659795aa905ca0d5ab4/88fc9d55e59e3659795aa905ca0d5ab4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345598.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ab4/88fc9d55e59e3659795aa905ca0d5ab4/88fc9d55e59e3659795aa905ca0d5ab4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345601.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/3f6/cce3c78ea41024a78014a657c5ad13f6/cce3c78ea41024a78014a657c5ad13f6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345601.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/3f6/cce3c78ea41024a78014a657c5ad13f6/cce3c78ea41024a78014a657c5ad13f6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345604.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e80/977857deb4d7009d0e50754496596e80/977857deb4d7009d0e50754496596e80.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345604.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e80/977857deb4d7009d0e50754496596e80/977857deb4d7009d0e50754496596e80.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345610.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/a5f/e1cf616facba3f6f034939a512cc0a5f/e1cf616facba3f6f034939a512cc0a5f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345610.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/a5f/e1cf616facba3f6f034939a512cc0a5f/e1cf616facba3f6f034939a512cc0a5f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345613.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/247/214ea89a94368e76f96d8ffab7653247/214ea89a94368e76f96d8ffab7653247.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345613.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/247/214ea89a94368e76f96d8ffab7653247/214ea89a94368e76f96d8ffab7653247.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345616.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/c69/e485e5b3ca2786d2eb162a460c4dac69/e485e5b3ca2786d2eb162a460c4dac69.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345616.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/c69/e485e5b3ca2786d2eb162a460c4dac69/e485e5b3ca2786d2eb162a460c4dac69.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345622.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/be9/b3037b17f4b2fccbf125c19db43eebe9/b3037b17f4b2fccbf125c19db43eebe9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345622.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/be9/b3037b17f4b2fccbf125c19db43eebe9/b3037b17f4b2fccbf125c19db43eebe9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345625.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/371/6106ef7c90a2c4416c0b57ddfa173371/6106ef7c90a2c4416c0b57ddfa173371.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345625.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/371/6106ef7c90a2c4416c0b57ddfa173371/6106ef7c90a2c4416c0b57ddfa173371.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345628.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/34a/c3f72458cb53f1dfc3892aaefe7b934a/c3f72458cb53f1dfc3892aaefe7b934a.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345628.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/34a/c3f72458cb53f1dfc3892aaefe7b934a/c3f72458cb53f1dfc3892aaefe7b934a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345634.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/612/f7cc4ec9d01b70c34157de2ebbb12612/f7cc4ec9d01b70c34157de2ebbb12612.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345634.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/612/f7cc4ec9d01b70c34157de2ebbb12612/f7cc4ec9d01b70c34157de2ebbb12612.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345637.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d0d/d32d7037cf1168c2cf9efa389f38dd0d/d32d7037cf1168c2cf9efa389f38dd0d.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345637.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d0d/d32d7037cf1168c2cf9efa389f38dd0d/d32d7037cf1168c2cf9efa389f38dd0d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345640.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ea7/0e27dbff000bf481fef60d7129b98ea7/0e27dbff000bf481fef60d7129b98ea7.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345640.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ea7/0e27dbff000bf481fef60d7129b98ea7/0e27dbff000bf481fef60d7129b98ea7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345646.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/5ec/8758d8619071fee95eaaac4ef274e5ec/8758d8619071fee95eaaac4ef274e5ec.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345646.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/5ec/8758d8619071fee95eaaac4ef274e5ec/8758d8619071fee95eaaac4ef274e5ec.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345649.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/0ec/952bc6b3e46f35d1bd309ab1be8110ec/952bc6b3e46f35d1bd309ab1be8110ec.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345649.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/0ec/952bc6b3e46f35d1bd309ab1be8110ec/952bc6b3e46f35d1bd309ab1be8110ec.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345652.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/85f/97bcc36d5980788ad59b7a128977685f/97bcc36d5980788ad59b7a128977685f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345652.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/85f/97bcc36d5980788ad59b7a128977685f/97bcc36d5980788ad59b7a128977685f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345658.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345658.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e/4cc5ac1bb4b431993aea8ee9f79c5f1e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345661.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/2f2/19f0e6c5a633313514d8f875a316f2f2/19f0e6c5a633313514d8f875a316f2f2.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345661.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/2f2/19f0e6c5a633313514d8f875a316f2f2/19f0e6c5a633313514d8f875a316f2f2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345664.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/f05/d249758a15fbe2e8fd594cf7e9bdbf05/d249758a15fbe2e8fd594cf7e9bdbf05.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345664.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/f05/d249758a15fbe2e8fd594cf7e9bdbf05/d249758a15fbe2e8fd594cf7e9bdbf05.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345670.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/02e/f71202e3621e56629d00f8162fcf502e/f71202e3621e56629d00f8162fcf502e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345670.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/02e/f71202e3621e56629d00f8162fcf502e/f71202e3621e56629d00f8162fcf502e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345673.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/856/c575241d8bae68c9ee8bb18941216856/c575241d8bae68c9ee8bb18941216856.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345673.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/856/c575241d8bae68c9ee8bb18941216856/c575241d8bae68c9ee8bb18941216856.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345676.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/049/874b04a3c3da3338299b4b4c488db049/874b04a3c3da3338299b4b4c488db049.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345676.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/049/874b04a3c3da3338299b4b4c488db049/874b04a3c3da3338299b4b4c488db049.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345682.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d38/bbfff7c5348cce1d1da80cd1e7f78d38/bbfff7c5348cce1d1da80cd1e7f78d38.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345682.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d38/bbfff7c5348cce1d1da80cd1e7f78d38/bbfff7c5348cce1d1da80cd1e7f78d38.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345685.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e2c/beaa7d16cc9ea04b699d6b4dd546be2c/beaa7d16cc9ea04b699d6b4dd546be2c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345685.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e2c/beaa7d16cc9ea04b699d6b4dd546be2c/beaa7d16cc9ea04b699d6b4dd546be2c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345688.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/76b/9eb70852c7694efd8dadaeb2f927076b/9eb70852c7694efd8dadaeb2f927076b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345688.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/76b/9eb70852c7694efd8dadaeb2f927076b/9eb70852c7694efd8dadaeb2f927076b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345694.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/3bd/1d1c1cefb960133235568057d5fd83bd/1d1c1cefb960133235568057d5fd83bd.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345694.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/3bd/1d1c1cefb960133235568057d5fd83bd/1d1c1cefb960133235568057d5fd83bd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345697.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e71/adb7ffe7789169f6548f3c45ada89e71/adb7ffe7789169f6548f3c45ada89e71.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345697.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/e71/adb7ffe7789169f6548f3c45ada89e71/adb7ffe7789169f6548f3c45ada89e71.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345700.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/852/d52130748b05ceaf9bd055b98b90a852/d52130748b05ceaf9bd055b98b90a852.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345700.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/852/d52130748b05ceaf9bd055b98b90a852/d52130748b05ceaf9bd055b98b90a852.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking ... ok test ERROR: Failed to unlink '/tmp/tmpx1oxalv0/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpx1oxalv0/containers/part/suffix/hash/container_1717345564.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpx1oxalv0/containers/part/suffix/hash/container_1717345564.00000.db': ['/tmp/tmpx1oxalv0/containers/part/suffix/hash/container.db', '/tmp/tmpx1oxalv0/containers/part/suffix/hash/container_1717345565.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpx1oxalv0/containers/part/suffix/hash/container_1717345565.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx3ac76388c0e94ae58a28e-00665c9d1d) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx3ac76388c0e94ae58a28e-00665c9d1d) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx3ac76388c0e94ae58a28e-00665c9d1d) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: tx3ac76388c0e94ae58a28e-00665c9d1d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp358agctu/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp358agctu/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp358agctu/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp358agctu/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0114 "-" 3542015 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0096 "-" 3542015 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0115 "-" 3542015 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0094 "-" 3542015 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx24020f18b8fa4b1bb1a9e-00665c9d1d" "proxy-server 3542015" 0.0017 "-" 3542015 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:48949/sda1/1/a/c" "tx24020f18b8fa4b1bb1a9e-00665c9d1d" "container-server 3542015" 0.0016 "-" 3542015 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:54289/sdb1/1/a/c" "tx24020f18b8fa4b1bb1a9e-00665c9d1d" "container-server 3542015" 0.0014 "-" 3542015 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx24020f18b8fa4b1bb1a9e-00665c9d1d" "proxy-server 3542015" 0.0358 "-" 3542015 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx24020f18b8fa4b1bb1a9e-00665c9d1d" "proxy-server 3542015" 0.0203 "-" 3542015 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/05 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx24020f18b8fa4b1bb1a9e-00665c9d1d - 0.0482 - - 1717345565.157759666 1717345565.205951691 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx91931da222c44e5fbf329-00665c9d1d" "proxy-server 3542015" 0.0017 "-" 3542015 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:48949/sda1/1/a1/c1" "tx91931da222c44e5fbf329-00665c9d1d" "container-server 3542015" 0.0016 "-" 3542015 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:54289/sdb1/1/a1/c1" "tx91931da222c44e5fbf329-00665c9d1d" "container-server 3542015" 0.0023 "-" 3542015 0 test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.test_locale.test_locale.TestTranslations.test_translations test.unit.test_locale.test_locale.TestTranslations.test_translations ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.7:1007/sdh (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server ERROR: ERROR Unhandled exception in request: ChunkReadTimeout (0.01s after 0.02s) (txn: tx46708f9c87f34e52b9a30-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.11:1011/sdl (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: tx3c4f416092f1454c80f71-00665c9d14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx45078f8359b04d20b1221-00665c9d15) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from primary (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries ... ok cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx91931da222c44e5fbf329-00665c9d1d" "proxy-server 3542015" 0.0320 "-" 3542015 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx91931da222c44e5fbf329-00665c9d1d" "proxy-server 3542015" 0.0205 "-" 3542015 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/05 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx91931da222c44e5fbf329-00665c9d1d - 0.0427 - - 1717345565.207843781 1717345565.250567675 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx690843b357af470b943f9-00665c9d1d" "proxy-server 3542015" 0.0067 "-" 3542015 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:48949/sda1/2/a/c1" "tx690843b357af470b943f9-00665c9d1d" "container-server 3542015" 0.0094 "-" 3542015 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:54289/sdb1/2/a/c1" "tx690843b357af470b943f9-00665c9d1d" "container-server 3542015" 0.0014 "-" 3542015 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx690843b357af470b943f9-00665c9d1d" "proxy-server 3542015" 0.0601 "-" 3542015 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx690843b357af470b943f9-00665c9d1d" "proxy-server 3542015" 0.0448 "-" 3542015 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/05 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx690843b357af470b943f9-00665c9d1d - 0.0765 - - 1717345565.252425671 1717345565.328877211 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf7e2e0b4f11342c89b919-00665c9d1d" "proxy-server 3542015" 0.0103 "-" 3542015 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:54289/sdb1/1/a/c2" "txf7e2e0b4f11342c89b919-00665c9d1d" "container-server 3542015" 0.0083 "-" 3542015 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:48949/sda1/1/a/c2" "txf7e2e0b4f11342c89b919-00665c9d1d" "container-server 3542015" 0.0015 "-" 3542015 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf7e2e0b4f11342c89b919-00665c9d1d" "proxy-server 3542015" 0.0387 "-" 3542015 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:05 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf7e2e0b4f11342c89b919-00665c9d1d" "proxy-server 3542015" 0.0283 "-" 3542015 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/05 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txf7e2e0b4f11342c89b919-00665c9d1d - 0.0585 - - 1717345565.330636024 1717345565.389153481 2 proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc317800149b74c68a4c20-00665c9d15) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9f6c90f4d93f4b6bb7c59-00665c9d18) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txe26db7ccf0e64191ba022-00665c9d18) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8a87643ee3b44d939edb4-00665c9d18) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to GET /a/c/o From Object Server 10.0.0.13:1013/sdn (txn: txe2a22d9e07464a95bb506-00665c9d18) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2fa26b69539d4ff1b6f3e-00665c9d19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.7:1007/sdh, error: ERROR Insufficient Storage (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.8:1008/sdi, error: ERROR Insufficient Storage (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.6:1006/sdg, error: ERROR Insufficient Storage (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.4:1004/sde, error: ERROR Insufficient Storage (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server ERROR: Object returning 503 for [507, 507, 507, 507, 507] (txn: tx5c8edcf5d3674dc9b6d29-00665c9d19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: txf97bbcc747c44cc49dd4b-00665c9d19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx226903a16ba84b59ba804-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple ... ok proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx14b8a9b0766d4921bcc6a-00665c9d19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9c1e3c3ebe36460592686-00665c9d1d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9bc1b72fdecd436a86805-00665c9d1e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345706.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/0cf/4c0a0860a244b1f8047c3e72eba490cf/4c0a0860a244b1f8047c3e72eba490cf.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345706.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/0cf/4c0a0860a244b1f8047c3e72eba490cf/4c0a0860a244b1f8047c3e72eba490cf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345709.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/4c6/84f81fdf3d261d8966658c6ffcac64c6/84f81fdf3d261d8966658c6ffcac64c6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345709.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/4c6/84f81fdf3d261d8966658c6ffcac64c6/84f81fdf3d261d8966658c6ffcac64c6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345712.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/87f/eb8f63b625d69ffd40bd998015cc887f/eb8f63b625d69ffd40bd998015cc887f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345712.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/87f/eb8f63b625d69ffd40bd998015cc887f/eb8f63b625d69ffd40bd998015cc887f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345718.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/944/569fe0393572811b20d45274ba51f944/569fe0393572811b20d45274ba51f944.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345718.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/944/569fe0393572811b20d45274ba51f944/569fe0393572811b20d45274ba51f944.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345721.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ec7/42d50d280fbd92825b081058e6f33ec7/42d50d280fbd92825b081058e6f33ec7.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345721.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/ec7/42d50d280fbd92825b081058e6f33ec7/42d50d280fbd92825b081058e6f33ec7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345724.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d74/9f8654f5660a088797f89446bd1ded74/9f8654f5660a088797f89446bd1ded74.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345724.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/d74/9f8654f5660a088797f89446bd1ded74/9f8654f5660a088797f89446bd1ded74.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmpvj245hyz/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345564.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/894/d305918d218fc1abeccb339f50955894/d305918d218fc1abeccb339f50955894.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345569.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/34d/b6795adad4f7702128f8b70a8e55834d/b6795adad4f7702128f8b70a8e55834d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345574.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/05d/2dcd6cd427dae4950102d46f401fd05d/2dcd6cd427dae4950102d46f401fd05d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345579.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345594.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f6c/8ca8248c151ad02fdc3903207cca8f6c/8ca8248c151ad02fdc3903207cca8f6c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345604.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e80/977857deb4d7009d0e50754496596e80/977857deb4d7009d0e50754496596e80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345609.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/da6/0d1a07dd079091eb51d98127e4117da6/0d1a07dd079091eb51d98127e4117da6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345614.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/cf0/788374a945691d4e5292fcb2c28bacf0/788374a945691d4e5292fcb2c28bacf0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345624.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/6ee/2764dcf0d824d4aa77da1e462dae26ee/2764dcf0d824d4aa77da1e462dae26ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345629.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e8d/4d1360234011af8a8d9288535f233e8d/4d1360234011af8a8d9288535f233e8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345634.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/612/f7cc4ec9d01b70c34157de2ebbb12612/f7cc4ec9d01b70c34157de2ebbb12612.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345639.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f62/8474b27a3f16620032180bcd8cceff62/8474b27a3f16620032180bcd8cceff62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345644.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/634/2e0e75e90c4f4e41c73240d181a5d634/2e0e75e90c4f4e41c73240d181a5d634.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345649.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0ec/952bc6b3e46f35d1bd309ab1be8110ec/952bc6b3e46f35d1bd309ab1be8110ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345654.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0f0/4b852f94a46beea11d3283cbc5f9a0f0/4b852f94a46beea11d3283cbc5f9a0f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345664.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f05/d249758a15fbe2e8fd594cf7e9bdbf05/d249758a15fbe2e8fd594cf7e9bdbf05.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345669.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f41/584b6061c173ee9a220615fda29aaf41/584b6061c173ee9a220615fda29aaf41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345674.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/cb8/702c6aa0a659067e36bf1d811c9bbcb8/702c6aa0a659067e36bf1d811c9bbcb8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345684.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c4a/a565be60affb76b8318162dbfda77c4a/a565be60affb76b8318162dbfda77c4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345689.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c0c/8f3905bfc9b21e32752ae32385a61c0c/8f3905bfc9b21e32752ae32385a61c0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345694.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3bd/1d1c1cefb960133235568057d5fd83bd/1d1c1cefb960133235568057d5fd83bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345699.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/1be/701a49feead126ddccd7e937c1acd1be/701a49feead126ddccd7e937c1acd1be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345704.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/647/f1e9e80e5e299c1c57756217e0325647/f1e9e80e5e299c1c57756217e0325647.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345709.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/4c6/84f81fdf3d261d8966658c6ffcac64c6/84f81fdf3d261d8966658c6ffcac64c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345714.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a34/a525cfa4f3077477439f47ea887a1a34/a525cfa4f3077477439f47ea887a1a34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345724.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d74/9f8654f5660a088797f89446bd1ded74/9f8654f5660a088797f89446bd1ded74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345729.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/23a/a7895cbf0c14cc69ec2444e666c8f23a/a7895cbf0c14cc69ec2444e666c8f23a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345734.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/266/dae58062b96935b774e503823f769266/dae58062b96935b774e503823f769266.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345744.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d11/9f27f86e05f906e2f92606a87c4d1d11/9f27f86e05f906e2f92606a87c4d1d11.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345749.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/336/8fc29c52c15d759c58c893fc7ad5b336/8fc29c52c15d759c58c893fc7ad5b336.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345754.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/bd7/c79c85a87e4d6013856fba1ab0937bd7/c79c85a87e4d6013856fba1ab0937bd7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345759.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/341/d9772fe63a4c24f892e39f1a7c29a341/d9772fe63a4c24f892e39f1a7c29a341.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345764.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/6d2/24aae638b782b2f7054abd8531e0b6d2/24aae638b782b2f7054abd8531e0b6d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345769.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/873/88f912cafd5782dfda29b15df3309873/88f912cafd5782dfda29b15df3309873.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345774.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3f3/3b9a5ab3076c0c48cfe95a7bbb69a3f3/3b9a5ab3076c0c48cfe95a7bbb69a3f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345779.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3fd/7cde512a498cc4f6e7e54bdd9fce83fd/7cde512a498cc4f6e7e54bdd9fce83fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345784.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b3c/b8e5b79491655984bde9523f1f8fdb3c/b8e5b79491655984bde9523f1f8fdb3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345789.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3cd/db7b5d359a1a77fc233cdc5d90e443cd/db7b5d359a1a77fc233cdc5d90e443cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345794.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/cf9/c20b0dd0eda11a64c0be99348f986cf9/c20b0dd0eda11a64c0be99348f986cf9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345799.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/788/e233b98157246fb2f16763643621d788/e233b98157246fb2f16763643621d788.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345804.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/63d/8151b8fed296fcde72f55acab4dd463d/8151b8fed296fcde72f55acab4dd463d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345809.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/bf6/a1fa7ce7310a18a2e4b727f7d8ad6bf6/a1fa7ce7310a18a2e4b727f7d8ad6bf6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345814.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/483/f343c96eaa4db4a6515de8f630959483/f343c96eaa4db4a6515de8f630959483.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345819.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f70/0acab4cfaeb4df90edc3dbff69a60f70/0acab4cfaeb4df90edc3dbff69a60f70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345824.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a33/6c82a641217f2be50ef33aa0d326da33/6c82a641217f2be50ef33aa0d326da33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345829.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7d8/8ba98416400f3ae3bbb9f5637c6a57d8/8ba98416400f3ae3bbb9f5637c6a57d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345834.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/8aa/20e511ddde4d634767d9567586ae68aa/20e511ddde4d634767d9567586ae68aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345839.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/5e3/aa48eb23cb4abb86c0917980853135e3/aa48eb23cb4abb86c0917980853135e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345844.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9ca/2983df72e4e269e3b94e72be5b0429ca/2983df72e4e269e3b94e72be5b0429ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345849.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/52c/c984c9a3324481175e06c8b5d777552c/c984c9a3324481175e06c8b5d777552c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345854.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/044/67376554e8bf4c54e79f65848e0c7044/67376554e8bf4c54e79f65848e0c7044.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345859.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/82a/5c7d54d86457214ab2b267672a27a82a/5c7d54d86457214ab2b267672a27a82a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345864.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/fdb/b354de0c0199cd18305adb6350fa5fdb/b354de0c0199cd18305adb6350fa5fdb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345869.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/99c/31380e3c31d5872649889a425a41399c/31380e3c31d5872649889a425a41399c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345874.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/24e/9922e3e3782078ba3177f343cebea24e/9922e3e3782078ba3177f343cebea24e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345879.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/277/b45fb34ccf3414c4389ce686a2cba277/b45fb34ccf3414c4389ce686a2cba277.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345884.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/92a/66e219a1b6e02d4ca1e7cfe50b65192a/66e219a1b6e02d4ca1e7cfe50b65192a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345889.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9cc/625074f4da62535c56599869829ba9cc/625074f4da62535c56599869829ba9cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345894.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f9c/1df73cb504328c9b033c3e3ad9f66f9c/1df73cb504328c9b033c3e3ad9f66f9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345904.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/cd4/a222c7372b2d133af8e22ccf38a71cd4/a222c7372b2d133af8e22ccf38a71cd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345909.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/417/b9eb057ae1e3b48e96e613a12ad94417/b9eb057ae1e3b48e96e613a12ad94417.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345924.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/dbc/5eb9fcf1d1efac0a71f8864594814dbc/5eb9fcf1d1efac0a71f8864594814dbc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345929.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b65/e499cb8202d58e7ad5f94e5e6fe38b65/e499cb8202d58e7ad5f94e5e6fe38b65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345939.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d69/82df5f6d5f930f5efcd5c422ac1bcd69/82df5f6d5f930f5efcd5c422ac1bcd69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345944.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7eb/5c7c29ada09e7f3d71c4282e565b37eb/5c7c29ada09e7f3d71c4282e565b37eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345954.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a73/682f3d8315d3c4aa31c9eb70cc5d1a73/682f3d8315d3c4aa31c9eb70cc5d1a73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345964.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c67/cd176eaa90f4d4ac9133cc9d19a15c67/cd176eaa90f4d4ac9133cc9d19a15c67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345969.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345974.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/469/6cc61909e7a11534cf9facccb2cd6469/6cc61909e7a11534cf9facccb2cd6469.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345984.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/8f4/4311c9db62e11eef65dee503d09048f4/4311c9db62e11eef65dee503d09048f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345989.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/975/40c65bb6fef1f9af0ce0f9a889d51975/40c65bb6fef1f9af0ce0f9a889d51975.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345999.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346004.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e22/5dae92396724bfb4eee10f10de10de22/5dae92396724bfb4eee10f10de10de22.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346014.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a32/e62cfd993cf033cf47f6d26df631ca32/e62cfd993cf033cf47f6d26df631ca32.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346024.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/974/7f371af32892408d278c8d2b4e2ad974/7f371af32892408d278c8d2b4e2ad974.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346029.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c51/014702cf3e6364f1cc17bf7f2d3dac51/014702cf3e6364f1cc17bf7f2d3dac51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346034.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e6b/1e17050f206eb050635baa94217f8e6b/1e17050f206eb050635baa94217f8e6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346044.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d2f/ca1905f392adc798e70bd9a684deed2f/ca1905f392adc798e70bd9a684deed2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346049.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/fa1/4a860ba03606b566205549d0529fafa1/4a860ba03606b566205549d0529fafa1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346054.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a65/827b1676df1f920fb9a53e5a1cae0a65/827b1676df1f920fb9a53e5a1cae0a65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346059.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/56f/9a8b64160e6ae421965eb79c4a6cc56f/9a8b64160e6ae421965eb79c4a6cc56f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346064.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3dc/3f52d56d64559e88e28eb55dd2df43dc/3f52d56d64559e88e28eb55dd2df43dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346069.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/6f3/d75465b42e5dc3b12c47edd8623656f3/d75465b42e5dc3b12c47edd8623656f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346074.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/53b/217ce7a7d09e39777860c6633c30c53b/217ce7a7d09e39777860c6633c30c53b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346079.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/078/6433e58e0348b3f47247aa2b74b0e078/6433e58e0348b3f47247aa2b74b0e078.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346084.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/6d1/9157046c9fe87946853035549dbb46d1/9157046c9fe87946853035549dbb46d1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346089.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/4d4/246c30bc5935967d840225fd570ca4d4/246c30bc5935967d840225fd570ca4d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346094.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/08c/6a2e5f7dcf475cf183f4a9516e0af08c/6a2e5f7dcf475cf183f4a9516e0af08c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346099.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a70/a7c571ebacba5fa57e73a3256a942a70/a7c571ebacba5fa57e73a3256a942a70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346104.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2ec/b74714108d3c09569c58c8d76c7342ec/b74714108d3c09569c58c8d76c7342ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346109.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/952/5570fc45ce32ad71e904ba2564923952/5570fc45ce32ad71e904ba2564923952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346114.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b11/04c52fcff46616dce478b3ff8e7e5b11/04c52fcff46616dce478b3ff8e7e5b11.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346119.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2d7/2128a81fd3c0376e7654966effa7b2d7/2128a81fd3c0376e7654966effa7b2d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346124.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/167/324c357ff9d42ac89e9cb793560e9167/324c357ff9d42ac89e9cb793560e9167.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346129.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/db1/db7683fddb5cf5e5bcb731ae54843db1/db7683fddb5cf5e5bcb731ae54843db1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346134.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/502/32756134775a8993c1c258262cbde502/32756134775a8993c1c258262cbde502.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346139.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/368/3b8b2bde8514da353a6b30929a3a8368/3b8b2bde8514da353a6b30929a3a8368.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346144.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/fb4/7f994b76be14c56697ed8eca9788cfb4/7f994b76be14c56697ed8eca9788cfb4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346149.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/883/bd721bff4f0ca78065850fac18d99883/bd721bff4f0ca78065850fac18d99883.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346154.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/002/346e0e57d1918d2e1822b2019bcb7002/346e0e57d1918d2e1822b2019bcb7002.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346159.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2e0/72f9effd13247b9a8a1bf5b5627fe2e0/72f9effd13247b9a8a1bf5b5627fe2e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346164.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/acb/9ec52493973925b67b6fca4a12667acb/9ec52493973925b67b6fca4a12667acb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346169.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/bd8/42bc7c3a3b03b275f2b739f070fcdbd8/42bc7c3a3b03b275f2b739f070fcdbd8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346174.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ff7/b255d085c3c53ca26177165958e1eff7/b255d085c3c53ca26177165958e1eff7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346179.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/5a8/e8d306be8eace3b9ffde02c606c385a8/e8d306be8eace3b9ffde02c606c385a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346184.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c4f/3d33e391e9c9395fc2b31203a8c40c4f/3d33e391e9c9395fc2b31203a8c40c4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346189.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ea5/012a04332b999578a193ef8d13e03ea5/012a04332b999578a193ef8d13e03ea5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346194.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/227/e62f94e3aab5b55c2f28961381a11227/e62f94e3aab5b55c2f28961381a11227.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346199.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/013/5fc61e3cfcebf5a8ce3b99cc5a58b013/5fc61e3cfcebf5a8ce3b99cc5a58b013.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346204.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/962/9cb103d526d43a66a5a8f048a1198962/9cb103d526d43a66a5a8f048a1198962.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346209.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/672/2c6d76486c1f97276fa47d9c9a44d672/2c6d76486c1f97276fa47d9c9a44d672.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346214.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/797/14b81786bac9a5b4b378afc86aed2797/14b81786bac9a5b4b378afc86aed2797.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346219.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/367/a959f8f9577df2fbf75cd54314923367/a959f8f9577df2fbf75cd54314923367.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346224.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/467/9ff781c5ed8226e3d2242ed7ab2ef467/9ff781c5ed8226e3d2242ed7ab2ef467.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346229.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/89a/d756bd0e65cdb91b393c05c8c57d889a/d756bd0e65cdb91b393c05c8c57d889a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346234.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/864/37a2a969437115a0d1fc0ad5d86ab864/37a2a969437115a0d1fc0ad5d86ab864.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346239.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e6b/1cc7179e299f33ae2a340c8eba37ae6b/1cc7179e299f33ae2a340c8eba37ae6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346244.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/583/e566ccefc66b1dcc5171a3cf8215a583/e566ccefc66b1dcc5171a3cf8215a583.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346249.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b94/c7d39ebb29e6831167390da6527edb94/c7d39ebb29e6831167390da6527edb94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346254.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d59/c02e67e3e106e65d589856186663ed59/c02e67e3e106e65d589856186663ed59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346259.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/581/8a845be8899b80641d35e00cf0e96581/8a845be8899b80641d35e00cf0e96581.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346264.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/af1/525c6711c809b0547e983393d721eaf1/525c6711c809b0547e983393d721eaf1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346269.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d3d/ce14c18ba1bc66f6f161d9e402d7fd3d/ce14c18ba1bc66f6f161d9e402d7fd3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346274.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/692/20b8a3dba44653e379ef9e7b2e925692/20b8a3dba44653e379ef9e7b2e925692.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346279.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/396/7272307e65b5d3207e58e03cf201f396/7272307e65b5d3207e58e03cf201f396.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346284.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/5b0/391482bb0808615005868af91bed35b0/391482bb0808615005868af91bed35b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346289.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/192/c006d88ae42284f36a5ba73896018192/c006d88ae42284f36a5ba73896018192.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346294.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/412/02df473d450e56b4fc7180e9347eb412/02df473d450e56b4fc7180e9347eb412.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346299.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/546/317e5fb0b11565af8533d73963d00546/317e5fb0b11565af8533d73963d00546.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346304.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/651/df803259ed0bdf6c6b86413887227651/df803259ed0bdf6c6b86413887227651.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346309.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/43e/17c969e22ad9fb4e4970a15b0b1a943e/17c969e22ad9fb4e4970a15b0b1a943e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346314.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/16d/82f1c2b3d47799b02aa83266c2ace16d/82f1c2b3d47799b02aa83266c2ace16d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346319.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/804/07fb83d270b00e323713215e60b07804/07fb83d270b00e323713215e60b07804.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346324.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e00/f9aec1c0cc05e85022504722e29fbe00/f9aec1c0cc05e85022504722e29fbe00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346329.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0eb/748e03e38b5817783fcc5adc9514b0eb/748e03e38b5817783fcc5adc9514b0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346334.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9eb/666078c758a3a7c0d411e438abd529eb/666078c758a3a7c0d411e438abd529eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346339.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7e2/99fc83c8df8f2e28c00f78586a53f7e2/99fc83c8df8f2e28c00f78586a53f7e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346344.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/50b/27fb210b4a7dae23f5d04ddbe7a7550b/27fb210b4a7dae23f5d04ddbe7a7550b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346349.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d76/b21fd4d1f56f43623336de89ab6a3d76/b21fd4d1f56f43623336de89ab6a3d76.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346354.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/925/83216fbb83b0cda56b09d0d97203d925/83216fbb83b0cda56b09d0d97203d925.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346359.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/547/63061137146992e76bb8199dded2f547/63061137146992e76bb8199dded2f547.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346364.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/457/112ddcec7164b61f9226c48b68957457/112ddcec7164b61f9226c48b68957457.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346369.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/19f/8d2af439a57a51976c2e8b54fc97b19f/8d2af439a57a51976c2e8b54fc97b19f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346374.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b0c/6a0fabe4b3328900eeab1ff0c9904b0c/6a0fabe4b3328900eeab1ff0c9904b0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346379.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ceb/e72472d16bcf71d72c22e0b26c450ceb/e72472d16bcf71d72c22e0b26c450ceb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346384.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ccb/010f921613a72dc81bed93a95285fccb/010f921613a72dc81bed93a95285fccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346389.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d63/2a16ebdedaca835956885cfdeca76d63/2a16ebdedaca835956885cfdeca76d63.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346394.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/73d/e904eb5e9cfd954f9aac001f0643f73d/e904eb5e9cfd954f9aac001f0643f73d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346399.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/db5/69687b9b60a4830eb53911f67a84fdb5/69687b9b60a4830eb53911f67a84fdb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346404.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9f3/405fbab0e9ce66cd0a73f7917308c9f3/405fbab0e9ce66cd0a73f7917308c9f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346409.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f68/609e91029b874e5764bcd6dc53796f68/609e91029b874e5764bcd6dc53796f68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346414.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/947/c80d28a340a6232377229046ccd53947/c80d28a340a6232377229046ccd53947.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346419.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7b4/83cdff253f148422fff5ca4c15f1d7b4/83cdff253f148422fff5ca4c15f1d7b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346424.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/378/6c2b07bcb2e9e0b25f3c8bea02462378/6c2b07bcb2e9e0b25f3c8bea02462378.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346429.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/012/ac016eb0258610b47ad658dcfca6f012/ac016eb0258610b47ad658dcfca6f012.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346434.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a9a/2322c1a1b57bf6c9b889c73c457b1a9a/2322c1a1b57bf6c9b889c73c457b1a9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346439.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d49/841ad5001983132019d54d89faa93d49/841ad5001983132019d54d89faa93d49.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346444.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/cb7/f1ee7718f93965a7ba5df08d75f2bcb7/f1ee7718f93965a7ba5df08d75f2bcb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346449.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b20/2d4a8281d8499e65d05224aac2b62b20/2d4a8281d8499e65d05224aac2b62b20.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346454.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/55c/19597d94f29b94b63a5b34bc8451555c/19597d94f29b94b63a5b34bc8451555c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346459.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0a9/75296332ce45eb9d13aa86716322b0a9/75296332ce45eb9d13aa86716322b0a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346464.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2b2/799ea392bd529e4d53a635807aad12b2/799ea392bd529e4d53a635807aad12b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346469.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/949/683b1a52b6ad7242a319309dcb310949/683b1a52b6ad7242a319309dcb310949.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346474.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7d7/34dbae561b47aa7379b4523a7f5167d7/34dbae561b47aa7379b4523a7f5167d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346479.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/407/9e49b6c940c3174ad635337f4a0d0407/9e49b6c940c3174ad635337f4a0d0407.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346484.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/217/2311a74e791bb24c77a14dadf9636217/2311a74e791bb24c77a14dadf9636217.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346489.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/881/fbca0193656dcf55e0502bed47f07881/fbca0193656dcf55e0502bed47f07881.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346494.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d7a/2187cf4bec89dcfbd29aa3ef01d4cd7a/2187cf4bec89dcfbd29aa3ef01d4cd7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346499.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/5b5/528b7083001c42642853b775cacf25b5/528b7083001c42642853b775cacf25b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346504.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/6d2/ac5d3f02c1041d21a841b92228b826d2/ac5d3f02c1041d21a841b92228b826d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346509.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/582/c554a928f073e9b98827212eb36cc582/c554a928f073e9b98827212eb36cc582.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346514.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/fae/3c483b5f692139fdd01e3d0ea08d8fae/3c483b5f692139fdd01e3d0ea08d8fae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346519.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9bd/e5402c7e1548cb3c369d6a2f1816b9bd/e5402c7e1548cb3c369d6a2f1816b9bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346524.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c59/baf6af78af602cd4f360fefd15774c59/baf6af78af602cd4f360fefd15774c59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346529.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/91c/4bc4708abefa999410e3b5fd751ba91c/4bc4708abefa999410e3b5fd751ba91c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346534.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2c1/e7a1682ca207bd38c60af64803a422c1/e7a1682ca207bd38c60af64803a422c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346539.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7ed/41ec06bd01c779cbd7118437be4d77ed/41ec06bd01c779cbd7118437be4d77ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346544.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/cc3/79eef62c81e8c417b91c40e4e4cf4cc3/79eef62c81e8c417b91c40e4e4cf4cc3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346549.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/2c0/8b9284141f12e46f87f4184c25dcf2c0/8b9284141f12e46f87f4184c25dcf2c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346554.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/12b/9bb3da0fec0b7a7d024ce10748dde12b/9bb3da0fec0b7a7d024ce10748dde12b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346559.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3f2/8cd286e864131bdaaebc8001c98473f2/8cd286e864131bdaaebc8001c98473f2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346564.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d5a/9ae70ab335f7dbcc5ec56e30ec44bd5a/9ae70ab335f7dbcc5ec56e30ec44bd5a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346569.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/003/4567a19e507591d943664b969053a003/4567a19e507591d943664b969053a003.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346574.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/efd/740df114b3703f4b8fddc15d8baeeefd/740df114b3703f4b8fddc15d8baeeefd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346579.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/461/52eecb5a21a312c63d361d1171c7a461/52eecb5a21a312c63d361d1171c7a461.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346584.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9ad/8e6c7ded34396f72b91f0045da40d9ad/8e6c7ded34396f72b91f0045da40d9ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346589.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/b77/612079a85dee173bbd599aca61015b77/612079a85dee173bbd599aca61015b77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346594.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/4cd/780e90710e8ba184cfe4734155a964cd/780e90710e8ba184cfe4734155a964cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346599.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/8e8/770b2104c5d1e92ca7ffd6823e3508e8/770b2104c5d1e92ca7ffd6823e3508e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346604.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/213/f9fcb53c793072af32eb334345069213/f9fcb53c793072af32eb334345069213.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346609.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/287/941c0e207c960dbd9b04329972c66287/941c0e207c960dbd9b04329972c66287.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346614.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/987/f9c23850017ac8881690b032b2865987/f9c23850017ac8881690b032b2865987.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346619.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/4e8/d4e33dbe358d1ab0ddc0b5e7e0c574e8/d4e33dbe358d1ab0ddc0b5e7e0c574e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346624.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/7a3/dddd38d439b29f42c149653e958bc7a3/dddd38d439b29f42c149653e958bc7a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346629.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/940/0000159b0387c3b7106e2897e8992940/0000159b0387c3b7106e2897e8992940.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346634.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/07b/423326c11c9a17405d4f7f9e5985707b/423326c11c9a17405d4f7f9e5985707b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346639.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/1de/aa85a8f683a35bc2ece861103982c1de/aa85a8f683a35bc2ece861103982c1de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346644.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/380/30fe46d502dca9a1f34b0d3e6dcba380/30fe46d502dca9a1f34b0d3e6dcba380.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346649.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/410/b22fc1ff74d5d455fd66a5cd8ba77410/b22fc1ff74d5d455fd66a5cd8ba77410.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346654.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/173/da0e11644432c597bc3ee60e6ca6e173/da0e11644432c597bc3ee60e6ca6e173.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346659.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/1e6/a767283c1cb1490f12d446a6943711e6/a767283c1cb1490f12d446a6943711e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346664.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/36d/efb8e3ce8efd2d281e16bce4e0e4836d/efb8e3ce8efd2d281e16bce4e0e4836d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346669.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/3dd/5663fd172b1fd767efa3481daee2f3dd/5663fd172b1fd767efa3481daee2f3dd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346674.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/35e/56a25bfc0e53d56c058b40caf77a535e/56a25bfc0e53d56c058b40caf77a535e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346679.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/409/a0d20fe252a008a606ed52b46e7a5409/a0d20fe252a008a606ed52b46e7a5409.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346684.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c4b/517171f252d08c1c9fb6a9dd88f2bc4b/517171f252d08c1c9fb6a9dd88f2bc4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346689.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/0cd/0d25f53176fc70d0048acb81e5f530cd/0d25f53176fc70d0048acb81e5f530cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346694.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/001/c6f1e7f7a67365566067c45beda8e001/c6f1e7f7a67365566067c45beda8e001.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346699.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/98b/197a4648d3baa1e88b6b245e0c9c198b/197a4648d3baa1e88b6b245e0c9c198b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346704.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a0c/027742e868db059b70c18720250f6a0c/027742e868db059b70c18720250f6a0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346709.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d07/04ad17e5c3b4aa8e6e8adccd25d19d07/04ad17e5c3b4aa8e6e8adccd25d19d07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346714.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/be3/5cee21064623ca2394ca60007472dbe3/5cee21064623ca2394ca60007472dbe3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346719.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/f89/eb502f2fac0e72aaf3cf40333b207f89/eb502f2fac0e72aaf3cf40333b207f89.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346724.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/74b/c35d7d36ad892264612bc78c536a974b/c35d7d36ad892264612bc78c536a974b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346729.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/13d/f6f4b130d55ae852ce09698af313b13d/f6f4b130d55ae852ce09698af313b13d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346734.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/925/613ff077fcfb9807049097b4acda8925/613ff077fcfb9807049097b4acda8925.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346739.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/5ed/07b4de0f9002c31951d2a006f4d1d5ed/07b4de0f9002c31951d2a006f4d1d5ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346744.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e08/5df782d85f60f91d19e4b468a2e7ae08/5df782d85f60f91d19e4b468a2e7ae08.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346749.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/9c3/777531632617e0b018b49190dbb079c3/777531632617e0b018b49190dbb079c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346754.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/c72/d4f0d5ef497e9dd8c1792a80d1e1dc72/d4f0d5ef497e9dd8c1792a80d1e1dc72.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346759.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/15a/da07386545c33c5a000ed75d015cf15a/da07386545c33c5a000ed75d015cf15a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346764.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a7e/5bb995f5d1afd810e3e28c9468263a7e/5bb995f5d1afd810e3e28c9468263a7e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346769.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/30c/568fcbd7483411091e17541cd9b3f30c/568fcbd7483411091e17541cd9b3f30c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346774.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/109/ecce68027d633e29c12b599ad50d2109/ecce68027d633e29c12b599ad50d2109.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346779.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/e74/343614eef3cf64aff54e5a2add4efe74/343614eef3cf64aff54e5a2add4efe74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346784.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/511/ea9326d9c91753595a319c7c899f5511/ea9326d9c91753595a319c7c899f5511.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346789.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/4d4/e141247637b6758c2b9c6df3a87744d4/e141247637b6758c2b9c6df3a87744d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346794.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/403/e61d54dd27800e578e52d4d3d9669403/e61d54dd27800e578e52d4d3d9669403.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346799.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/d23/41d7c2c7f635e60d3ceabf629e907d23/41d7c2c7f635e60d3ceabf629e907d23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346804.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a96/69c6356efbdfd7980d1c1197e0e6fa96/69c6356efbdfd7980d1c1197e0e6fa96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346809.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/09b/3d3231568a42b3262dc90e06ad36209b/3d3231568a42b3262dc90e06ad36209b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346814.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ed7/c62b8da553f61fc683007acd19577ed7/c62b8da553f61fc683007acd19577ed7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346819.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/904/51a781328b80ec70e6299b6aa2c4e904/51a781328b80ec70e6299b6aa2c4e904.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346824.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/ed4/bde45470a7088d629eb6d8679b456ed4/bde45470a7088d629eb6d8679b456ed4.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving ... ok proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txad96ec731544436abda84-00665c9d1f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346829.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/87f/079bcfc3149c5ac14b4857768841b87f/079bcfc3149c5ac14b4857768841b87f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346834.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/913/97b35cb7e71a7d38591472a0462c0913/97b35cb7e71a7d38591472a0462c0913.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346839.00000-0, db: /tmp/tmpbgngmorf/sda/containers/0/a13/363e1c59293ee28ce16f308002263a13/363e1c59293ee28ce16f308002263a13.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345571.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/e8f/704a4acec9f82dcba9c13def74959e8f/704a4acec9f82dcba9c13def74959e8f_1717345571.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/8b0/f8ae096f335e76e81d948650f16eb8b0/f8ae096f335e76e81d948650f16eb8b0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345575.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/8b0/f8ae096f335e76e81d948650f16eb8b0/f8ae096f335e76e81d948650f16eb8b0_1717345575.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345579.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa_1717345579.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f_1717345583.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345587.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e_1717345587.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/494/736bc72b1b2f43d54260fe0ac1ced494/736bc72b1b2f43d54260fe0ac1ced494.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345591.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/494/736bc72b1b2f43d54260fe0ac1ced494/736bc72b1b2f43d54260fe0ac1ced494_1717345591.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345595.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/70e/8d19cf7e804e73646317b5148210770e/8d19cf7e804e73646317b5148210770e_1717345595.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb_1717345599.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a45/98fc46327bc3d858f7b8d63d7ab08a45/98fc46327bc3d858f7b8d63d7ab08a45.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345603.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a45/98fc46327bc3d858f7b8d63d7ab08a45/98fc46327bc3d858f7b8d63d7ab08a45_1717345603.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345607.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/cc9/467e0dbf91b1fb7b52803d5785379cc9/467e0dbf91b1fb7b52803d5785379cc9_1717345607.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345611.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2_1717345611.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f1e/d2df3b46c47d0a6c39116672a5fe8f1e/d2df3b46c47d0a6c39116672a5fe8f1e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345615.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f1e/d2df3b46c47d0a6c39116672a5fe8f1e/d2df3b46c47d0a6c39116672a5fe8f1e_1717345615.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345619.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/ed4/484ad95d0ed9693d61f5f5901830eed4/484ad95d0ed9693d61f5f5901830eed4_1717345619.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53_1717345623.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/142/427fb0be2423394f2109eb3f824ed142/427fb0be2423394f2109eb3f824ed142.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345627.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/142/427fb0be2423394f2109eb3f824ed142/427fb0be2423394f2109eb3f824ed142_1717345627.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345631.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/9b5/22897c70739c747a9dc7413f152299b5/22897c70739c747a9dc7413f152299b5_1717345631.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d_1717345635.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f62/8474b27a3f16620032180bcd8cceff62/8474b27a3f16620032180bcd8cceff62.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345639.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f62/8474b27a3f16620032180bcd8cceff62/8474b27a3f16620032180bcd8cceff62_1717345639.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345643.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/cd1/6f5fd576c148c02c6e08e23a78a0dcd1/6f5fd576c148c02c6e08e23a78a0dcd1_1717345643.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345647.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66_1717345647.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/648/fbfc5e169925794f41d80b045b5bb648/fbfc5e169925794f41d80b045b5bb648.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345651.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/648/fbfc5e169925794f41d80b045b5bb648/fbfc5e169925794f41d80b045b5bb648_1717345651.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345655.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d75/0ea6ca45d249f2d506e083fa7b55bd75/0ea6ca45d249f2d506e083fa7b55bd75_1717345655.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5_1717345659.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a56/86591c8dc136221e54b9f1ec5334ea56/86591c8dc136221e54b9f1ec5334ea56.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345663.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a56/86591c8dc136221e54b9f1ec5334ea56/86591c8dc136221e54b9f1ec5334ea56_1717345663.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345667.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a44/082deae58a1939a8a55e97e75cf75a44/082deae58a1939a8a55e97e75cf75a44_1717345667.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345671.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e_1717345671.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/b39/ade3a336b3114f0025b8f6ca4fa05b39/ade3a336b3114f0025b8f6ca4fa05b39.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345675.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/b39/ade3a336b3114f0025b8f6ca4fa05b39/ade3a336b3114f0025b8f6ca4fa05b39_1717345675.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345679.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/02c/52e3aaa2b2775125441720058960902c/52e3aaa2b2775125441720058960902c_1717345679.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345683.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90_1717345683.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f81/75edaf0cd030162232217d2fef599f81/75edaf0cd030162232217d2fef599f81.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345687.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f81/75edaf0cd030162232217d2fef599f81/75edaf0cd030162232217d2fef599f81_1717345687.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345691.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/cb7/ce06ea804676f9629390e7a3e4db3cb7/ce06ea804676f9629390e7a3e4db3cb7_1717345691.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2_1717345695.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/1be/701a49feead126ddccd7e937c1acd1be/701a49feead126ddccd7e937c1acd1be.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345699.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/1be/701a49feead126ddccd7e937c1acd1be/701a49feead126ddccd7e937c1acd1be_1717345699.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345703.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f84/56c4d089f40a176b3d73c625dcd37f84/56c4d089f40a176b3d73c625dcd37f84_1717345703.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345707.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82_1717345707.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/657/3e8ef64022f6dfedbe89412c42002657/3e8ef64022f6dfedbe89412c42002657.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345711.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/657/3e8ef64022f6dfedbe89412c42002657/3e8ef64022f6dfedbe89412c42002657_1717345711.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345715.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/3a3/da90d3b64529c713a265bcc3ed2ae3a3/da90d3b64529c713a265bcc3ed2ae3a3_1717345715.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48_1717345719.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/07c/2fde1ec26ef1340c593460c15d84707c/2fde1ec26ef1340c593460c15d84707c.db' proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.8:1008/sdi (txn: tx876a5d784685436488c45-00665c9d20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.6:1006/sdg (txn: txe75271a389b64906a1648-00665c9d20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.3:1003/sdd (txn: txb34811c5a0a141119b62b-00665c9d20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx6091f935ae004f0180ace-00665c9d20) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx6091f935ae004f0180ace-00665c9d20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx584477fb54894092bbbc0-00665c9d21) proxy-server ERROR: Object GET returning 503 for [] (txn: tx584477fb54894092bbbc0-00665c9d21) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac75861861714c7899992-00665c9d21) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1a89262756ac450ba78a2-00665c9d21) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe3d02e8cfb7f47378ddb5-00665c9d21) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmpiekfr90g/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpiekfr90g/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmpiekfr90g/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpiekfr90g/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0111 "-" 3542001 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0121 "-" 3542001 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0146 "-" 3542001 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0099 "-" 3542001 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf62eda560ee348f7b5c4f-00665c9d22" "proxy-server 3542001" 0.0019 "-" 3542001 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:55967/sda1/1/a/c" "txf62eda560ee348f7b5c4f-00665c9d22" "container-server 3542001" 0.0019 "-" 3542001 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:41495/sdb1/1/a/c" "txf62eda560ee348f7b5c4f-00665c9d22" "container-server 3542001" 0.0020 "-" 3542001 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf62eda560ee348f7b5c4f-00665c9d22" "proxy-server 3542001" 0.0222 "-" 3542001 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf62eda560ee348f7b5c4f-00665c9d22" "proxy-server 3542001" 0.0404 "-" 3542001 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/10 PUT /v1/a/c HTTP/1.0 201 - - t - - - txf62eda560ee348f7b5c4f-00665c9d22 - 0.0525 - - 1717345570.573883057 1717345570.626404047 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txebf1edca9c0547d285521-00665c9d22" "proxy-server 3542001" 0.0018 "-" 3542001 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:55967/sda1/1/a1/c1" "txebf1edca9c0547d285521-00665c9d22" "container-server 3542001" 0.0016 "-" 3542001 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:41495/sdb1/1/a1/c1" "txebf1edca9c0547d285521-00665c9d22" "container-server 3542001" 0.0017 "-" 3542001 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txebf1edca9c0547d285521-00665c9d22" "proxy-server 3542001" 0.0359 "-" 3542001 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txebf1edca9c0547d285521-00665c9d22" "proxy-server 3542001" 0.0230 "-" 3542001 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/10 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txebf1edca9c0547d285521-00665c9d22 - 0.0489 - - 1717345570.628668785 1717345570.677544117 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx961cf25f4b3644ab9d2e1-00665c9d22" "proxy-server 3542001" 0.0081 "-" 3542001 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:55967/sda1/2/a/c1" "tx961cf25f4b3644ab9d2e1-00665c9d22" "container-server 3542001" 0.0120 "-" 3542001 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:41495/sdb1/2/a/c1" "tx961cf25f4b3644ab9d2e1-00665c9d22" "container-server 3542001" 0.0012 "-" 3542001 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx961cf25f4b3644ab9d2e1-00665c9d22" "proxy-server 3542001" 0.0320 "-" 3542001 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx961cf25f4b3644ab9d2e1-00665c9d22" "proxy-server 3542001" 0.0463 "-" 3542001 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/10 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx961cf25f4b3644ab9d2e1-00665c9d22 - 0.0640 - - 1717345570.679768324 1717345570.743766546 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx8e04eb627e3d4fcb98893-00665c9d22" "proxy-server 3542001" 0.0074 "-" 3542001 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:41495/sdb1/1/a/c2" "tx8e04eb627e3d4fcb98893-00665c9d22" "container-server 3542001" 0.0013 "-" 3542001 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:55967/sda1/1/a/c2" "tx8e04eb627e3d4fcb98893-00665c9d22" "container-server 3542001" 0.0076 "-" 3542001 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx8e04eb627e3d4fcb98893-00665c9d22" "proxy-server 3542001" 0.0337 "-" 3542001 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:26:10 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx8e04eb627e3d4fcb98893-00665c9d22" "proxy-server 3542001" 0.0234 "-" 3542001 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/26/10 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx8e04eb627e3d4fcb98893-00665c9d22 - 0.0498 - - 1717345570.745425224 1717345570.795228243 2 sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345723.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/07c/2fde1ec26ef1340c593460c15d84707c/2fde1ec26ef1340c593460c15d84707c_1717345723.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345727.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/8df/61c5afdeab6305bff0f0f351a28038df/61c5afdeab6305bff0f0f351a28038df_1717345727.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765_1717345731.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/85c/6cc1446b3d3747a132dfe6e37382285c/6cc1446b3d3747a132dfe6e37382285c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345735.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/85c/6cc1446b3d3747a132dfe6e37382285c/6cc1446b3d3747a132dfe6e37382285c_1717345735.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345739.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/e12/0d532b71bc50082309e4df0442b49e12/0d532b71bc50082309e4df0442b49e12_1717345739.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345743.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089_1717345743.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/bf8/64e7e0002088f0e061b43e4dd89b3bf8/64e7e0002088f0e061b43e4dd89b3bf8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345747.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/bf8/64e7e0002088f0e061b43e4dd89b3bf8/64e7e0002088f0e061b43e4dd89b3bf8_1717345747.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345751.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6_1717345751.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/db9/d57ac6f55bb096b7b53bce53eb985db9/d57ac6f55bb096b7b53bce53eb985db9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345755.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/db9/d57ac6f55bb096b7b53bce53eb985db9/d57ac6f55bb096b7b53bce53eb985db9_1717345755.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/341/d9772fe63a4c24f892e39f1a7c29a341/d9772fe63a4c24f892e39f1a7c29a341.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345759.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/341/d9772fe63a4c24f892e39f1a7c29a341/d9772fe63a4c24f892e39f1a7c29a341_1717345759.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/eba/c8656bf9dd67ae39a5ff0e8b0ed5beba/c8656bf9dd67ae39a5ff0e8b0ed5beba.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345763.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/eba/c8656bf9dd67ae39a5ff0e8b0ed5beba/c8656bf9dd67ae39a5ff0e8b0ed5beba_1717345763.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/82a/af7a7d95593bebe38159c18cd239482a/af7a7d95593bebe38159c18cd239482a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345767.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/82a/af7a7d95593bebe38159c18cd239482a/af7a7d95593bebe38159c18cd239482a_1717345767.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f13/13dcd0d823693a4a26ba602b28f31f13/13dcd0d823693a4a26ba602b28f31f13.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345771.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f13/13dcd0d823693a4a26ba602b28f31f13/13dcd0d823693a4a26ba602b28f31f13_1717345771.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a51/0357613d8a0b68cbecaaf47a2ea38a51/0357613d8a0b68cbecaaf47a2ea38a51.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345775.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a51/0357613d8a0b68cbecaaf47a2ea38a51/0357613d8a0b68cbecaaf47a2ea38a51_1717345775.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/3fd/7cde512a498cc4f6e7e54bdd9fce83fd/7cde512a498cc4f6e7e54bdd9fce83fd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345779.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/3fd/7cde512a498cc4f6e7e54bdd9fce83fd/7cde512a498cc4f6e7e54bdd9fce83fd_1717345779.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/ebd/6b695f37531b4d4f75d96dac331f6ebd/6b695f37531b4d4f75d96dac331f6ebd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345783.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/ebd/6b695f37531b4d4f75d96dac331f6ebd/6b695f37531b4d4f75d96dac331f6ebd_1717345783.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/ee6/ae9d9a47225312c25d2fa1f7d8b2eee6/ae9d9a47225312c25d2fa1f7d8b2eee6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345787.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/ee6/ae9d9a47225312c25d2fa1f7d8b2eee6/ae9d9a47225312c25d2fa1f7d8b2eee6_1717345787.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a04/4e8dd23d553a52a0e9b40fb0b2444a04/4e8dd23d553a52a0e9b40fb0b2444a04.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345791.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a04/4e8dd23d553a52a0e9b40fb0b2444a04/4e8dd23d553a52a0e9b40fb0b2444a04_1717345791.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/09a/a7d77580e4fd4195087224c157e0a09a/a7d77580e4fd4195087224c157e0a09a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345795.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/09a/a7d77580e4fd4195087224c157e0a09a/a7d77580e4fd4195087224c157e0a09a_1717345795.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/788/e233b98157246fb2f16763643621d788/e233b98157246fb2f16763643621d788.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345799.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/788/e233b98157246fb2f16763643621d788/e233b98157246fb2f16763643621d788_1717345799.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/22f/14ec906a79bed1ec1a02553c81f6022f/14ec906a79bed1ec1a02553c81f6022f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345803.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/22f/14ec906a79bed1ec1a02553c81f6022f/14ec906a79bed1ec1a02553c81f6022f_1717345803.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/01b/d96b6495f96d1d875f1a325b4345701b/d96b6495f96d1d875f1a325b4345701b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345807.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/01b/d96b6495f96d1d875f1a325b4345701b/d96b6495f96d1d875f1a325b4345701b_1717345807.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/b5d/a0ca57477be8e33821cf5fd06a32cb5d/a0ca57477be8e33821cf5fd06a32cb5d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345811.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/b5d/a0ca57477be8e33821cf5fd06a32cb5d/a0ca57477be8e33821cf5fd06a32cb5d_1717345811.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/b34/65a2ad93f8f5b2927b3701941925bb34/65a2ad93f8f5b2927b3701941925bb34.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345815.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/b34/65a2ad93f8f5b2927b3701941925bb34/65a2ad93f8f5b2927b3701941925bb34_1717345815.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f70/0acab4cfaeb4df90edc3dbff69a60f70/0acab4cfaeb4df90edc3dbff69a60f70.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345819.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f70/0acab4cfaeb4df90edc3dbff69a60f70/0acab4cfaeb4df90edc3dbff69a60f70_1717345819.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/e03/e8af4b8e29298723c43e8a2b6decce03/e8af4b8e29298723c43e8a2b6decce03.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345823.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/e03/e8af4b8e29298723c43e8a2b6decce03/e8af4b8e29298723c43e8a2b6decce03_1717345823.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/fdf/bc59eaa8477b3dc14f979f6d49281fdf/bc59eaa8477b3dc14f979f6d49281fdf.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345827.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/fdf/bc59eaa8477b3dc14f979f6d49281fdf/bc59eaa8477b3dc14f979f6d49281fdf_1717345827.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/648/5d054da4c811982b1ec46474cc885648/5d054da4c811982b1ec46474cc885648.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345831.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/648/5d054da4c811982b1ec46474cc885648/5d054da4c811982b1ec46474cc885648_1717345831.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/025/600568b26e686e91b350099bbf036025/600568b26e686e91b350099bbf036025.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345835.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/025/600568b26e686e91b350099bbf036025/600568b26e686e91b350099bbf036025_1717345835.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/5e3/aa48eb23cb4abb86c0917980853135e3/aa48eb23cb4abb86c0917980853135e3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345839.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/5e3/aa48eb23cb4abb86c0917980853135e3/aa48eb23cb4abb86c0917980853135e3_1717345839.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/cc5/133b0ea6c0251a0aceb196fb46903cc5/133b0ea6c0251a0aceb196fb46903cc5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345843.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/cc5/133b0ea6c0251a0aceb196fb46903cc5/133b0ea6c0251a0aceb196fb46903cc5_1717345843.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/68e/02c60b29fedc06909056c2660953068e/02c60b29fedc06909056c2660953068e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345847.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/68e/02c60b29fedc06909056c2660953068e/02c60b29fedc06909056c2660953068e_1717345847.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345851.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3_1717345851.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/0b3/7da809de52d6c0590cd8f1957c7d40b3/7da809de52d6c0590cd8f1957c7d40b3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345855.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/0b3/7da809de52d6c0590cd8f1957c7d40b3/7da809de52d6c0590cd8f1957c7d40b3_1717345855.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/82a/5c7d54d86457214ab2b267672a27a82a/5c7d54d86457214ab2b267672a27a82a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345859.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/82a/5c7d54d86457214ab2b267672a27a82a/5c7d54d86457214ab2b267672a27a82a_1717345859.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f35/b7205c0fa015d4356e1a4e6c19164f35/b7205c0fa015d4356e1a4e6c19164f35.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345863.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f35/b7205c0fa015d4356e1a4e6c19164f35/b7205c0fa015d4356e1a4e6c19164f35_1717345863.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/10a/103cd16479c54c9e582618292a3c110a/103cd16479c54c9e582618292a3c110a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345867.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/10a/103cd16479c54c9e582618292a3c110a/103cd16479c54c9e582618292a3c110a_1717345867.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/577/ff75c4e36d977172a3d89f3beab16577/ff75c4e36d977172a3d89f3beab16577.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345871.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/577/ff75c4e36d977172a3d89f3beab16577/ff75c4e36d977172a3d89f3beab16577_1717345871.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345875.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80_1717345875.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/277/b45fb34ccf3414c4389ce686a2cba277/b45fb34ccf3414c4389ce686a2cba277.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345879.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/277/b45fb34ccf3414c4389ce686a2cba277/b45fb34ccf3414c4389ce686a2cba277_1717345879.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a86/9f96247331cfe4b0e61fad020caada86/9f96247331cfe4b0e61fad020caada86.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345883.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a86/9f96247331cfe4b0e61fad020caada86/9f96247331cfe4b0e61fad020caada86_1717345883.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/e71/1484e75f1d95065b67c23a8eff750e71/1484e75f1d95065b67c23a8eff750e71.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345887.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/e71/1484e75f1d95065b67c23a8eff750e71/1484e75f1d95065b67c23a8eff750e71_1717345887.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/20d/4eb4de89a5b06c3f96b6b1e27ba4320d/4eb4de89a5b06c3f96b6b1e27ba4320d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345891.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/20d/4eb4de89a5b06c3f96b6b1e27ba4320d/4eb4de89a5b06c3f96b6b1e27ba4320d_1717345891.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345895.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031_1717345895.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882_1717345899.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/892/b50ba0993aeed6c54f87327f9a014892/b50ba0993aeed6c54f87327f9a014892.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345903.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/892/b50ba0993aeed6c54f87327f9a014892/b50ba0993aeed6c54f87327f9a014892_1717345903.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345907.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226_1717345907.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0_1717345911.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d41/740cb5d220733796133a1347f1910d41/740cb5d220733796133a1347f1910d41.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345915.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d41/740cb5d220733796133a1347f1910d41/740cb5d220733796133a1347f1910d41_1717345915.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9_1717345919.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/0bd/4cb334d4f01ecd9abfdf3bdca8df30bd/4cb334d4f01ecd9abfdf3bdca8df30bd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345923.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/0bd/4cb334d4f01ecd9abfdf3bdca8df30bd/4cb334d4f01ecd9abfdf3bdca8df30bd_1717345923.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/000/9cc39cf5e202188f3cb365ab1f84b000/9cc39cf5e202188f3cb365ab1f84b000.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345927.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/000/9cc39cf5e202188f3cb365ab1f84b000/9cc39cf5e202188f3cb365ab1f84b000_1717345927.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47_1717345931.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e_1717345935.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d69/82df5f6d5f930f5efcd5c422ac1bcd69/82df5f6d5f930f5efcd5c422ac1bcd69.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345939.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d69/82df5f6d5f930f5efcd5c422ac1bcd69/82df5f6d5f930f5efcd5c422ac1bcd69_1717345939.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc_1717345943.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a17/d7ac936fa77aa2f44d71ebdc60230a17/d7ac936fa77aa2f44d71ebdc60230a17.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345947.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a17/d7ac936fa77aa2f44d71ebdc60230a17/d7ac936fa77aa2f44d71ebdc60230a17_1717345947.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/6f3/5145c6253305303db4393a56882626f3/5145c6253305303db4393a56882626f3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345951.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/6f3/5145c6253305303db4393a56882626f3/5145c6253305303db4393a56882626f3_1717345951.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029_1717345955.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589_1717345959.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/1c6/923db0dcb5f23439340d0336b4df81c6/923db0dcb5f23439340d0336b4df81c6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345963.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/1c6/923db0dcb5f23439340d0336b4df81c6/923db0dcb5f23439340d0336b4df81c6_1717345963.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2_1717345967.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345971.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01_1717345971.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/1c6/06727a10627bb589627c0835996451c6/06727a10627bb589627c0835996451c6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345975.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/1c6/06727a10627bb589627c0835996451c6/06727a10627bb589627c0835996451c6_1717345975.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b_1717345979.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/2fe/14e8e0360f977223aacbcdb1876582fe/14e8e0360f977223aacbcdb1876582fe.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345983.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/2fe/14e8e0360f977223aacbcdb1876582fe/14e8e0360f977223aacbcdb1876582fe_1717345983.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/a2f/0976486a84408c4d0a9612b9c99f0a2f/0976486a84408c4d0a9612b9c99f0a2f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345987.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/a2f/0976486a84408c4d0a9612b9c99f0a2f/0976486a84408c4d0a9612b9c99f0a2f_1717345987.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29_1717345991.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345995.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35_1717345995.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345999.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8_1717345999.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0_1717346003.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/d9a/cd426bceff16dee8a0035688912e7d9a/cd426bceff16dee8a0035688912e7d9a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346007.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/d9a/cd426bceff16dee8a0035688912e7d9a/cd426bceff16dee8a0035688912e7d9a_1717346007.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/9b1/cecec856fd6c654aee8d170536e519b1/cecec856fd6c654aee8d170536e519b1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346011.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/9b1/cecec856fd6c654aee8d170536e519b1/cecec856fd6c654aee8d170536e519b1_1717346011.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488_1717346015.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e_1717346019.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/f7e/c76a6f341fe0b5daa394696434d01f7e/c76a6f341fe0b5daa394696434d01f7e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346023.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/f7e/c76a6f341fe0b5daa394696434d01f7e/c76a6f341fe0b5daa394696434d01f7e_1717346023.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364_1717346027.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893_1717346031.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/037/21fe1f4898825a6a856e1f376b464037/21fe1f4898825a6a856e1f376b464037.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346035.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/037/21fe1f4898825a6a856e1f376b464037/21fe1f4898825a6a856e1f376b464037_1717346035.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9_1717346039.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/34e/65d7cd5609f1cd05bcfa5f850928b34e/65d7cd5609f1cd05bcfa5f850928b34e.db' test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346043.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/34e/65d7cd5609f1cd05bcfa5f850928b34e/65d7cd5609f1cd05bcfa5f850928b34e_1717346043.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/e10/0c21378f348cc36f4019f220b9ec9e10/0c21378f348cc36f4019f220b9ec9e10.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346047.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/e10/0c21378f348cc36f4019f220b9ec9e10/0c21378f348cc36f4019f220b9ec9e10_1717346047.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346051.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630_1717346051.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/76b/8517e9828eb7eca20964951da6d8176b/8517e9828eb7eca20964951da6d8176b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346055.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/76b/8517e9828eb7eca20964951da6d8176b/8517e9828eb7eca20964951da6d8176b_1717346055.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/56f/9a8b64160e6ae421965eb79c4a6cc56f/9a8b64160e6ae421965eb79c4a6cc56f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346059.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/56f/9a8b64160e6ae421965eb79c4a6cc56f/9a8b64160e6ae421965eb79c4a6cc56f_1717346059.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/c7a/87b0db4a3a68a6cad2f717ee23cacc7a/87b0db4a3a68a6cad2f717ee23cacc7a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346063.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/c7a/87b0db4a3a68a6cad2f717ee23cacc7a/87b0db4a3a68a6cad2f717ee23cacc7a_1717346063.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/0eb/8bffce534889c3e9760d4e4c7707f0eb/8bffce534889c3e9760d4e4c7707f0eb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346067.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/0eb/8bffce534889c3e9760d4e4c7707f0eb/8bffce534889c3e9760d4e4c7707f0eb_1717346067.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/32d/2293f80a7b9db3f5f9ab86f42c9cb32d/2293f80a7b9db3f5f9ab86f42c9cb32d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346071.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/32d/2293f80a7b9db3f5f9ab86f42c9cb32d/2293f80a7b9db3f5f9ab86f42c9cb32d_1717346071.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/256/8c28cad3e3414d4c0358c374c3b65256/8c28cad3e3414d4c0358c374c3b65256.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346075.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/256/8c28cad3e3414d4c0358c374c3b65256/8c28cad3e3414d4c0358c374c3b65256_1717346075.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7mgcp8r3/sda/containers/0/078/6433e58e0348b3f47247aa2b74b0e078/6433e58e0348b3f47247aa2b74b0e078.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346079.00000-0, db: /tmp/tmp7mgcp8r3/sda/containers/0/078/6433e58e0348b3f47247aa2b74b0e078/6433e58e0348b3f47247aa2b74b0e078_1717346079.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345578.00000-0, db: /tmp/tmpjxqtne86/sda/containers/0/f5a/08d426fb17444b5307e0c9bf53ee0f5a/08d426fb17444b5307e0c9bf53ee0f5a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345582.00000-0, db: /tmp/tmpjxqtne86/sda/containers/0/6b0/c31b1b1cf10ab8381f658325d79186b0/c31b1b1cf10ab8381f658325d79186b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmpjxqtne86/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345590.00000-0, db: /tmp/tmpjxqtne86/sda/containers/0/665/d1b77861918be119806951a3d5f2c665/d1b77861918be119806951a3d5f2c665.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345579.00000-0, db: /tmp/tmpj2sqsh7b/sda/containers/0/6aa/3b392815654a2ad1c6ac97c1a9e016aa/3b392815654a2ad1c6ac97c1a9e016aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345582.00000-0, db: /tmp/tmpj2sqsh7b/sda/containers/0/6b0/c31b1b1cf10ab8381f658325d79186b0/c31b1b1cf10ab8381f658325d79186b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345585.00000-0, db: /tmp/tmpj2sqsh7b/sda/containers/0/6ab/d0509d170b0f1e444db30456185ae6ab/d0509d170b0f1e444db30456185ae6ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345588.00000-0, db: /tmp/tmpj2sqsh7b/sda/containers/0/0f7/a399e0156f613b9af5986c8db6f180f7/a399e0156f613b9af5986c8db6f180f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345588.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/0f7/a399e0156f613b9af5986c8db6f180f7/a399e0156f613b9af5986c8db6f180f7.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345588.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/0f7/a399e0156f613b9af5986c8db6f180f7/a399e0156f613b9af5986c8db6f180f7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345588.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/0f7/a399e0156f613b9af5986c8db6f180f7/a399e0156f613b9af5986c8db6f180f7.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345592.00000-0, db: /tmp/tmpd7nunbof/sda/containers/0/7ae/33a90cc113bf7ca0579295c49d1cc7ae/33a90cc113bf7ca0579295c49d1cc7ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmpgjhsfkgq/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345583.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/65f/daac403bcace00efd4c4a7022de5665f/daac403bcace00efd4c4a7022de5665f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345589.00000-0, db: /tmp/tmp6muohoh8/sda/containers/0/42f/1ecef9ec974d61152562b31a2e08842f/1ecef9ec974d61152562b31a2e08842f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345580.00000-0, db: /tmp/tmppxdoaykc/sda/containers/0/2da/e5ea683ee399a419a92ac036afbec2da/e5ea683ee399a419a92ac036afbec2da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345582.00000-0, db: /tmp/tmppxdoaykc/sda/containers/0/6b0/c31b1b1cf10ab8381f658325d79186b0/c31b1b1cf10ab8381f658325d79186b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmppxdoaykc/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345586.00000-0, db: /tmp/tmppxdoaykc/sda/containers/0/ad1/4eabd9bcb176b9a14fe12f979b407ad1/4eabd9bcb176b9a14fe12f979b407ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345581.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/e5e/bd17dc7507cafd8287d6ddd9b16f1e5e/bd17dc7507cafd8287d6ddd9b16f1e5e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345581.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/e5e/bd17dc7507cafd8287d6ddd9b16f1e5e/bd17dc7507cafd8287d6ddd9b16f1e5e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345584.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/457/7ee25d64b39f874cc8f69f050921d457/7ee25d64b39f874cc8f69f050921d457.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345587.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345587.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/f2e/e41ec7f0645cd944e45891e1c22d6f2e/e41ec7f0645cd944e45891e1c22d6f2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345590.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/665/d1b77861918be119806951a3d5f2c665/d1b77861918be119806951a3d5f2c665.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345590.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/665/d1b77861918be119806951a3d5f2c665/d1b77861918be119806951a3d5f2c665.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345593.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d43/17fcd2db99fb1128cb59b33b9c5f5d43/17fcd2db99fb1128cb59b33b9c5f5d43.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345593.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d43/17fcd2db99fb1128cb59b33b9c5f5d43/17fcd2db99fb1128cb59b33b9c5f5d43.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345596.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b66/ee856aabe3617cfbe35c1cbbebfdab66/ee856aabe3617cfbe35c1cbbebfdab66.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345596.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b66/ee856aabe3617cfbe35c1cbbebfdab66/ee856aabe3617cfbe35c1cbbebfdab66.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345599.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/4fb/9d8841baa57f13426dbe68220cad44fb/9d8841baa57f13426dbe68220cad44fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345602.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/902/ba5206a6f0f12285c7483a3abbdb7902/ba5206a6f0f12285c7483a3abbdb7902.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345602.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/902/ba5206a6f0f12285c7483a3abbdb7902/ba5206a6f0f12285c7483a3abbdb7902.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345605.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/9a2/21d48763d166823a83f7bccfddde79a2/21d48763d166823a83f7bccfddde79a2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345605.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/9a2/21d48763d166823a83f7bccfddde79a2/21d48763d166823a83f7bccfddde79a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345608.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/794/cf237200e87816fe2a6902a0db1be794/cf237200e87816fe2a6902a0db1be794.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345608.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/794/cf237200e87816fe2a6902a0db1be794/cf237200e87816fe2a6902a0db1be794.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345611.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345611.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/4b2/f90618be37dc41f99ef3145f4a8b44b2/f90618be37dc41f99ef3145f4a8b44b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345614.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/cf0/788374a945691d4e5292fcb2c28bacf0/788374a945691d4e5292fcb2c28bacf0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345614.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/cf0/788374a945691d4e5292fcb2c28bacf0/788374a945691d4e5292fcb2c28bacf0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345617.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/6ee/b61d7eb326b5bd22579e455a028c76ee/b61d7eb326b5bd22579e455a028c76ee.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345617.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/6ee/b61d7eb326b5bd22579e455a028c76ee/b61d7eb326b5bd22579e455a028c76ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345620.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/32d/89aa309dc2bc18859f87bfdf186f232d/89aa309dc2bc18859f87bfdf186f232d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345620.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/32d/89aa309dc2bc18859f87bfdf186f232d/89aa309dc2bc18859f87bfdf186f232d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345623.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b53/3a32da154dca9342515364d036b58b53/3a32da154dca9342515364d036b58b53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345626.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/e82/fb71348258fba5121301995547140e82/fb71348258fba5121301995547140e82.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345626.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/e82/fb71348258fba5121301995547140e82/fb71348258fba5121301995547140e82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345629.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/e8d/4d1360234011af8a8d9288535f233e8d/4d1360234011af8a8d9288535f233e8d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345629.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/e8d/4d1360234011af8a8d9288535f233e8d/4d1360234011af8a8d9288535f233e8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345632.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c42/f25709b5fba76fbae88631d8a2559c42/f25709b5fba76fbae88631d8a2559c42.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345632.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c42/f25709b5fba76fbae88631d8a2559c42/f25709b5fba76fbae88631d8a2559c42.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345635.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/58d/920fc4bb0593a4ed173f29f65291158d/920fc4bb0593a4ed173f29f65291158d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345638.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/772/c5c8cb7a45a94eb9f31df844973bc772/c5c8cb7a45a94eb9f31df844973bc772.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345638.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/772/c5c8cb7a45a94eb9f31df844973bc772/c5c8cb7a45a94eb9f31df844973bc772.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345641.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/024/ef7fb1418739317889300ac6cc0fa024/ef7fb1418739317889300ac6cc0fa024.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345641.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/024/ef7fb1418739317889300ac6cc0fa024/ef7fb1418739317889300ac6cc0fa024.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345644.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/634/2e0e75e90c4f4e41c73240d181a5d634/2e0e75e90c4f4e41c73240d181a5d634.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345644.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/634/2e0e75e90c4f4e41c73240d181a5d634/2e0e75e90c4f4e41c73240d181a5d634.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345647.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345647.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b66/9950be34c21aed868feec352e5ac0b66/9950be34c21aed868feec352e5ac0b66.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345650.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d8a/93ac8b776635264cfbf2799e40604d8a/93ac8b776635264cfbf2799e40604d8a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345650.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d8a/93ac8b776635264cfbf2799e40604d8a/93ac8b776635264cfbf2799e40604d8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345653.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/87e/56720e73abbf79e0ee7201c21a94f87e/56720e73abbf79e0ee7201c21a94f87e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345653.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/87e/56720e73abbf79e0ee7201c21a94f87e/56720e73abbf79e0ee7201c21a94f87e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345656.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b3e/7e951f854c92d595ff5ed3a128809b3e/7e951f854c92d595ff5ed3a128809b3e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345656.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b3e/7e951f854c92d595ff5ed3a128809b3e/7e951f854c92d595ff5ed3a128809b3e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345659.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/da5/5843f1ec529fc374c6f56b5d06e0cda5/5843f1ec529fc374c6f56b5d06e0cda5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345662.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b42/e116ce37ada6e141b5aa7c8c4a7f0b42/e116ce37ada6e141b5aa7c8c4a7f0b42.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345662.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b42/e116ce37ada6e141b5aa7c8c4a7f0b42/e116ce37ada6e141b5aa7c8c4a7f0b42.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345665.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d35/4416bf20dce847d9330869aa30b6cd35/4416bf20dce847d9330869aa30b6cd35.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345665.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d35/4416bf20dce847d9330869aa30b6cd35/4416bf20dce847d9330869aa30b6cd35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345668.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/1cc/d21c11f36bc12891d59b072b3cbe81cc/d21c11f36bc12891d59b072b3cbe81cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345668.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/1cc/d21c11f36bc12891d59b072b3cbe81cc/d21c11f36bc12891d59b072b3cbe81cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345671.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345671.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d1e/8e02bfefe21c2b3f2242145aa20b4d1e/8e02bfefe21c2b3f2242145aa20b4d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345674.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/cb8/702c6aa0a659067e36bf1d811c9bbcb8/702c6aa0a659067e36bf1d811c9bbcb8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345674.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/cb8/702c6aa0a659067e36bf1d811c9bbcb8/702c6aa0a659067e36bf1d811c9bbcb8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345677.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/5e0/1f1490949b36ea4d2408d2e0ece365e0/1f1490949b36ea4d2408d2e0ece365e0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345677.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/5e0/1f1490949b36ea4d2408d2e0ece365e0/1f1490949b36ea4d2408d2e0ece365e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345680.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/fd4/e112f233af5e69d5723623c9c8973fd4/e112f233af5e69d5723623c9c8973fd4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345680.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/fd4/e112f233af5e69d5723623c9c8973fd4/e112f233af5e69d5723623c9c8973fd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345683.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345683.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/a90/c3c8c9a78928e1f2a8182eda7e1a5a90/c3c8c9a78928e1f2a8182eda7e1a5a90.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345686.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c0d/b21310a0ef2c08d387896222d61b4c0d/b21310a0ef2c08d387896222d61b4c0d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345686.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c0d/b21310a0ef2c08d387896222d61b4c0d/b21310a0ef2c08d387896222d61b4c0d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345689.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c0c/8f3905bfc9b21e32752ae32385a61c0c/8f3905bfc9b21e32752ae32385a61c0c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345689.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c0c/8f3905bfc9b21e32752ae32385a61c0c/8f3905bfc9b21e32752ae32385a61c0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345692.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/40f/dd0ec7ae854058eac1f8c42325c2840f/dd0ec7ae854058eac1f8c42325c2840f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345692.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/40f/dd0ec7ae854058eac1f8c42325c2840f/dd0ec7ae854058eac1f8c42325c2840f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345695.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/ab2/36bf26f4fdf94080c622ed2668a98ab2/36bf26f4fdf94080c622ed2668a98ab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345698.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/ec9/520de35a13ba0a5bedf8195b50e4eec9/520de35a13ba0a5bedf8195b50e4eec9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345698.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/ec9/520de35a13ba0a5bedf8195b50e4eec9/520de35a13ba0a5bedf8195b50e4eec9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345701.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c0d/2e3bc36c174cc8af41496f4175a5fc0d/2e3bc36c174cc8af41496f4175a5fc0d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345701.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/c0d/2e3bc36c174cc8af41496f4175a5fc0d/2e3bc36c174cc8af41496f4175a5fc0d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345704.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/647/f1e9e80e5e299c1c57756217e0325647/f1e9e80e5e299c1c57756217e0325647.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345704.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/647/f1e9e80e5e299c1c57756217e0325647/f1e9e80e5e299c1c57756217e0325647.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345707.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345707.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/d82/945e321e19e3a44f2fe3fc617cc21d82/945e321e19e3a44f2fe3fc617cc21d82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345710.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b17/c79d9c4aa512a13a70989135029bbb17/c79d9c4aa512a13a70989135029bbb17.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345710.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b17/c79d9c4aa512a13a70989135029bbb17/c79d9c4aa512a13a70989135029bbb17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345713.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/3ec/f1df66959dc7e5114b129ffd10ba03ec/f1df66959dc7e5114b129ffd10ba03ec.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345713.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/3ec/f1df66959dc7e5114b129ffd10ba03ec/f1df66959dc7e5114b129ffd10ba03ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345716.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/7fb/6f44efff34033fc1a4544b5d858a07fb/6f44efff34033fc1a4544b5d858a07fb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345716.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/7fb/6f44efff34033fc1a4544b5d858a07fb/6f44efff34033fc1a4544b5d858a07fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345719.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/b48/2d49774c6869a825f6adc9aca2b6cb48/2d49774c6869a825f6adc9aca2b6cb48.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345722.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/109/a4255094feedfe0aeafbe649a84df109/a4255094feedfe0aeafbe649a84df109.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345722.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/109/a4255094feedfe0aeafbe649a84df109/a4255094feedfe0aeafbe649a84df109.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345725.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/3de/ea4c6ecdba57bedb47dac12c90c5f3de/ea4c6ecdba57bedb47dac12c90c5f3de.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345725.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/3de/ea4c6ecdba57bedb47dac12c90c5f3de/ea4c6ecdba57bedb47dac12c90c5f3de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345728.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/63e/bd8eb1cbe6d9142364d634b06b7a763e/bd8eb1cbe6d9142364d634b06b7a763e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345728.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/63e/bd8eb1cbe6d9142364d634b06b7a763e/bd8eb1cbe6d9142364d634b06b7a763e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345734.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/266/dae58062b96935b774e503823f769266/dae58062b96935b774e503823f769266.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345734.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/266/dae58062b96935b774e503823f769266/dae58062b96935b774e503823f769266.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345737.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/7dc/81e40bdaa90b574f8adda54c0c7747dc/81e40bdaa90b574f8adda54c0c7747dc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345737.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/7dc/81e40bdaa90b574f8adda54c0c7747dc/81e40bdaa90b574f8adda54c0c7747dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345740.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/063/2b96708c5e6d033025fbc861d30ee063/2b96708c5e6d033025fbc861d30ee063.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345740.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/063/2b96708c5e6d033025fbc861d30ee063/2b96708c5e6d033025fbc861d30ee063.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345743.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_check_node test.unit.container.test_sharder.TestSharder.test_check_node ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345743.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/089/654127a78e9c666f60e97be99b817089/654127a78e9c666f60e97be99b817089.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345746.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/288/6421fbddb250ac23384da03a1e4f2288/6421fbddb250ac23384da03a1e4f2288.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345746.00000-0, db: /tmp/tmpwjp58im_/sda/containers/0/288/6421fbddb250ac23384da03a1e4f2288/6421fbddb250ac23384da03a1e4f2288.db sharder-test WARNING: Skipping d100 as it is not mounted sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00028514862060546875s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016427040100097656s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_0 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpeuiqex1g/sda/containers/0/b85/9d7a328814f4e99b3c0e6a0597883b85/9d7a328814f4e99b3c0e6a0597883b85.db, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Cleaved ShardRange in 0.009s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002765655517578125s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000152587890625s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpeuiqex1g/sda/containers/0/069/5a8214a697fa9e61b996df73488a7069/5a8214a697fa9e61b996df73488a7069.db, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication ... ok sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: misplaced object source bounds [('', 'obj004')], path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017523765563964844s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00010347366333007812s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017499923706054688s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002703666687011719s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016164779663085938s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345350.88339, (5, 5) as of 1717345578.84895, cleaved as of 1717345350.88339> in 0.035s, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpeuiqex1g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.47962.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002567768096923828s, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0002219676971435547s, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test INFO: Cleaved ShardRange in 0.033s, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024199485778808594s, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0001780986785888672s, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717345350.88339, (1, 6144) as of 1717345579.11332, cleaved as of 1717345350.88339> in 0.034s, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpj0iv8uld/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345578.91853.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.000278472900390625s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016117095947265625s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaved ShardRange in 0.034s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: got 6 rows (deleted=False) in 0.00027871131896972656s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015425682067871094s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345350.88339, (6, 6) as of 1717345579.36546, cleaved as of 1717345350.88339> in 0.026s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'ade2300f-183b-44c6-bcee-8f71c4285ff6-sda', 'cursor': '', 'max_row': 12, 'cleave_to_row': 10, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Completed cleaving, DB remaining in sharding state, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db test.unit.container.test_sharder.TestSharder.test_cleave_repeated test.unit.container.test_sharder.TestSharder.test_cleave_repeated ... ok sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaving from row 10 into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002415180206298828s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001537799835205078s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaved ShardRange in 0.019s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaving from row 10 into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345350.88339, (6, 6) as of 1717345579.36546, cleaved as of 1717345350.88339>, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024366378784179688s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015735626220703125s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345350.88339, (6, 6) as of 1717345579.36546, cleaved as of 1717345350.88339> in 0.033s, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp1xj9voki/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.17698.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002605915069580078s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016188621520996094s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaved ShardRange in 0.023s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, found as of 1717345350.88339>, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.000293731689453125s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014543533325195312s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpi_zkrpud/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - shard db already in sync, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717345350.88339, (3, 6) as of 1717345580.28711, cleaved as of 1717345350.88339> in 0.007s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002384185791015625s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014734268188476562s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717345350.88339, (1, 100) as of 1717345580.33599, cleaved as of 1717345350.88339> in 0.019s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002384185791015625s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00015211105346679688s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717345350.88339, (1, 1000) as of 1717345580.41463, cleaved as of 1717345350.88339> in 0.024s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717345350.88339, (2, 15) as of 1717345580.45165, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002334117889404297s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013327598571777344s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db test.unit.container.test_sharder.TestSharder.test_cleave_root test.unit.container.test_sharder.TestSharder.test_cleave_root ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges ... ok sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717345350.88339, (2, 15) as of 1717345580.45165, active as of 1717345350.88339> in 0.014s, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmpi_zkrpud/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345579.61803.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022602081298828125s, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001373291015625s, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test INFO: Cleaved ShardRange in 0.015s, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpolkzlq9x/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.54217.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002257823944091797s, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000133514404296875s, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003249645233154297s, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017690658569335938s, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'d' to 'x' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - zero objects found, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'x' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000301361083984375s, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001766681671142578s, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'x' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - zero objects found, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpry1_2xs0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.67626.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003426074981689453s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020647048950195312s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012922286987304688s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001938343048095703s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaved ShardRange in 0.03s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, found as of 1717345350.88339>, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003185272216796875s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001480579376220703s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000164031982421875s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016427040100097656s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019168853759765625s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpst56n1i3/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - shard db already in sync, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717345350.88339, (3, 6) as of 1717345581.26421, cleaved as of 1717345350.88339> in 0.01s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one ... ok sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003025531768798828s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00011587142944335938s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018310546875s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717345350.88339, (1, 100) as of 1717345581.32858, cleaved as of 1717345350.88339> in 0.023s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00033855438232421875s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012612342834472656s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.000202178955078125s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00010561943054199219s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717345350.88339, (1, 1000) as of 1717345581.42324, cleaved as of 1717345350.88339> in 0.03s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717345350.88339, (2, 15) as of 1717345581.46564, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002815723419189453s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010251998901367188s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016117095947265625s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717345350.88339, (2, 15) as of 1717345581.46564, active as of 1717345350.88339> in 0.019s, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmpst56n1i3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345580.92039.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change ... ok sharder-test DEBUG: got 4 rows (deleted=False) in 0.00033926963806152344s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018453598022460938s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Cleaved ShardRange in 0.022s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717345350.88339, (500000, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003571510314941406s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002033710479736328s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_x_1 for ShardRange, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Cleaved ShardRange<'d' to 'x' as of 1717345350.88339, (2, 103) as of 1717345581.76356, cleaved as of 1717345350.88339> in 0.027s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: Continuing to cleave (2 done, 1 todo), path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'d' to MaxBound as of 1717345581.80298, (500000, 0) as of 1717345581.80298, created as of 1717345581.80298>, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003314018249511719s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001895427703857422s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__2 for ShardRange, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test INFO: Cleaved ShardRange<'d' to MaxBound as of 1717345581.80298, (3, 1103) as of 1717345581.86987, cleaved as of 1717345581.80298> in 0.037s, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpe7rq6rdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345581.57084.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003883838653564453s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000125885009765625s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019407272338867188s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/c_here_0 in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpg3psp9t1/sda/containers/0/e19/9e88c3bdfd03a830d1fc09d35d671e19/9e88c3bdfd03a830d1fc09d35d671e19.db, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018024444580078125s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00019621849060058594s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010728836059570312s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00013136863708496094s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db test.unit.container.test_sharder.TestSharder.test_cleave_shard test.unit.container.test_sharder.TestSharder.test_cleave_shard ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range ... ok sharder-test INFO: Cleaving from row None into .shards_a/c_there_0 for ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00032830238342285156s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001690387725830078s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_there_0 for ShardRange<'here' to 'where' as of 1717345581.90873, (0, 0) as of 1717345581.90873, sharding as of 1717345581.90873>, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717345350.88339, (3, 6) as of 1717345582.11181, cleaved as of 1717345350.88339> in 0.022s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00030684471130371094s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00012445449829101562s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001938343048095703s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018310546875s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018143653869628906s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_1 for ShardRange<'there' to 'where' as of 1717345350.88339, (2, 15) as of 1717345582.14257, cleaved as of 1717345350.88339>, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00029540061950683594s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001685619354248047s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_where_1 for ShardRange<'here' to 'where' as of 1717345581.90873, (0, 0) as of 1717345581.90873, sharding as of 1717345581.90873>, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717345350.88339, (2, 15) as of 1717345582.14257, cleaved as of 1717345350.88339> in 0.017s, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpg3psp9t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345581.90874.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6lqukfyd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021719932556152344s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012159347534179688s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012087821960449219s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011944770812988281s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_1_ for ShardRange<'h' to MaxBound as of 1717345582.39378, (1000, 11111) as of 1717345582.39378, created as of 1717345582.39378>, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002510547637939453s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013828277587890625s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_1_ for ShardRange<'h' to 'w' as of 1717345605.00000, (0, 0) as of 1717345594.00000, shrunk as of 1717345594.00000>, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717345582.39378, (1000, 11111) as of 1717345582.39378, created as of 1717345582.39378> in 0.023s, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_1, db: /tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345595.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000209808349609375s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012111663818359375s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001277923583984375s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001571178436279297s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_2_ for ShardRange<'h' to MaxBound as of 1717345582.54435, (1000, 11111) as of 1717345582.54435, cleaved as of 1717345582.54435>, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00025200843811035156s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014638900756835938s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_2_ for ShardRange<'h' to 'w' as of 1717345632.00000, (0, 0) as of 1717345621.00000, shrunk as of 1717345621.00000>, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717345582.54435, (1000, 11111) as of 1717345582.54435, cleaved as of 1717345582.54435> in 0.024s, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_2, db: /tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345622.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020933151245117188s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012087821960449219s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012373924255371094s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001266002655029297s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_3_ for ShardRange<'h' to MaxBound as of 1717345582.69685, (1000, 11111) as of 1717345582.69685, active as of 1717345582.69685>, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002524852752685547s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000141143798828125s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_3_ for ShardRange<'h' to 'w' as of 1717345659.00000, (0, 0) as of 1717345648.00000, shrunk as of 1717345648.00000>, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717345582.69685, (1000, 11111) as of 1717345582.69685, active as of 1717345582.69685> in 0.024s, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_3, db: /tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345649.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020933151245117188s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011968612670898438s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012159347534179688s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001232624053955078s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_k for ShardRange<'d' to 'k' as of 1717345582.84892, (1000, 11111) as of 1717345582.84892, created as of 1717345582.84892>, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024271011352539062s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001556873321533203s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_k for ShardRange<'h' to 'w' as of 1717345675.00000, (0, 0) as of 1717345675.00000, shrinking as of 1717345675.00000>, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345582.84892, (1000, 11111) as of 1717345582.84892, created as of 1717345582.84892> in 0.024s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_ for ShardRange<'k' to MaxBound as of 1717345582.84897, (1000, 11111) as of 1717345582.84897, created as of 1717345582.84897>, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002453327178955078s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001347064971923828s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_ for ShardRange<'h' to 'w' as of 1717345675.00000, (0, 0) as of 1717345675.00000, shrinking as of 1717345675.00000>, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717345582.84897, (1000, 11111) as of 1717345582.84897, created as of 1717345582.84897> in 0.016s, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_4, db: /tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717345676.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020647048950195312s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012159347534179688s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015997886657714844s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_k for ShardRange<'d' to 'k' as of 1717345583.05228, (1000, 11111) as of 1717345583.05228, cleaved as of 1717345583.05228>, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024700164794921875s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001518726348876953s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_k for ShardRange<'h' to 'w' as of 1717345708.00000, (0, 0) as of 1717345708.00000, shrinking as of 1717345708.00000>, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345583.05228, (1000, 11111) as of 1717345583.05228, cleaved as of 1717345583.05228> in 0.018s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_ for ShardRange<'k' to MaxBound as of 1717345583.05234, (1000, 11111) as of 1717345583.05234, cleaved as of 1717345583.05234>, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.000240325927734375s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012922286987304688s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_ for ShardRange<'h' to 'w' as of 1717345708.00000, (0, 0) as of 1717345708.00000, shrinking as of 1717345708.00000>, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717345583.05234, (1000, 11111) as of 1717345583.05234, cleaved as of 1717345583.05234> in 0.015s, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_5, db: /tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717345709.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020503997802734375s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012159347534179688s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000125885009765625s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012373924255371094s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_k for ShardRange<'d' to 'k' as of 1717345583.25013, (1000, 11111) as of 1717345583.25013, active as of 1717345583.25013>, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024771690368652344s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001442432403564453s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_k for ShardRange<'h' to 'w' as of 1717345741.00000, (0, 0) as of 1717345741.00000, shrinking as of 1717345741.00000>, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345583.25013, (1000, 11111) as of 1717345583.25013, active as of 1717345583.25013> in 0.015s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_ for ShardRange<'k' to MaxBound as of 1717345583.25018, (1000, 11111) as of 1717345583.25018, active as of 1717345583.25018>, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024509429931640625s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013875961303710938s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_ for ShardRange<'h' to 'w' as of 1717345741.00000, (0, 0) as of 1717345741.00000, shrinking as of 1717345741.00000>, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717345583.25018, (1000, 11111) as of 1717345583.25018, active as of 1717345583.25018> in 0.015s, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_6, db: /tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717345742.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020813941955566406s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012350082397460938s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017023086547851562s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001232624053955078s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_k for ShardRange<'d' to 'k' as of 1717345583.43147, (1000, 11111) as of 1717345583.43147, cleaved as of 1717345583.43147>, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028252601623535156s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000148773193359375s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_k for ShardRange<'h' to 'w' as of 1717345774.00000, (0, 0) as of 1717345774.00000, shrinking as of 1717345774.00000>, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345583.43147, (1000, 11111) as of 1717345583.43147, cleaved as of 1717345583.43147> in 0.016s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_t for ShardRange<'k' to 't' as of 1717345583.43152, (1000, 11111) as of 1717345583.43152, cleaved as of 1717345583.43152>, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023818016052246094s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001385211944580078s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_t for ShardRange<'h' to 'w' as of 1717345774.00000, (0, 0) as of 1717345774.00000, shrinking as of 1717345774.00000>, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345583.43152, (1000, 11111) as of 1717345583.43152, cleaved as of 1717345583.43152> in 0.015s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_ for ShardRange<'t' to MaxBound as of 1717345583.43155, (1000, 11111) as of 1717345583.43155, cleaved as of 1717345583.43155>, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002288818359375s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013399124145507812s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_ for ShardRange<'h' to 'w' as of 1717345774.00000, (0, 0) as of 1717345774.00000, shrinking as of 1717345774.00000>, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717345583.43155, (1000, 11111) as of 1717345583.43155, cleaved as of 1717345583.43155> in 0.015s, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_7, db: /tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717345775.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002162456512451172s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015687942504882812s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001316070556640625s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001251697540283203s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_k for ShardRange<'d' to 'k' as of 1717345583.67776, (1000, 11111) as of 1717345583.67776, created as of 1717345583.67776>, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002455711364746094s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013756752014160156s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_k for ShardRange<'h' to 'w' as of 1717345813.00000, (0, 0) as of 1717345813.00000, shrinking as of 1717345813.00000>, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345583.67776, (1000, 11111) as of 1717345583.67776, created as of 1717345583.67776> in 0.015s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_t for ShardRange<'k' to 't' as of 1717345583.67781, (1000, 11111) as of 1717345583.67781, created as of 1717345583.67781>, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023221969604492188s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013208389282226562s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_t for ShardRange<'h' to 'w' as of 1717345813.00000, (0, 0) as of 1717345813.00000, shrinking as of 1717345813.00000>, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345583.67781, (1000, 11111) as of 1717345583.67781, created as of 1717345583.67781> in 0.016s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_ for ShardRange<'t' to MaxBound as of 1717345583.67784, (1000, 11111) as of 1717345583.67784, created as of 1717345583.67784>, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023055076599121094s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016450881958007812s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_ for ShardRange<'h' to 'w' as of 1717345813.00000, (0, 0) as of 1717345813.00000, shrinking as of 1717345813.00000>, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717345583.67784, (1000, 11111) as of 1717345583.67784, created as of 1717345583.67784> in 0.014s, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_8, db: /tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717345814.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002052783966064453s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001227855682373047s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012350082397460938s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012087821960449219s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_k for ShardRange<'d' to 'k' as of 1717345583.91775, (1000, 11111) as of 1717345583.91775, active as of 1717345583.91775>, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023984909057617188s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013971328735351562s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_k for ShardRange<'h' to 'w' as of 1717345852.00000, (0, 0) as of 1717345852.00000, shrinking as of 1717345852.00000>, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345583.91775, (1000, 11111) as of 1717345583.91775, active as of 1717345583.91775> in 0.016s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_t for ShardRange<'k' to 't' as of 1717345583.91781, (1000, 11111) as of 1717345583.91781, active as of 1717345583.91781>, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023412704467773438s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001347064971923828s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_t for ShardRange<'h' to 'w' as of 1717345852.00000, (0, 0) as of 1717345852.00000, shrinking as of 1717345852.00000>, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345583.91781, (1000, 11111) as of 1717345583.91781, active as of 1717345583.91781> in 0.021s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_ for ShardRange<'t' to MaxBound as of 1717345583.91784, (1000, 11111) as of 1717345583.91784, active as of 1717345583.91784>, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024056434631347656s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001456737518310547s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_ for ShardRange<'h' to 'w' as of 1717345852.00000, (0, 0) as of 1717345852.00000, shrinking as of 1717345852.00000>, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717345583.91784, (1000, 11111) as of 1717345583.91784, active as of 1717345583.91784> in 0.018s, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_9, db: /tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717345853.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpulo7ibtb/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020503997802734375s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012302398681640625s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013327598571777344s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012540817260742188s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_10_k for ShardRange<'d' to 'k' as of 1717345584.16580, (1000, 11111) as of 1717345584.16580, created as of 1717345584.16580>, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002377033233642578s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014472007751464844s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_10_k for ShardRange<'h' to 'w' as of 1717345891.00000, (0, 0) as of 1717345891.00000, shrinking as of 1717345891.00000>, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345584.16580, (1000, 11111) as of 1717345584.16580, created as of 1717345584.16580> in 0.015s, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '9d611714-eb75-4bfa-8674-491cc37c2200-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 0}, path: .shards_a/donor_10, db: /tmp/tmpulo7ibtb/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717345892.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002205371856689453s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012254714965820312s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001220703125s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012683868408203125s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_k for ShardRange<'d' to 'k' as of 1717345584.32198, (1000, 11111) as of 1717345584.32198, cleaved as of 1717345584.32198>, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002384185791015625s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014662742614746094s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_k for ShardRange<'h' to 'w' as of 1717345917.00000, (0, 0) as of 1717345917.00000, shrinking as of 1717345917.00000>, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345584.32198, (1000, 11111) as of 1717345584.32198, cleaved as of 1717345584.32198> in 0.029s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_t for ShardRange<'k' to 't' as of 1717345584.32203, (1000, 11111) as of 1717345584.32203, cleaved as of 1717345584.32203>, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023484230041503906s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001392364501953125s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_t for ShardRange<'h' to 'w' as of 1717345917.00000, (0, 0) as of 1717345917.00000, shrinking as of 1717345917.00000>, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345584.32203, (1000, 11111) as of 1717345584.32203, cleaved as of 1717345584.32203> in 0.015s, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e7205e9d-c80d-45fa-8d80-e60a083beeb8-sda', 'cursor': 't', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: .shards_a/donor_11, db: /tmp/tmpulo7ibtb/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717345918.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020313262939453125s, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000125885009765625s, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012540817260742188s, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012350082397460938s, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking ... ok sharder-test WARNING: Repeat cleaving required, context: {'ref': '6ec4ecd5-4a76-46b3-9bc2-1dd6dc137441-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 1}, path: .shards_a/donor_12, db: /tmp/tmpulo7ibtb/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717345950.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021004676818847656s, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012159347534179688s, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001201629638671875s, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011897087097167969s, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'cdbd2fe4-156d-4bd7-8ffc-0dca71f23693-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 2}, path: .shards_a/donor_13, db: /tmp/tmpulo7ibtb/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717345970.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002048015594482422s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011849403381347656s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016117095947265625s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012087821960449219s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_14_k for ShardRange<'d' to 'k' as of 1717345584.72627, (1000, 11111) as of 1717345584.72627, cleaved as of 1717345584.72627>, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000244140625s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015616416931152344s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_14_k for ShardRange<'h' to 'w' as of 1717345989.00000, (0, 0) as of 1717345989.00000, shrinking as of 1717345989.00000>, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345584.72627, (1000, 11111) as of 1717345584.72627, cleaved as of 1717345584.72627> in 0.016s, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test INFO: Stopped cleave at gap: 'k' - 't', path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '944d081c-9a8b-4dbd-811c-66dc690d2f4f-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 1}, path: .shards_a/donor_14, db: /tmp/tmpulo7ibtb/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717345990.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020503997802734375s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011682510375976562s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015282630920410156s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001239776611328125s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Cleaving from row None into .shards/other for ShardRange<'here' to 'where' as of 1717345587.00000, (0, 0) as of 1717345587.00000, active as of 1717345587.00000>, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023746490478515625s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001354217529296875s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards/other for ShardRange<'here' to 'there' as of 1717345584.95277, (0, 0) as of 1717345585.00000, shrunk as of 1717345585.00000>, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Cleaved ShardRange<'here' to 'where' as of 1717345587.00000, (0, 0) as of 1717345587.00000, active as of 1717345587.00000> in 0.019s, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp_24ig9ek/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002048015594482422s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011992454528808594s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012040138244628906s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001246929168701172s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Cleaving from row None into .shards_a/shard_d for ShardRange<'here' to MaxBound as of 1717345587.00000, (0, 0) as of 1717345587.00000, active as of 1717345587.00000>, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002377033233642578s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013399124145507812s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/shard_d for ShardRange<'here' to 'there' as of 1717345585.00000, (0, 0) as of 1717345585.00000, sharding as of 1717345585.00000>, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Cleaved ShardRange<'here' to MaxBound as of 1717345587.00000, (0, 0) as of 1717345587.00000, active as of 1717345587.00000> in 0.015s, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp7wn2j3ur/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002658367156982422s, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001537799835205078s, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015735626220703125s, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016069412231445312s, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor ... ok sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002827644348144531s, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016188621520996094s, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp8xn0v7sm/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00026488304138183594s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0006132125854492188s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016689300537109375s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017452239990234375s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003032684326171875s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001735687255859375s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container a/c for ShardRange<'here' to 'there' as of 1717345585.40002, (0, 0) as of 1717345586.00000, shrunk as of 1717345586.00000>, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test INFO: Cleaved ShardRange in 0.02s, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpf44k7n86/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345587.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaving from row None into .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003261566162109375s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019669532775878906s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaved ShardRange in 0.025s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_2 for ShardRange<'b' to 'd' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003230571746826172s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017905235290527344s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found ... ok sharder-test INFO: Replicating new shard container .shards_a/c_d_2 for ShardRange, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaved ShardRange<'b' to 'd' as of 1717345350.88339, (2, 3) as of 1717345585.61555, cleaved as of 1717345350.88339> in 0.021s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Continuing to cleave (2 done, 2 todo), path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaving from row None into .shards_a/c_f_3 for ShardRange<'d' to 'f' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002884864807128906s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017905235290527344s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_f_3 for ShardRange, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaved ShardRange<'d' to 'f' as of 1717345350.88339, (2, 103) as of 1717345585.68662, cleaved as of 1717345350.88339> in 0.021s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Stopped cleave at unready ShardRange<'f' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, found as of 1717345350.88339>, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'f' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345585.71362>, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003082752227783203s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00020813941955566406s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test INFO: Cleaved ShardRange<'f' to MaxBound as of 1717345350.88339, (1, 1000) as of 1717345585.77928, cleaved as of 1717345585.71362> in 0.027s, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpjlmgl6jv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.43096.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003273487091064453s, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00022292137145996094s, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test INFO: Cleaved ShardRange in 0.025s, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00031256675720214844s, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics ... ok sharder-test DEBUG: got 2 rows (deleted=True) in 0.00023865699768066406s, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717345350.88339, (1, 18432) as of 1717345585.98723, cleaved as of 1717345350.88339> in 0.026s, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp4rgn300i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345585.82011.db REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, sync, (5, '23b6b64b04b0cc8003526977424b3151', 'eff760ed-f24b-439b-9e1e-9108fba37cfb-sdb', '1717345586.05791', '0000000000.00000', '0', '') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_items, ([{'ROWID': 1, 'name': 'obj002', 'created_at': '1717345588.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'obj003', 'created_at': '1717345589.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'obj004', 'created_at': '1717345590.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'obj005', 'created_at': '1717345591.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'obj006', 'created_at': '1717345592.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'eff760ed-f24b-439b-9e1e-9108fba37cfb-sdb') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_syncs, ([],) sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0003514289855957031s, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019812583923339844s, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaved ShardRange in 0.027s, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003299713134765625s, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018644332885742188s, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345350.88339, (3, 3) as of 1717345586.33086, cleaved as of 1717345350.88339> in 0.047s, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp215bbmfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range ... ok REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, sync, (7, '00000000000000000000000000000000', 'f969cf7d-b348-4e2c-b585-802f33b7c4b2-', '1717345586.03554', '0000000000.00000', '0', '{"X-Container-Sysmeta-Shard-Context-46b3e62a-502e-40e0-84d1-5aceccf803aa-sda": ["{\\"ref\\": \\"46b3e62a-502e-40e0-84d1-5aceccf803aa-sda\\", \\"cursor\\": \\"\\", \\"max_row\\": 7, \\"cleave_to_row\\": 7, \\"last_cleave_to_row\\": null, \\"cleaving_done\\": true, \\"misplaced_done\\": true, \\"ranges_done\\": 2, \\"ranges_todo\\": 0}", "1717345586.37888"]}') REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, merge_shard_ranges, ([{'name': '.shards_a/c_obj004_0', 'timestamp': '1717345350.88339', 'lower': '', 'upper': 'obj004', 'object_count': 4, 'bytes_used': 4, 'meta_timestamp': '1717345586.21518', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345350.88339', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/c__1', 'timestamp': '1717345350.88339', 'lower': 'obj004', 'upper': '', 'object_count': 3, 'bytes_used': 3, 'meta_timestamp': '1717345586.33086', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345350.88339', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345586.10982', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345586.38795', 'deleted': 0, 'state': 70, 'state_timestamp': '1717345586.10990', 'epoch': '1717345586.10990', 'reported': 0, 'tombstones': -1}], 'f969cf7d-b348-4e2c-b585-802f33b7c4b2-') sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000301361083984375s, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019502639770507812s, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaved ShardRange in 0.018s, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345350.88339, (3, 3) as of 1717345586.33086, active as of 1717345350.88339>, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00030994415283203125s, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018358230590820312s, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345350.88339, (3, 3) as of 1717345586.33086, active as of 1717345350.88339> in 0.021s, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp215bbmfl/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.10990.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmp113qtd06/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'a8db53e1-a77f-4684-ba74-16563b35b038-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpl43o1b8o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'a8db53e1-a77f-4684-ba74-16563b35b038-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpl43o1b8o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.00000.db test.unit.container.test_sharder.TestSharder.test_complete_sharding_root test.unit.container.test_sharder.TestSharder.test_complete_sharding_root ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap ... ok sharder-test WARNING: Repeat cleaving required, context: {'ref': 'a8db53e1-a77f-4684-ba74-16563b35b038-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpl43o1b8o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'a8db53e1-a77f-4684-ba74-16563b35b038-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpl43o1b8o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'f19d8686-3b91-4721-9420-3ba76a2482ba-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpl43o1b8o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3f1ad091-85f9-4044-a324-0c5c30604807-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmp7i6f31w5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3f1ad091-85f9-4044-a324-0c5c30604807-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmp7i6f31w5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3f1ad091-85f9-4044-a324-0c5c30604807-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmp7i6f31w5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3f1ad091-85f9-4044-a324-0c5c30604807-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmp7i6f31w5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '9d8a8ef6-6b93-4817-83e8-05443793a475-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmp7i6f31w5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345586.00000.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: TypeError('the JSON object must be str, bytes or bytearray, not dict'), path: a/c, db: /tmp/tmpk31_lv2y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: JSONDecodeError('Expecting value: line 1 column 1 (char 0)'), path: a/c, db: /tmp/tmpk31_lv2y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: ValueError('not a list'), path: a/c, db: /tmp/tmpk31_lv2y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('name'), path: a/c, db: /tmp/tmpk31_lv2y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('timestamp'), path: a/c, db: /tmp/tmpk31_lv2y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('object_count'), path: a/c, db: /tmp/tmpk31_lv2y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type None, path: a/c, db: /tmp/tmptrahuur2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'object', path: a/c, db: /tmp/tmptrahuur2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'disco', path: a/c, db: /tmp/tmptrahuur2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get shard ranges from a/c: Unexpected response: 404, path: a/c, db: /tmp/tmpq5ah8r_4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpidm7p81d/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmpidm7p81d/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Found 2 compactible sequences of length(s) [2, 2], path: a/c, db: /tmp/tmpidm7p81d/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpidm7p81d/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange<'c' to 'e' as of 1717345587.35230, (100000, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpidm7p81d/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp1fej3m8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp1fej3m8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp1fej3m8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpm9452smy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size ... ok test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part ... ok sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpjstj7hod/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345587.00000.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpwgqyobor/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpwgqyobor/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpwgqyobor/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: a/c, db: /tmp/tmpwgqyobor/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpwgqyobor/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpwgqyobor/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmpkq_08vom/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmpkq_08vom/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmpkq_08vom/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: .shards_a/c_, db: /tmp/tmpkq_08vom/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmpkq_08vom/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmpkq_08vom/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpuer_29zs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpuer_29zs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpuer_29zs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpuer_29zs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp4js9o4k_/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp4js9o4k_/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp4js9o4k_/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmp4js9o4k_/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmppfdsw2jk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmppfdsw2jk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmppfdsw2jk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmppfdsw2jk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmp3_j4kwcy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmp3_j4kwcy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmp3_j4kwcy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmp3_j4kwcy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp6uq448_x/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp6uq448_x/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp6uq448_x/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate ... ok test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats test.unit.container.test_sharder.TestSharder.test_increment_stats ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd ... ok sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:28 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:29 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test WARNING: Repeat cleaving required, context: {'ref': '1c10ff71-e040-4c87-ab95-3a35594aa22f-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmphl6g9e34/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T16:26:29.000000. DB state: sharding, own_shard_range state: sharding, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 2, 'active': 0}, path: a/c, db: /tmp/tmphl6g9e34/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T16:26:30.000000. DB state: sharding, own_shard_range state: shrinking, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 0, 'active': 1}, path: .shards_a/shard_c, db: /tmp/tmpo7jnpeqy/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345590.00000.db test.unit.container.test_sharder.TestSharder.test_init test.unit.container.test_sharder.TestSharder.test_init ... ok test.unit.container.test_sharder.TestSharder.test_init_deprecated_options test.unit.container.test_sharder.TestSharder.test_init_deprecated_options ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name ... ok test.unit.container.test_sharder.TestSharder.test_log_broker test.unit.container.test_sharder.TestSharder.test_log_broker ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception test.unit.container.test_sharder.TestSharder.test_log_broker_exception ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_levels test.unit.container.test_sharder.TestSharder.test_log_broker_levels ... ok sharder-test WARNING: shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test DEBUG: bonjour mes amis, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: bonjour mes amis, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: bonjour mes amis, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: bonjour mes amis, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpuurlp5nz/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: exception: test, path: a/c, db: /tmp/tmp3dfnc8q_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 448, in test_log_broker_exception raise ValueError('test') ValueError: test sharder-test DEBUG: bonjour mes amis, path: , db: /tmp/tmpfbdbtc7q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: bonjour mes amis, path: a/c, db: sharder-test INFO: bonjour mes amis, path: , db: /tmp/tmpfbdbtc7q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: bonjour mes amis, path: a/c, db: sharder-test WARNING: bonjour mes amis, path: , db: /tmp/tmpfbdbtc7q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: bonjour mes amis, path: a/c, db: sharder-test ERROR: bonjour mes amis, path: , db: /tmp/tmpfbdbtc7q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: bonjour mes amis, path: a/c, db: sharder-test DEBUG: Unlinked retiring db '/tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028443336486816406s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002799034118652344s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00030803680419921875s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003788471221923828s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003628730773925781s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002493858337402344s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002391338348388672s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002300739288330078s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00023794174194335938s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0005445480346679688s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00039649009704589844s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345589.44551, (0, 0) as of 1717345589.44551, active as of 1717345589.44551>, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345589.44555, (0, 0) as of 1717345589.44555, active as of 1717345589.44555>, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/there-where in state active, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345589.44561, (0, 0) as of 1717345589.44561, active as of 1717345589.44561>, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/where-yonder in state active, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000217437744140625s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019073486328125s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023055076599121094s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001971721649169922s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019621849060058594s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021123886108398438s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002129077911376953s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002040863037109375s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002117156982421875s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020360946655273438s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018644332885742188s, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345589.44555, (0, 0) as of 1717345589.44555, active as of 1717345589.44555>, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345589.44561, (0, 0) as of 1717345589.44561, active as of 1717345589.44561>, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0_c7huou/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.44987.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003154277801513672s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0003185272216796875s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002720355987548828s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002903938293457031s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00036597251892089844s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00032448768615722656s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025534629821777344s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002617835998535156s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002510547637939453s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026035308837890625s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00025200843811035156s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002465248107910156s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.000247955322265625s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002472400665283203s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00025653839111328125s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002498626708984375s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0003094673156738281s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002753734588623047s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024628639221191406s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023031234741210938s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022363662719726562s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002231597900390625s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002791881561279297s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002613067626953125s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002827644348144531s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023889541625976562s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002334117889404297s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002791881561279297s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002562999725341797s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023603439331054688s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023603439331054688s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00040912628173828125s, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp06xh_549/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345589.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00036263465881347656s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003902912139892578s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003285408020019531s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003304481506347656s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00038504600524902344s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025391578674316406s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002586841583251953s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00027680397033691406s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002605915069580078s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000396728515625s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00033092498779296875s, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345590.23382, (0, 0) as of 1717345590.23382, active as of 1717345590.23382>, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345590.23385, (0, 0) as of 1717345590.23385, active as of 1717345590.23385>, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345590.23388, (0, 0) as of 1717345590.23388, active as of 1717345590.23388>, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 1 required (not removing objects), shard db: /tmp/tmpfnogaq02/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfnogaq02/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.23833.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0004150867462158203s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003268718719482422s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003254413604736328s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas ... ok sharder-test DEBUG: got 2 rows (deleted=False) in 0.00034117698669433594s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00037217140197753906s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026154518127441406s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002491474151611328s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002338886260986328s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00025725364685058594s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0004096031188964844s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003654956817626953s, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345590.43641, (0, 0) as of 1717345590.43641, active as of 1717345590.43641>, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345590.43644, (0, 0) as of 1717345590.43644, active as of 1717345590.43644>, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/there-where in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpzqbhrvdw/sda/containers/0/170/533f29a8ddb31e55151765e09ccbb170/533f29a8ddb31e55151765e09ccbb170.db, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345590.43647, (0, 0) as of 1717345590.43647, active as of 1717345590.43647>, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzqbhrvdw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.44399.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00032901763916015625s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003228187561035156s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003211498260498047s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003490447998046875s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003750324249267578s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025963783264160156s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002586841583251953s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023484230041503906s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00035309791564941406s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00039196014404296875s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003185272216796875s, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345590.63492, (0, 0) as of 1717345590.63492, active as of 1717345590.63492>, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/here-there in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmplwi_4awr/sda/containers/0/ed5/9b40bf3cdc484061806b8d0a545d2ed5/9b40bf3cdc484061806b8d0a545d2ed5.db, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345590.63496, (0, 0) as of 1717345590.63496, active as of 1717345590.63496>, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345590.63499, (0, 0) as of 1717345590.63499, active as of 1717345590.63499>, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 2 required (not removing objects), shard db: /tmp/tmplwi_4awr/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects ... ok sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplwi_4awr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.63914.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00036787986755371094s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003685951232910156s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00031948089599609375s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003628730773925781s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00040793418884277344s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002589225769042969s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021576881408691406s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023293495178222656s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002570152282714844s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00042247772216796875s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003204345703125s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345590.81804, (0, 0) as of 1717345590.81804, active as of 1717345590.81804>, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345590.81808, (0, 0) as of 1717345590.81808, active as of 1717345590.81808>, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345590.81810, (0, 0) as of 1717345590.81810, active as of 1717345590.81810>, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025773048400878906s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002300739288330078s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002346038818359375s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000255584716796875s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002791881561279297s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002849102020263672s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022602081298828125s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002727508544921875s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002319812774658203s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023293495178222656s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002033710479736328s, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345590.81804, (0, 0) as of 1717345590.81804, active as of 1717345590.81804>, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp2woeoeic/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345590.82241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00034165382385253906s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002548694610595703s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028133392333984375s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025200843811035156s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002460479736328125s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003554821014404297s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00034356117248535156s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'yonder')], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00030303001403808594s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002636909484863281s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00026226043701171875s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024247169494628906s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002465248107910156s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002598762512207031s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003180503845214844s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00026702880859375s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002465248107910156s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002613067626953125s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00034618377685546875s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002951622009277344s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024175643920898438s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003380775451660156s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0003612041473388672s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00041937828063964844s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003199577331542969s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023508071899414062s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002396106719970703s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022530555725097656s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023674964904785156s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00039839744567871094s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00043272972106933594s, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'yonder' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Placed 3 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoitk1ups/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00035309791564941406s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002608299255371094s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003185272216796875s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003368854522705078s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00032711029052734375s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003502368927001953s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002658367156982422s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003376007080078125s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Placed 1 misplaced objects (2 unplaced), path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026702880859375s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00026106834411621094s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002734661102294922s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002593994140625s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002524852752685547s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002722740173339844s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026607513427734375s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002598762512207031s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00030159950256347656s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002474784851074219s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026535987854003906s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025391578674316406s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024628639221191406s, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Placed 5 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj3hiycwi/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345592.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00034427642822265625s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002048015594482422s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000202178955078125s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000240325927734375s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026106834411621094s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002605915069580078s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023746490478515625s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026726722717285156s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002503395080566406s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00025081634521484375s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002636909484863281s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022602081298828125s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002486705780029297s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002422332763671875s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002989768981933594s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022649765014648438s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002231597900390625s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022935867309570312s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002219676971435547s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002760887145996094s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002384185791015625s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00025773048400878906s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002532005310058594s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002460479736328125s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded ... ok sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024366378784179688s, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmvcfmjcr/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003409385681152344s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020122528076171875s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019121170043945312s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001957416534423828s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025725364685058594s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023889541625976562s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003814697265625s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021195411682128906s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023174285888671875s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002727508544921875s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002925395965576172s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022339820861816406s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002465248107910156s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018453598022460938s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002465248107910156s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002288818359375s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025391578674316406s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024199485778808594s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002613067626953125s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002524852752685547s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022721290588378906s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one ... ok sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002467632293701172s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023102760314941406s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023317337036132812s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023794174194335938s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002722740173339844s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002422332763671875s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002498626708984375s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021767616271972656s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002505779266357422s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024080276489257812s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002582073211669922s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002295970916748047s, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmps33c6e5y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00041222572326660156s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020766258239746094s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019431114196777344s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020074844360351562s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002703666687011719s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025343894958496094s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00041174888610839844s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023555755615234375s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002567768096923828s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023365020751953125s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024008750915527344s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002346038818359375s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00026702880859375s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025272369384765625s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002567768096923828s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002760887145996094s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023698806762695312s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002751350402832031s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022912025451660156s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023651123046875s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002684593200683594s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024247169494628906s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002353191375732422s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024127960205078125s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024318695068359375s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002493858337402344s, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, active as of 1717345350.88339>, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpvl34wbrn/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00036144256591796875s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00033402442932128906s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003342628479003906s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003752708435058594s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000362396240234375s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002627372741699219s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024199485778808594s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023937225341796875s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00025153160095214844s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003879070281982422s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003314018249511719s, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345592.68474, (0, 0) as of 1717345592.68474, active as of 1717345592.68474>, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345592.68478, (0, 0) as of 1717345592.68478, active as of 1717345592.68478>, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345592.68481, (0, 0) as of 1717345592.68481, active as of 1717345592.68481>, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmfwjp18o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345592.68924.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpumg84syf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpumg84syf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpumg84syf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpumg84syf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp6j2pnhbv/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp6j2pnhbv/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp6j2pnhbv/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmp6j2pnhbv/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp9m4fbt1r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp9m4fbt1r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp9m4fbt1r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp9m4fbt1r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmptddedp0h/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmptddedp0h/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmptddedp0h/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmptddedp0h/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:2 success:2 failure:0 skipped:1 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted test.unit.container.test_sharder.TestSharder.test_one_shard_cycle test.unit.container.test_sharder.TestSharder.test_one_shard_cycle ... ok sharder-test ERROR: Unhandled exception while processing: kapow!, path: a/c1, db: /tmp/tmpjkyosa6c/sdb/containers/0/ash/c1hash/c1hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2516, in _one_shard_cycle self._process_broker(broker, node, part) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1245, in _execute_mock_call result = effect(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 886, in mock_processing raise Exception('kapow!') Exception: kapow! sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:2 failure:1 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjkyosa6c/sda/containers/0/ash/c0hash/c0hash.db' sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Found no containers directories sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers ... ok sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.05s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 0) sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.01s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdc) sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.01s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdb, sdc) sharder-test INFO: Since Sun Jun 2 04:26:33 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:33 2024 audit_shard - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters ... ok test.unit.container.test_sharder.TestSharder.test_periodic_warning test.unit.container.test_sharder.TestSharder.test_periodic_warning ... ok test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard ... ok sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmp1g7gyp3p/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmp1g7gyp3p/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmp1g7gyp3p/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmp1g7gyp3p/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp59wg07yw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345593.76785.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmpx0uiqmhb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx0uiqmhb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx0uiqmhb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx0uiqmhb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded (deleted), path: a/c, db: /tmp/tmpx0uiqmhb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrinking' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharding' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharded' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others ... ok sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrunk' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpcbv8u1f3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpry6_mexb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003383159637451172s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015878677368164062s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00031566619873046875s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018930435180664062s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash60False/hash60False_1717345594.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003077983856201172s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001881122589111328s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002791881561279297s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017118453979492188s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash60True/hash60True_1717345613.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003273487091064453s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020194053649902344s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003275871276855469s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020122528076171875s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/lse/hash50False/hash50False_1717345632.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges ... ok sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002048015594482422s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012350082397460938s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020647048950195312s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012493133544921875s, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - zero objects found, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmprp5j4_st/sda/containers/0/rue/hash50True/hash50True_1717345648.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h60/hash60/hash60_1717345594.92114.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h50/hash50/hash50_1717345594.98699.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin ... ok test.unit.container.test_sharder.TestSharder.test_run_forever test.unit.container.test_sharder.TestSharder.test_run_forever ... ok sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp59q6sj4x/sda/containers/0/h70/hash70/hash70_1717345595.04640.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:01 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 5.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 3604.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 13:00:10 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 30.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test ERROR: Unhandled exception while dumping progress: Test over, path: a/c2, db: /tmp/tmpobwhvflz/sda/containers/0/ash/c2hash/c2hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over sharder-test ERROR: Exception in sharder: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2553, in run_forever self._one_shard_cycle(devices_to_shard=Everything(), File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2530, in _one_shard_cycle self._periodic_report_stats() File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1143, in _periodic_report_stats if (time.time() - self.reported) >= self.stats_interval: ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_send_shard_ranges test.unit.container.test_sharder.TestSharder.test_send_shard_ranges ... ok sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpa9kre6di/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00024247169494628906s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013184547424316406s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj002_0 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmp81cgmqxo/sda/containers/0/8fc/857fe81dc763e7cff7e7cd987d6c28fc/857fe81dc763e7cff7e7cd987d6c28fc.db, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaved ShardRange in 0.007s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00023603439331054688s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001876354217529297s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmp81cgmqxo/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015687942504882812s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014209747314453125s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339> - shard db already in sync, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmp81cgmqxo/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.274482727050781e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.96453857421875e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339> - shard db already in sync, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 0 required, shard db: /tmp/tmp81cgmqxo/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.012222290039062e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.749961853027344e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339> - shard db already in sync, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaved ShardRange<'obj002' to 'obj004' as of 1717345350.88339, (0, 0) as of 1717345350.88339, cleaved as of 1717345350.88339> in 0.003s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj006_2 for ShardRange<'obj004' to 'obj006' as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00023674964904785156s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001385211944580078s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj006_2 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaved ShardRange<'obj004' to 'obj006' as of 1717345350.88339, (2, 2) as of 1717345595.65188, cleaved as of 1717345350.88339> in 0.02s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.322166442871094e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.940696716308594e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00023865699768066406s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013256072998046875s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__3 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmp81cgmqxo/sda/containers/0/683/f7a6bb6d9367b7ae7b6b30a833289683/f7a6bb6d9367b7ae7b6b30a833289683.db, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.560585021972656e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.893013000488281e-05s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339>, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaving ShardRange<'obj006' to MaxBound as of 1717345350.88339, (0, 0) as of 1717345350.88339, created as of 1717345350.88339> - shard db already in sync, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures ... ok test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts ... ok test.unit.container.test_sharder.TestSharder.test_sharding_enabled test.unit.container.test_sharder.TestSharder.test_sharding_enabled ... ok test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump ... ok sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test INFO: Cleaved ShardRange<'obj006' to MaxBound as of 1717345350.88339, (3, 3) as of 1717345595.76994, cleaved as of 1717345350.88339> in 0.009s, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp81cgmqxo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.30241.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '98978f51-b0ad-4c84-b8f0-a99efff9d655-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp0ebilq0r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '98978f51-b0ad-4c84-b8f0-a99efff9d655-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp0ebilq0r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '98978f51-b0ad-4c84-b8f0-a99efff9d655-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp0ebilq0r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '98978f51-b0ad-4c84-b8f0-a99efff9d655-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp0ebilq0r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '5e8c20e7-6c1d-4902-8cef-be47abde2074-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp0ebilq0r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345595.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi1kp8wgv/sda/containers/0/ash/c2hash/c2hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi1kp8wgv/sda/containers/0/ash/c1hash/c1hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi1kp8wgv/sda/containers/0/ash/c3hash/c3hash.db' sharder-test INFO: Since Sun Jun 2 04:26:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:26:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range ... ok test.unit.container.test_sharder.TestSharder.test_update_stat test.unit.container.test_sharder.TestSharder.test_update_stat ... ok sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpfojotdal/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpwyduwmnc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpwyduwmnc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpwyduwmnc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpwyduwmnc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpwyduwmnc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpwyduwmnc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 3, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=3, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 4, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=4, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 5, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=5, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 6, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=2, tombstones=6, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 7, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=1, tombstones=7, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 8, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=8, path: a/c, db: /tmp/tmpmrz1g3bx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=False) in 0.00038695335388183594s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=True) in 0.00030493736267089844s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=False) in 0.00022459030151367188s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects test.unit.container.test_sharder.TestSharder.test_yield_objects ... ok sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002601146697998047s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.0002193450927734375s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00024700164794921875s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002644062042236328s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00025534629821777344s, path: a/c, db: /tmp/tmp821zzqb7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003027915954589844s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.00032520294189453125s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00029587745666503906s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00031876564025878906s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00028014183044433594s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.00028824806213378906s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021600723266601562s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00024771690368652344s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00021719932556152344s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0003364086151123047s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021767616271972656s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00032258033752441406s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00021791458129882812s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00027680397033691406s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00022339820861816406s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00022912025451660156s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002262592315673828s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00023865699768066406s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021791458129882812s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002541542053222656s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002186298370361328s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021409988403320312s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002269744873046875s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0003132820129394531s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00024127960205078125s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.0002543926239013672s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002429485321044922s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.0002338886260986328s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00022339820861816406s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00022721290588378906s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0002281665802001953s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002930164337158203s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00024271011352539062s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002338886260986328s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00023746490478515625s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002002716064453125s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range ... ok test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run test.unit.obj.test_replicator.TestObjectReplicator.test_run ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once test.unit.obj.test_replicator.TestObjectReplicator.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state ... ok sharder-test DEBUG: got 10 rows (deleted=False) in 0.0002727508544921875s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002524852752685547s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.0004487037658691406s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00028061866760253906s, path: a/c, db: /tmp/tmpan1b479o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003414154052734375s, path: a/c, db: /tmp/tmpx2gl76ig/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018095970153808594s, path: a/c, db: /tmp/tmpx2gl76ig/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: Setting object count to 1032 (txn: txf64ac89f3e7347618d6ea-00665c9d3f) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: Setting object count to 48 (txn: tx5bc657575a6a4f4ca3c40-00665c9d3f) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: Setting object count to 1064 (txn: tx00dad88eb7a1407896405-00665c9d3f) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: tx1779acd549fa452aa4350-00665c9d3f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1779acd549fa452aa4350-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1779acd549fa452aa4350-00665c9d3f) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx1779acd549fa452aa4350-00665c9d3f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1779acd549fa452aa4350-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1779acd549fa452aa4350-00665c9d3f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx1779acd549fa452aa4350-00665c9d3f) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfd6da2f5160b44f590879-00665c9d3f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Setting object count to 1065 (txn: txe1f2f0d00ccf49aead0fd-00665c9d41) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Setting object count to 1065 (txn: tx104de6c5735d4f7c95222-00665c9d44) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7aa26609c1624968a87b1-00665c9d46) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: tx7aa26609c1624968a87b1-00665c9d46) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txd9044a9134c04d92bbcc9-00665c9d48) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error ... ok test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: txd9044a9134c04d92bbcc9-00665c9d48) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txd9044a9134c04d92bbcc9-00665c9d48) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: txd9044a9134c04d92bbcc9-00665c9d48) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx631e90e703e14d00b987e-00665c9d48) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx631e90e703e14d00b987e-00665c9d48) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx631e90e703e14d00b987e-00665c9d48) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: tx631e90e703e14d00b987e-00665c9d48) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx2a4596dadbfd47e78609b-00665c9d48) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2a4596dadbfd47e78609b-00665c9d48) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a4596dadbfd47e78609b-00665c9d48) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx2a4596dadbfd47e78609b-00665c9d48) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: Setting object count to 3120 (txn: tx2fa377a4d9344c769408f-00665c9d4b) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: Setting object count to 3120 (txn: tx1eba211f2592421690ce7-00665c9d4b) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: Setting object count to 3120 (txn: tx2a57cdf36f7740309442a-00665c9d4b) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: Setting object count to 3120 (txn: tx3928b28f145b4613a3ea4-00665c9d4b) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: Setting object count to 3120 (txn: tx038543d6e4c741658b236-00665c9d4b) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx3ffae57346b04f8fb0ff7-00665c9d4b) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx295eb014a5964035a8469-00665c9d4b) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx5c228903a27e49698715b-00665c9d4b) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txb2ee516a5a5f4288983c9-00665c9d4b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb2ee516a5a5f4288983c9-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb2ee516a5a5f4288983c9-00665c9d4b) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txb2ee516a5a5f4288983c9-00665c9d4b) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txdf2a0bf387dd4913afd7d-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txdf2a0bf387dd4913afd7d-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txdf2a0bf387dd4913afd7d-00665c9d4b) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txdf2a0bf387dd4913afd7d-00665c9d4b) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx6feb7d73035646cfaaabf-00665c9d4b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6feb7d73035646cfaaabf-00665c9d4b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx6feb7d73035646cfaaabf-00665c9d4b) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx6feb7d73035646cfaaabf-00665c9d4b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx0663fea855a84c82adadb-00665c9d4d) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache ... ok test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: Setting object count to 1068 (txn: tx0663fea855a84c82adadb-00665c9d4d) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Setting object count to 1068 (txn: tx0d80dda8a2af44188e9a3-00665c9d4d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: tx0d5801dbbce946eb9d3ef-00665c9d50) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: txf2f17a82b5804ac6bcccc-00665c9d52) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc78f0ebf476940ea98f5f-00665c9d54) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Setting object count to 1058 (txn: txc78f0ebf476940ea98f5f-00665c9d54) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Setting object count to 1024 (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Setting object count to 1024 (txn: tx08f675752b3148aeb761b-00665c9d57) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Setting object count to 1072 (txn: tx12288128a80848209ee30-00665c9d59) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Setting object count to 1072 (txn: txae57a2d45bb04af492a67-00665c9d5b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Setting object count to 1072 (txn: tx5e90baca14e44979a9942-00665c9d5d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache ... ok test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Setting object count to 1072 (txn: txdf55e98160504af596d52-00665c9d60) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txcca759f86cd5466382fc4-00665c9d62) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txcca759f86cd5466382fc4-00665c9d62) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txcca759f86cd5466382fc4-00665c9d62) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txcca759f86cd5466382fc4-00665c9d62) test ERROR: Aborting listing from shards due to bad response: [404] (txn: txcca759f86cd5466382fc4-00665c9d62) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx86cb999c59294a78a690f-00665c9d64) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: tx882f7b0990214753b9edc-00665c9d67) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test DEBUG: Setting object count to 3120 (txn: tx24dd3fab531444b48ca7f-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txebca77615ac34e03bf314-00665c9d6a) test DEBUG: Setting object count to 3120 (txn: txebca77615ac34e03bf314-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txdeff4c5946c441729ad11-00665c9d6a) test DEBUG: Setting object count to 3120 (txn: txdeff4c5946c441729ad11-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txccd33c149a14487bae5ba-00665c9d6a) test DEBUG: Setting object count to 3120 (txn: txccd33c149a14487bae5ba-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test DEBUG: Setting object count to 3120 (txn: tx6391eec0b3524dcc9a5ba-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx133c69be954e40e5af764-00665c9d6a) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx133c69be954e40e5af764-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txe98941f706e548e684b51-00665c9d6a) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txe98941f706e548e684b51-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txe98941f706e548e684b51-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txe98941f706e548e684b51-00665c9d6a) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txe98941f706e548e684b51-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe98941f706e548e684b51-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe98941f706e548e684b51-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txe98941f706e548e684b51-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx9beb7ad04b2841378d2b9-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txe754479b91a642f9a781d-00665c9d6a) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txe754479b91a642f9a781d-00665c9d6a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe754479b91a642f9a781d-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe754479b91a642f9a781d-00665c9d6a) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txe754479b91a642f9a781d-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx6ee599045f3b483d80c0a-00665c9d6a) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx6ee599045f3b483d80c0a-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6ee599045f3b483d80c0a-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx6ee599045f3b483d80c0a-00665c9d6a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx6ee599045f3b483d80c0a-00665c9d6a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx7347f0a89a1d4135af507-00665c9d6a) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx7347f0a89a1d4135af507-00665c9d6a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7347f0a89a1d4135af507-00665c9d6a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx7347f0a89a1d4135af507-00665c9d6a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx7347f0a89a1d4135af507-00665c9d6a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: tx7e502b8abfd54e458cc6d-00665c9d6d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txbcec5af2698246289d445-00665c9d6f) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found ... ok test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txbcec5af2698246289d445-00665c9d6f) test DEBUG: Setting object count to 1065 (txn: txbcec5af2698246289d445-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/ascii/ascii From Object Server 10.0.0.0:1000/sda (txn: tx4fb6b35f0dfc418c9c726-00665c9d6f) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/툎/鐉 From Object Server 10.0.0.0:1000/sda (txn: txc9a9cb4669eb4accaffbd-00665c9d6f) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx58fcfbc2ab5b435b92b7f-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx4f525b6a059b46ac83365-00665c9d6f) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx4f525b6a059b46ac83365-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txd4d41959e034477881192-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txd4d41959e034477881192-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/ascii/ascii: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: txd4d41959e034477881192-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txea01d1fa5649419186b46-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txea01d1fa5649419186b46-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/툎/鐉: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: txea01d1fa5649419186b46-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx433c99618a0c4e8a8ee47-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx6b86a4e8701e41b2994c9-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx6b86a4e8701e41b2994c9-00665c9d6f) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx6b86a4e8701e41b2994c9-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: tx6b86a4e8701e41b2994c9-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: my-txn-id) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx0e6a6d49a198467ba2e21-00665c9d6f) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx0e6a6d49a198467ba2e21-00665c9d6f) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx0e6a6d49a198467ba2e21-00665c9d6f) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx0e6a6d49a198467ba2e21-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx1f58f75b65c047dca8ca8-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx4a0f7c53e5de4cd1a1328-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx4a0f7c53e5de4cd1a1328-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: tx4a0f7c53e5de4cd1a1328-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txb546bfbd5e8b47b8a68fc-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txb546bfbd5e8b47b8a68fc-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6a6a933fb1524cb49ef40-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx80b02cbc3f9248f2ac646-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: tx80b02cbc3f9248f2ac646-00665c9d6f) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance ... ok proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6505ddbb5e6b4e87b803d-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: tx6505ddbb5e6b4e87b803d-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6d64fd52aa6446869c5e9-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx6d64fd52aa6446869c5e9-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6336a5536a6c4402a17c6-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx6336a5536a6c4402a17c6-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: tx6336a5536a6c4402a17c6-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx1f83d3d9487d441fb93b7-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx1f83d3d9487d441fb93b7-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txa35c581ac06d4dfaa1349-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe55cc0553c8041b798077-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe55cc0553c8041b798077-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txf5a6aa8247d34b988f9f0-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txf5a6aa8247d34b988f9f0-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: txf5a6aa8247d34b988f9f0-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txbfc16eccc2f84186b3df3-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txbfc16eccc2f84186b3df3-00665c9d6f) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple ... ok proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txbfc16eccc2f84186b3df3-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: txbfc16eccc2f84186b3df3-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx66a899bdece845ba90c94-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx66a899bdece845ba90c94-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx66a899bdece845ba90c94-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: tx66a899bdece845ba90c94-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txba2e97d300de44318a8aa-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txba2e97d300de44318a8aa-00665c9d6f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txba2e97d300de44318a8aa-00665c9d6f) proxy-server ERROR: Object GET returning 503 for [] (txn: txba2e97d300de44318a8aa-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txb9d536c24d30413fa400a-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: tx1543faf83455426c9f7ac-00665c9d6f) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: tx1543faf83455426c9f7ac-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txc50e6bb49c3843599f482-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txc83237e41e1147e69bb34-00665c9d6f) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: txe478fa0c589b49dab78de-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx83b6c67e1ba245ad997f7-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: tx8e45fd0c95294a5a9df54-00665c9d6f) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: tx8e45fd0c95294a5a9df54-00665c9d6f) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: tx8e45fd0c95294a5a9df54-00665c9d6f) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx8e45fd0c95294a5a9df54-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: txc3a8ab9de41e420abdc74-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txa7f6dad1738f44afb2250-00665c9d6f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: txb86f895db7804ca79a914-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: tx7854297e01104d0198f28-00665c9d70) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.0:1000/sda (txn: tx7854297e01104d0198f28-00665c9d70) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: tx7854297e01104d0198f28-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx8a73b640c24846e69017c-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx8a73b640c24846e69017c-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx6c46fe895b6f4a3fa2b3e-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txabecd7344d864c1b853a1-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: txabecd7344d864c1b853a1-00665c9d70) proxy-server ERROR: Object HEAD returning 503 for [] (txn: txabecd7344d864c1b853a1-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx76c6a9f6d9bb42039b3da-00665c9d70) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx76c6a9f6d9bb42039b3da-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx76c6a9f6d9bb42039b3da-00665c9d70) proxy-server ERROR: Object HEAD returning 503 for [507] (txn: tx76c6a9f6d9bb42039b3da-00665c9d70) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txf3545cbacb7c4ed6a8072-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txf3545cbacb7c4ed6a8072-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: txf3545cbacb7c4ed6a8072-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after ... ok proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe19644008bb244749d699-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe19644008bb244749d699-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe19644008bb244749d699-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9b4853d3e708403599e6e-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9b4853d3e708403599e6e-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9b4853d3e708403599e6e-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx40af207c7b6d48c4b527d-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx40af207c7b6d48c4b527d-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx40af207c7b6d48c4b527d-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: tx7dbc69eef04242dfb5271-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Expect: 100-continue on /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: Connection refused: Please insert ten dollars (txn: tx458f647df71b470f82d0c-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.0:1000/sda (txn: txabd0c004dc0d48019437d-00665c9d70) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.1:1001/sdb (txn: tx93b56fd2d6214c52aabfe-00665c9d70) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.2:1002/sdc (txn: tx5af60a039155410dbf5dd-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Expect: 100-continue on /a/c/o.jpg: Timeout (Nones) (txn: txf9a8127a2f224b339cd07-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Expect: 100-continue on /a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx1ad7a05a433540359180d-00665c9d70) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.2:1002/sdc (txn: tx054f038af460443c9f2f0-00665c9d70) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txc3bb45b18b3d4e7682477-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/a/c/o.jpg: Timeout (Nones) (txn: tx382ce5fc39d749dea53e0-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to get final status of PUT to /v1/a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception (txn: tx8ba3645be8924e448cbf1-00665c9d70) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx2ce6a230c02a46628585f-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: txb88453d048a04779ac455-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx8dc2205e054b4231841e1-00665c9d70) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx8dc2205e054b4231841e1-00665c9d70) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx8dc2205e054b4231841e1-00665c9d70) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: tx8dc2205e054b4231841e1-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1020, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1662, in read raise Exception('exception message') Exception: exception message (txn: tx970573b4ced14010bf622-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception: boom (txn: tx92310baabb9e4827938a6-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Timeout (Nones) (txn: tx4276f9122fbe4aac88a7a-00665c9d70) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx6b55ac6343944449ad81b-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx153fcd1d7f8d412dbfda5-00665c9d70) proxy-server ERROR: ERROR 500 b'' Trying to PUT /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89 From Object Server 10.0.0.1:1001/sdb (txn: tx153fcd1d7f8d412dbfda5-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txe7faf61c40844f2d873e7-00665c9d70) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx1ad48e191f7049c6a5a45-00665c9d70) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.1:1001/sdb (txn: tx1ad48e191f7049c6a5a45-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [412] (txn: tx342ce797e5bf4c5c93a00-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending enough data (txn: tx97480d466a0a4da5baf76-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1795, in _send_chunk self.conn.send(to_send) File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 926, in send self.give_send(self, data) File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1426, in capture_send raise exc Exception: Exception while sending data on connection (txn: txe62ef8a5d70e4f5183eca-00665c9d70) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: ChunkWriteTimeout (Nones) (txn: tx02bf43fb0ec746bc8a015-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: tx9882f88a77544ef69539b-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345648.61464 <= '409 (1717345648.61464), 409 (1717345648.61464), 409 (1717345648.61464)' (txn: txca710c67dc074fa6b23fa-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345648.61760 <= '409 (1717345648.61760), 409 (1717345648.61760), 409 (1717345648.61760)' (txn: tx1cacbad6e66d42a580929-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345648.62044 <= '409 (1717345648.62044), 409 (1717345648.62044), 409 (1717345648.62044)' (txn: tx7384e0ca9ea0463382057-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345648.00000 <= '409 (1717345649.00000), 409 (1717345649.00000), 409 (1717345649.00000)' (txn: tx11d27fcdbd2a44e79d5b9-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345650.00000 <= '409 (1717345651.00000), 409 (1717345651.00000), 409 (1717345651.00000)' (txn: tx20e815ff0d0240a3aa6c2-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345652.00000 <= '409 (1717345653.00000), 409 (1717345653.00000), 409 (1717345653.00000)' (txn: txba5e97a4bf484c8b98c23-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345648.00000 <= '409 (1717345648.00000), 409 (1717345648.00000), 409 (1717345648.00000)' (txn: tx50bcf68bed8e4c5785bcb-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345649.00000 <= '409 (1717345649.00000), 409 (1717345649.00000), 409 (1717345649.00000)' (txn: txa1d776d12fc046c08d96f-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345650.00000 <= '409 (1717345650.00000), 409 (1717345650.00000), 409 (1717345650.00000)' (txn: tx2460b99df993460a9046f-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345648.00000 <= '409 (1717345648.00000)' (txn: tx20489666d1304a8d9326e-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345649.00000 <= '409 (1717345649.00000)' (txn: tx3ce3933abef2430ab16f4-00665c9d70) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345650.00000 <= '409 (1717345650.00000)' (txn: tx564d1c887ae44e9d854fd-00665c9d70) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx01076b983a654d9c8b333-00665c9d71) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345649.00000 <= '409 (1717345650.00000)' (txn: tx200589dfb6fe406187a67-00665c9d71) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [409, 412, 412] (txn: txe75df5feae074c598a89b-00665c9d71) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345649.00000 <= '409 (unknown), 409 (unknown), 409 (unknown)' (txn: txd4b16d21c0c4495986a37-00665c9d71) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345649.00000 <= '409 (1717345650.00000), 201 (unknown), 201 (unknown)' (txn: tx114cf2be30c044e8a9c19-00665c9d71) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpd2hq3f30/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpd2hq3f30/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpd2hq3f30/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpd2hq3f30/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0257 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0117 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0086 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0097 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx430c1a2ef414487faa577-00665c9d71" "proxy-server 3542033" 0.0016 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:48479/sda1/1/a/c" "tx430c1a2ef414487faa577-00665c9d71" "container-server 3542033" 0.0013 "-" 3542033 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:35981/sdb1/1/a/c" "tx430c1a2ef414487faa577-00665c9d71" "container-server 3542033" 0.0012 "-" 3542033 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx430c1a2ef414487faa577-00665c9d71" "proxy-server 3542033" 0.0264 "-" 3542033 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx430c1a2ef414487faa577-00665c9d71" "proxy-server 3542033" 0.0166 "-" 3542033 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/29 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx430c1a2ef414487faa577-00665c9d71 - 0.0368 - - 1717345649.309957743 1717345649.346710682 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txdf1ea15c2a3f485daafbc-00665c9d71" "proxy-server 3542033" 0.0014 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:48479/sda1/1/a1/c1" "txdf1ea15c2a3f485daafbc-00665c9d71" "container-server 3542033" 0.0012 "-" 3542033 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35981/sdb1/1/a1/c1" "txdf1ea15c2a3f485daafbc-00665c9d71" "container-server 3542033" 0.0012 "-" 3542033 0 /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.proxy.test_mem_server.TestAccountController.test_DELETE test.unit.proxy.test_mem_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET test.unit.proxy.test_mem_server.TestAccountController.test_GET ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD test.unit.proxy.test_mem_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT test.unit.proxy.test_mem_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txdf1ea15c2a3f485daafbc-00665c9d71" "proxy-server 3542033" 0.0149 "-" 3542033 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txdf1ea15c2a3f485daafbc-00665c9d71" "proxy-server 3542033" 0.0268 "-" 3542033 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/29 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txdf1ea15c2a3f485daafbc-00665c9d71 - 0.0357 - - 1717345649.348402977 1717345649.384077311 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe5326f4fafbb41429e505-00665c9d71" "proxy-server 3542033" 0.0057 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:48479/sda1/2/a/c1" "txe5326f4fafbb41429e505-00665c9d71" "container-server 3542033" 0.0068 "-" 3542033 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:35981/sdb1/2/a/c1" "txe5326f4fafbb41429e505-00665c9d71" "container-server 3542033" 0.0013 "-" 3542033 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe5326f4fafbb41429e505-00665c9d71" "proxy-server 3542033" 0.0311 "-" 3542033 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe5326f4fafbb41429e505-00665c9d71" "proxy-server 3542033" 0.0215 "-" 3542033 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/29 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txe5326f4fafbb41429e505-00665c9d71 - 0.0454 - - 1717345649.385762453 1717345649.431116581 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx09de7fe3be3841dcb330d-00665c9d71" "proxy-server 3542033" 0.0055 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:48479/sda1/1/a/c2" "tx09de7fe3be3841dcb330d-00665c9d71" "container-server 3542033" 0.0013 "-" 3542033 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:35981/sdb1/1/a/c2" "tx09de7fe3be3841dcb330d-00665c9d71" "container-server 3542033" 0.0095 "-" 3542033 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx09de7fe3be3841dcb330d-00665c9d71" "proxy-server 3542033" 0.0236 "-" 3542033 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:29 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx09de7fe3be3841dcb330d-00665c9d71" "proxy-server 3542033" 0.0353 "-" 3542033 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/29 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx09de7fe3be3841dcb330d-00665c9d71 - 0.0485 - - 1717345649.432767391 1717345649.481232882 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpyb35gulv/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpyb35gulv/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpyb35gulv/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpyb35gulv/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0108 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0089 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0102 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0097 "-" 3542033 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txcd2b37bd17a04a76a749d-00665c9d72" "proxy-server 3542033" 0.0015 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:54941/sdb1/1/a/c" "txcd2b37bd17a04a76a749d-00665c9d72" "container-server 3542033" 0.0014 "-" 3542033 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:55571/sda1/1/a/c" "txcd2b37bd17a04a76a749d-00665c9d72" "container-server 3542033" 0.0022 "-" 3542033 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txcd2b37bd17a04a76a749d-00665c9d72" "proxy-server 3542033" 0.0271 "-" 3542033 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txcd2b37bd17a04a76a749d-00665c9d72" "proxy-server 3542033" 0.0174 "-" 3542033 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/30 PUT /v1/a/c HTTP/1.0 201 - - t - - - txcd2b37bd17a04a76a749d-00665c9d72 - 0.0372 - - 1717345650.092421293 1717345650.129580736 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txae1d7ca9a8264e6b8ce97-00665c9d72" "proxy-server 3542033" 0.0015 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:55571/sda1/1/a1/c1" "txae1d7ca9a8264e6b8ce97-00665c9d72" "container-server 3542033" 0.0013 "-" 3542033 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:54941/sdb1/1/a1/c1" "txae1d7ca9a8264e6b8ce97-00665c9d72" "container-server 3542033" 0.0012 "-" 3542033 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txae1d7ca9a8264e6b8ce97-00665c9d72" "proxy-server 3542033" 0.0151 "-" 3542033 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txae1d7ca9a8264e6b8ce97-00665c9d72" "proxy-server 3542033" 0.0265 "-" 3542033 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/30 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txae1d7ca9a8264e6b8ce97-00665c9d72 - 0.0355 - - 1717345650.131270885 1717345650.166792870 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa836d6cde67c4ff89c707-00665c9d72" "proxy-server 3542033" 0.0076 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:54941/sdb1/2/a/c1" "txa836d6cde67c4ff89c707-00665c9d72" "container-server 3542033" 0.0013 "-" 3542033 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:55571/sda1/2/a/c1" "txa836d6cde67c4ff89c707-00665c9d72" "container-server 3542033" 0.0052 "-" 3542033 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa836d6cde67c4ff89c707-00665c9d72" "proxy-server 3542033" 0.0201 "-" 3542033 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa836d6cde67c4ff89c707-00665c9d72" "proxy-server 3542033" 0.0323 "-" 3542033 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/30 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txa836d6cde67c4ff89c707-00665c9d72 - 0.0478 - - 1717345650.168434143 1717345650.216202021 1 test.unit.proxy.test_server.TestController.test_account_info_200 test.unit.proxy.test_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_404 test.unit.proxy.test_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_container_count test.unit.proxy.test_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_account test.unit.proxy.test_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_cache test.unit.proxy.test_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_200 test.unit.proxy.test_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_404 test.unit.proxy.test_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_invalid_account test.unit.proxy.test_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_no_cache test.unit.proxy.test_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_make_requests test.unit.proxy.test_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_transfer_headers test.unit.proxy.test_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx90cdf6dd3df043088a470-00665c9d72" "proxy-server 3542033" 0.0074 "-" 3542033 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:54941/sdb1/1/a/c2" "tx90cdf6dd3df043088a470-00665c9d72" "container-server 3542033" 0.0054 "-" 3542033 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:55571/sda1/1/a/c2" "tx90cdf6dd3df043088a470-00665c9d72" "container-server 3542033" 0.0012 "-" 3542033 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx90cdf6dd3df043088a470-00665c9d72" "proxy-server 3542033" 0.0297 "-" 3542033 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:30 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx90cdf6dd3df043088a470-00665c9d72" "proxy-server 3542033" 0.0203 "-" 3542033 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/30 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx90cdf6dd3df043088a470-00665c9d72 - 0.0458 - - 1717345650.217895269 1717345650.263730288 2 ---------------------------------------------------------------------- Ran 8924 tests in 317.885s OK (skipped=1814) + stestr slowest Test id Runtime (s) ----------------------------------------------------------------------------------------------------------------- ----------- test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET 21.658 test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving 17.034 test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges 15.774 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys 14.881 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases 13.382 test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state 11.966 test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments 11.502 test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop 11.397 test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT 11.257 test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile 11.211 + rm -rf .stestr + [ 3.11 = disabled ] + + cut -d. -f1 echo 3.11 + PYMAJOR=3 + echo ===> Testing with python (python3) ===> Testing with python (python3) + pwd + [ -d /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages ] + [ -z /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages ] + [ -e .stestr.conf ] + [ -x /usr/bin/python3-stestr ] + STESTR=stestr + rm -rf .stestr + subunit2pyunit + PYTHON=python3.11 stestr run --parallel --subunit --test-path test/unit test\.unit\.(?!common\.test_utils\.TestUtils\.test_get_logger_sysloghandler_plumbing|common\.middleware\.test_cname_lookup\.TestCNAMELookup.*|common\.test_db\.TestDatabaseBroker\.test_get.*|container\.test_sync\.TestContainerSync\.test_init.*|common\.test_utils\.TestPunchHoleReally\.test_punch_a_hole.*|common\.test_utils\.Test_LibcWrapper\.test_argument_plumbing.*|common\.test_utils\.TestUtils\.test_load_pkg_resource_importlib.*|common\.test_db_replicator\.TestHandoffsOnly.*|common\.ring\.test_builder\.TestGetRequiredOverload\.test_perfect_four_zone_four_replica_bad_placement|common\.test_wsgi\.TestWSGI.*|obj\.test_expirer\.TestObjectExpirer\.test_delete_actual_object|common\.test_memcached\.TestMemcached\.test_get_conns_hostname6|common\.test_memcached\.TestMemcached\.test_get_conns_v6|common\.test_memcached\.TestMemcached\.test_get_conns_v6_default|obj\.test_reconstructor\.TestWorkerReconstructor\.test_run_forever_recon_aggregation|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_with_duplicate_but_sufficient_frag_indexes|proxy\.test_mem_server\.TestProxyServer\.test_statsd_prefix|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_read_timeout|proxy\.controllers\.test_base\.TestGetOrHeadHandler\.test_disconnected_logging|obj\.test_expirer\.TestObjectExpirer\.test_failed_delete_continues_on|obj\.test_replicator\.TestObjectReplicator\.test_update) Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! 2 RLock(s) were not greened, to fix this error make sure you run eventlet.monkey_patch() before importing any other modules. test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset ... ok test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv ... ok test.unit.account.test_server.TestAccountController.test_DELETE_empty test.unit.account.test_server.TestAccountController.test_DELETE_empty ... ok test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_empty test.unit.account.test_server.TestAccountController.test_DELETE_not_empty ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_found test.unit.account.test_server.TestAccountController.test_DELETE_not_found ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range ... ok test.unit.account.test_server.TestAccountController.test_DELETE_now_empty test.unit.account.test_server.TestAccountController.test_DELETE_now_empty ... ok test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_json test.unit.account.test_server.TestAccountController.test_GET_accept_json ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_xml test.unit.account.test_server.TestAccountController.test_GET_accept_xml ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_json test.unit.account.test_server.TestAccountController.test_GET_empty_account_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes ... ok test.unit.account.test_server.TestAccountController.test_GET_invalid_accept test.unit.account.test_server.TestAccountController.test_GET_invalid_accept ... ok test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root ... ok test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder ... ok test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg ... ok test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature ... ok test.unit.cli.test_info.TestCliInfo.test_print_db_syncs test.unit.cli.test_info.TestCliInfo.test_print_db_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_info test.unit.cli.test_info.TestCliInfo.test_print_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options ... ok test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim ... ok test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers ... ok test.unit.account.test_backend.TestAccountBroker.test_chexor test.unit.account.test_backend.TestAccountBroker.test_chexor ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta ... ok test.unit.account.test_backend.TestAccountBroker.test_creation test.unit.account.test_backend.TestAccountBroker.test_creation ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict ... ok test.unit.account.test_backend.TestAccountBroker.test_delete_container test.unit.account.test_backend.TestAccountBroker.test_delete_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_delete_db_status test.unit.account.test_backend.TestAccountBroker.test_delete_db_status ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name ... ok test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict ... ok test INFO: Begin account audit "once" mode test INFO: Account audit "once" mode completed: 0.00s test INFO: - - - [02/Jun/2024:16:27:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0133 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:53 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0099 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:53 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0006 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0126 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0046 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0184 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0358 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0068 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0222 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0221 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0205 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0252 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0180 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0349 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0222 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0326 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0241 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0196 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0177 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0130 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0143 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0017 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0115 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0172 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0183 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0034 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0179 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0123 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0092 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0117 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0109 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0118 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0128 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0160 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0159 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0170 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0195 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0167 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0169 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0037 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0036 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0037 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0124 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0235 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0139 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0208 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0136 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0034 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0120 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0103 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0128 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0142 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0175 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:54 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0032 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0176 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0024 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0239 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0238 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0150 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0206 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0559 "-" 3543384 - test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml ... ok test.unit.account.test_backend.TestAccountBroker.test_empty test.unit.account.test_backend.TestAccountBroker.test_empty ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id ... ok test.unit.account.test_server.TestAccountController.test_GET_log_requests_false test.unit.account.test_server.TestAccountController.test_GET_log_requests_false ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_GET_log_requests_true test.unit.account.test_server.TestAccountController.test_GET_log_requests_true ... ok test.unit.account.test_backend.TestAccountBroker.test_exception test.unit.account.test_backend.TestAccountBroker.test_exception ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status ... ok test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter ... ok test.unit.cli.test_recon.TestHelpers.test_size_suffix test.unit.cli.test_recon.TestHelpers.test_size_suffix ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter ... ok test.unit.cli.test_recon.TestReconCommands.test_disk_usage test.unit.cli.test_recon.TestReconCommands.test_disk_usage ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info test.unit.account.test_backend.TestAccountBroker.test_get_info ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_GET_not_found_json test.unit.account.test_server.TestAccountController.test_GET_not_found_json ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_plain test.unit.account.test_server.TestAccountController.test_GET_not_found_plain ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok ... ok test.unit.cli.test_recon.TestReconCommands.test_load_check test.unit.cli.test_recon.TestReconCommands.test_load_check ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok ... ok test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types ... ok test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json ... ok test.unit.account.test_backend.TestAccountBroker.test_initialize_fail test.unit.account.test_backend.TestAccountBroker.test_initialize_fail ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_GET_not_found_xml test.unit.account.test_server.TestAccountController.test_GET_not_found_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_over_limit test.unit.account.test_server.TestAccountController.test_GET_over_limit ... ok test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail ... ok test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted ... ok test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db ... ok test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json ... ok test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted ... ok test.unit.cli.test_recon.TestReconCommands.test_replication_check test.unit.cli.test_recon.TestReconCommands.test_replication_check ... ok test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_info test.unit.account.test_backend.TestCommonAccountBroker.test_get_info ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short ... ok test.unit.cli.test_recon.TestReconCommands.test_server_type_check test.unit.cli.test_recon.TestReconCommands.test_server_type_check ... ok test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestReconCommands.test_sharding_check test.unit.cli.test_recon.TestReconCommands.test_sharding_check ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items test.unit.account.test_backend.TestAccountBroker.test_merge_items ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check test.unit.cli.test_recon.TestReconCommands.test_time_check ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items ... ok test.unit.account.test_backend.TestAccountBroker.test_newid test.unit.account.test_backend.TestAccountBroker.test_newid ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get ... ok test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor ... ok test.unit.cli.test_reload.TestMain.test_check_failed test.unit.cli.test_reload.TestMain.test_check_failed ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.account.test_backend.TestAccountBroker.test_put_container test.unit.account.test_backend.TestAccountBroker.test_put_container ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check test.unit.cli.test_recon.TestReconCommands.test_version_check ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check_differs test.unit.cli.test_recon.TestReconCommands.test_version_check_differs ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation ... ok test.unit.cli.test_reload.TestMain.test_good test.unit.cli.test_reload.TestMain.test_good ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline ... ok test.unit.account.test_backend.TestAccountBroker.test_reclaim test.unit.account.test_backend.TestAccountBroker.test_reclaim ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container ... ok test.unit.cli.test_reload.TestMain.test_needs_pid test.unit.cli.test_reload.TestMain.test_needs_pid ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status ... ok test.unit.cli.test_reload.TestMain.test_timeout test.unit.cli.test_reload.TestMain.test_timeout ... ok test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET ... ok test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT ... ok test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose test.unit.cli.test_ringcomposer.TestCommands.test_compose ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_replicator.TestReplicatorSync.test_local_ids test.unit.account.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset ... ok test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta ... ok test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ringcomposer.TestCommands.test_show test.unit.cli.test_ringcomposer.TestCommands.test_show ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_replicator.TestReplicatorSync.test_sync test.unit.account.test_replicator.TestReplicatorSync.test_sync ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments ... ok test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid ... ok test.unit.cli.test_recon.TestRecon.test_async_check test.unit.cli.test_recon.TestRecon.test_async_check ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state ... ok test.unit.cli.test_recon.TestRecon.test_drive_audit_check test.unit.cli.test_recon.TestRecon.test_drive_audit_check ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest ... ok test.unit.cli.test_recon.TestRecon.test_gen_stats test.unit.cli.test_recon.TestRecon.test_gen_stats ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret ... ok test.unit.account.test_reaper.TestReaper.test_basic_reap_account test.unit.account.test_reaper.TestReaper.test_basic_reap_account ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_response test.unit.account.test_utils.TestAccountUtils.test_account_listing_response ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.cli.test_recon.TestRecon.test_get_error_ringnames test.unit.cli.test_recon.TestRecon.test_get_error_ringnames ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment ... ok test.unit.cli.test_recon.TestRecon.test_get_hosts test.unit.cli.test_recon.TestRecon.test_get_hosts ... ok test.unit.account.test_reaper.TestReaper.test_creation test.unit.account.test_reaper.TestReaper.test_creation ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions ... ok test.unit.cli.test_recon.TestRecon.test_get_ring_names test.unit.cli.test_recon.TestRecon.test_get_ring_names ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default ... ok test.unit.cli.test_recon.TestRecon.test_get_ringmd5 test.unit.cli.test_recon.TestRecon.test_get_ringmd5 ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.account.test_reaper.TestReaper.test_reap_account test.unit.account.test_reaper.TestReaper.test_reap_account ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_no_container test.unit.account.test_reaper.TestReaper.test_reap_account_no_container ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.cli.test_recon.TestRecon.test_ptime test.unit.cli.test_recon.TestRecon.test_ptime ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail ... ok test.unit.cli.test_recon.TestRecon.test_quarantine_check test.unit.cli.test_recon.TestRecon.test_quarantine_check ... ok test.unit.account.test_reaper.TestReaper.test_reap_container test.unit.account.test_reaper.TestReaper.test_reap_container ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Not a path query: /, skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Cached creds valid test DEBUG: Connecting with tenant: tenant_id test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Cached keystone credentials test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny ... ok test.unit.cli.test_recon.TestRecon.test_umount_check test.unit.cli.test_recon.TestRecon.test_umount_check ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path ... ok test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id ... ok test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_stubs test.unit.common.test_daemon.TestWorkerDaemon.test_stubs ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container ... ok test.unit.common.test_db_auditor.TestAuditor.test_database_auditor test.unit.common.test_db_auditor.TestAuditor.test_database_auditor ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid ... ok test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace ... ok test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests ... ok test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac test.unit.common.test_digest.TestDigestUtils.test_get_hmac ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_good test.unit.cli.test_reload.TestValidateManagerPid.test_good ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_python test.unit.cli.test_reload.TestValidateManagerPid.test_non_python ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_reload.TestValidateManagerPid.test_non_server test.unit.cli.test_reload.TestValidateManagerPid.test_non_server ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_open_error test.unit.cli.test_reload.TestValidateManagerPid.test_open_error ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_worker test.unit.cli.test_reload.TestValidateManagerPid.test_worker ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_init test.unit.common.test_internal_client.TestCompressingfileReader.test_init ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestCompressingfileReader.test_read test.unit.common.test_internal_client.TestCompressingfileReader.test_read ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestCompressingfileReader.test_seek test.unit.common.test_internal_client.TestCompressingfileReader.test_seek ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post ... ok test.unit.cli.test_relinker.TestRelinker.test_conf_file test.unit.cli.test_relinker.TestRelinker.test_conf_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default test.unit.common.test_memcached.TestLoadMemcache.test_conf_default ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read ... ok test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins ... ok test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins ... ok test.unit.cli.test_relinker.TestRelinker.test_rehashing test.unit.cli.test_relinker.TestRelinker.test_rehashing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request ... ok test.unit.cli.test_recon.TestScout.test_scout_http_error test.unit.cli.test_recon.TestScout.test_scout_http_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestScout.test_scout_ok test.unit.cli.test_recon.TestScout.test_scout_ok ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment ... ok test.unit.common.test_splice.TestTee.test_available test.unit.common.test_splice.TestTee.test_available ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment ... ok test.unit.common.test_splice.TestTee.test_errno test.unit.common.test_splice.TestTee.test_errno ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestScout.test_scout_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_socket_timeout ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring ... ok test.unit.cli.test_recon.TestScout.test_scout_url_error test.unit.cli.test_recon.TestScout.test_scout_url_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring test.unit.cli.test_ringbuilder.TestCommands.test_create_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_file test.unit.cli.test_relinker.TestRelinker.test_relink_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default test.unit.cli.test_ringbuilder.TestCommands.test_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output test.unit.cli.test_ringbuilder.TestCommands.test_default_output ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config ... ok test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ringbuilder.TestCommands.test_find_parts test.unit.cli.test_ringbuilder.TestCommands.test_find_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name ... ok test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container ... ok test.unit.common.ring.test_ring.TestRing.test_creation test.unit.common.ring.test_ring.TestRing.test_creation ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes test.unit.common.ring.test_ring.TestRing.test_get_more_nodes ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv ... ok test.unit.common.ring.test_ring.TestRing.test_get_nodes test.unit.common.ring.test_ring.TestRing.test_get_nodes ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container ... ok test.unit.common.ring.test_ring.TestRing.test_get_part test.unit.common.ring.test_ring.TestRing.test_get_part ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_ring.TestRing.test_get_part_nodes test.unit.common.ring.test_ring.TestRing.test_get_part_nodes ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object ... ok test.unit.common.ring.test_ring.TestRing.test_has_changed test.unit.common.ring.test_ring.TestRing.test_has_changed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once ... ok test.unit.common.ring.test_ring.TestRing.test_reload test.unit.common.ring.test_ring.TestRing.test_reload ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized ... ok test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_ring.TestRing.test_reload_without_replication test.unit.common.ring.test_ring.TestRing.test_reload_without_replication ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_ring.TestRing.test_replica_count test.unit.common.ring.test_ring.TestRing.test_replica_count ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty ... ok test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error ... ok test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key ... ok test.unit.cli.test_relinker.TestRelinker.test_state_file test.unit.cli.test_relinker.TestRelinker.test_state_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.cli.test_relinker.TestRelinker.test_workers_children test.unit.cli.test_relinker.TestRelinker.test_workers_children ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part ... ok test.unit.cli.test_relinker.TestRelinker.test_workers_parent test.unit.cli.test_relinker.TestRelinker.test_workers_parent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_init test.unit.common.test_memcached.TestMemcacheCommand.test_init ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error ... ok test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{}' test DEBUG: Error on keystone reply: 201 b'{}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'[]' test DEBUG: Error on keystone reply: 201 b'[]' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'null' test DEBUG: Error on keystone reply: 201 b'null' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'"foo"' test DEBUG: Error on keystone reply: 201 b'"foo"' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'1' test DEBUG: Error on keystone reply: 201 b'1' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'true' test DEBUG: Error on keystone reply: 201 b'true' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, rejecting request with error: 400 Bad Request test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_eq test.unit.common.test_utils.TestNamespaceBoundList.test_eq ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_parse test.unit.common.test_utils.TestNamespaceBoundList.test_parse ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing ... ok test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0042 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0132 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0137 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0148 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0136 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0142 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0123 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0149 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0135 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0034 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0032 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0030 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0033 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0033 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0033 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0136 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0229 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0017 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0192 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0123 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0103 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0113 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0128 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0115 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0127 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0121 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0137 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0172 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0142 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0308 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0161 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0035 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0033 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0174 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0128 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0930 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0175 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0136 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0168 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0171 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0288 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0141 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0180 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0123 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0253 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0198 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0119 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0173 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0141 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0137 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0093 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0159 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0128 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0118 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0135 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0210 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0183 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0115 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0122 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0171 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0039 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0259 "-" 3543384 - test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_json test.unit.account.test_server.TestAccountController.test_GET_with_containers_json ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path ... ok test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file ... ok test.unit.account.test_server.TestAccountController.test_HEAD_empty_account test.unit.account.test_server.TestAccountController.test_HEAD_empty_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args ... ok Something useful Also useful Boring, but not secret test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations ... ok test.unit.common.test_manager.TestManager.test_create test.unit.common.test_manager.TestManager.test_create ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition ... ok test.unit.common.test_manager.TestManager.test_default_strict test.unit.common.test_manager.TestManager.test_default_strict ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section ... ok test.unit.account.test_server.TestAccountController.test_HEAD_not_found test.unit.account.test_server.TestAccountController.test_HEAD_not_found ... ok test.unit.common.test_manager.TestManager.test_force_reload test.unit.common.test_manager.TestManager.test_force_reload ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length ... ok test.unit.account.test_server.TestAccountController.test_HEAD_with_containers test.unit.account.test_server.TestAccountController.test_HEAD_with_containers ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file ... ok test.unit.common.test_manager.TestManager.test_get_command test.unit.common.test_manager.TestManager.test_get_command ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length ... ok test.unit.account.test_server.TestAccountController.test_OPTIONS test.unit.account.test_server.TestAccountController.test_OPTIONS ... ok test.unit.common.test_manager.TestManager.test_iter test.unit.common.test_manager.TestManager.test_iter ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance ... ok test.unit.common.test_direct_client.TestDirectClient.test_gen_headers test.unit.common.test_direct_client.TestDirectClient.test_gen_headers ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata ... ok test.unit.common.test_manager.TestManager.test_list_commands test.unit.common.test_manager.TestManager.test_list_commands ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry test.unit.common.test_direct_client.TestDirectClient.test_retry ... ok test.unit.common.test_manager.TestManager.test_no_daemon test.unit.common.test_manager.TestManager.test_no_daemon ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission ... ok test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region ... ok test.unit.common.test_manager.TestManager.test_no_wait test.unit.common.test_manager.TestManager.test_no_wait ... ok test.unit.account.test_server.TestAccountController.test_POST_insufficient_space test.unit.account.test_server.TestAccountController.test_POST_insufficient_space ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception ... ok test.unit.common.test_manager.TestManager.test_once test.unit.common.test_manager.TestManager.test_once ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power ... ok test.unit.account.test_server.TestAccountController.test_POST_invalid_partition test.unit.account.test_server.TestAccountController.test_POST_invalid_partition ... ok test.unit.common.test_internal_client.TestGetAuth.test_invalid_version test.unit.common.test_internal_client.TestGetAuth.test_invalid_version ... ok test.unit.common.test_manager.TestManager.test_reload test.unit.common.test_manager.TestManager.test_reload ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance ... ok test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float ... ok test.unit.common.test_internal_client.TestGetAuth.test_ok test.unit.common.test_internal_client.TestGetAuth.test_ok ... ok test.unit.common.test_manager.TestManager.test_restart test.unit.common.test_manager.TestManager.test_restart ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_PUT test.unit.account.test_server.TestAccountController.test_PUT ... ok test.unit.common.test_linkat.TestLinkat.test_available test.unit.common.test_linkat.TestLinkat.test_available ... ok test.unit.common.test_manager.TestManager.test_run_command test.unit.common.test_manager.TestManager.test_run_command ... ok test.unit.common.test_linkat.TestLinkat.test_errno test.unit.common.test_linkat.TestLinkat.test_errno ... ok test.unit.common.test_manager.TestManager.test_shutdown test.unit.common.test_manager.TestManager.test_shutdown ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices ... ok test.unit.common.test_linkat.TestLinkat.test_flags test.unit.common.test_linkat.TestLinkat.test_flags ... ok test.unit.common.test_manager.TestManager.test_start test.unit.common.test_manager.TestManager.test_start ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_update test.unit.common.middleware.s3api.test_utils.TestConfig.test_update ... ok test.unit.common.test_manager.TestManager.test_status test.unit.common.test_manager.TestManager.test_status ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_success test.unit.common.test_linkat.TestLinkat.test_linkat_success ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload ... ok test.unit.common.test_manager.TestManager.test_stop test.unit.common.test_manager.TestManager.test_stop ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable test.unit.common.test_linkat.TestLinkat.test_unavailable ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced ... ok test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_manager.TestManagerModule.test_command_wrapper test.unit.common.test_manager.TestManagerModule.test_command_wrapper ... ok test.unit.common.test_swob.TestEtag.test_normalize_bytes test.unit.common.test_swob.TestEtag.test_normalize_bytes ... ok test.unit.common.test_manager.TestManagerModule.test_exc test.unit.common.test_manager.TestManagerModule.test_exc ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance ... ok test.unit.common.test_manager.TestManagerModule.test_format_server_name test.unit.common.test_manager.TestManagerModule.test_format_server_name ... ok test.unit.common.test_swob.TestEtag.test_normalize_etag test.unit.common.test_swob.TestEtag.test_normalize_etag ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account ... ok test.unit.common.test_swob.TestRequest.test_401_invalid_path test.unit.common.test_swob.TestRequest.test_401_invalid_path ... ok test.unit.common.test_manager.TestManagerModule.test_safe_kill test.unit.common.test_manager.TestManagerModule.test_safe_kill ... ok test.unit.common.test_manager.TestManagerModule.test_servers test.unit.common.test_manager.TestManagerModule.test_servers ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through ... ok test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring ... ok test.unit.common.test_manager.TestManagerModule.test_setup_env test.unit.common.test_manager.TestManagerModule.test_setup_env ... ok test.unit.common.test_swob.TestRequest.test_401_unauthorized test.unit.common.test_swob.TestRequest.test_401_unauthorized ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id test.unit.common.ring.test_builder.TestRingBuilder.test_id ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json ... ok test.unit.common.test_manager.TestManagerModule.test_verify_server test.unit.common.test_manager.TestManagerModule.test_verify_server ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json ... ok test.unit.common.test_swob.TestRequest.test_401_valid_account_path test.unit.common.test_swob.TestRequest.test_401_valid_account_path ... ok test.unit.common.test_manager.TestManagerModule.test_watch_server_pids test.unit.common.test_manager.TestManagerModule.test_watch_server_pids ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file ... ok test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors ... ok test.unit.common.test_memcached.TestMemcached.test_connection_pooling test.unit.common.test_memcached.TestMemcached.test_connection_pooling ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json ... ok test.unit.common.test_swob.TestRequest.test_accept_header test.unit.common.test_swob.TestRequest.test_accept_header ... ok test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect ... ok test.unit.common.test_swob.TestRequest.test_allow_reserved_names test.unit.common.test_swob.TestRequest.test_allow_reserved_names ... ok test.unit.common.test_swob.TestRequest.test_as_referer test.unit.common.test_swob.TestRequest.test_as_referer ... ok test.unit.common.test_swob.TestRequest.test_bad_path_info_pop test.unit.common.test_swob.TestRequest.test_bad_path_info_pop ... ok test.unit.common.test_memcached.TestMemcached.test_decr test.unit.common.test_memcached.TestMemcached.test_decr ... ok test.unit.common.test_memcached.TestMemcached.test_delete test.unit.common.test_memcached.TestMemcached.test_delete ... ok test.unit.common.test_swob.TestRequest.test_bad_range test.unit.common.test_swob.TestRequest.test_bad_range ... ok test.unit.common.test_swob.TestRequest.test_blank test.unit.common.test_swob.TestRequest.test_blank ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json ... ok test.unit.common.test_swob.TestRequest.test_blank_body_precedence test.unit.common.test_swob.TestRequest.test_blank_body_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_parsing test.unit.common.test_swob.TestRequest.test_blank_parsing ... ok test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_init test.unit.common.ring.test_builder.TestRingBuilder.test_init ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas ... ok test.unit.common.test_swob.TestRequest.test_copy_get test.unit.common.test_swob.TestRequest.test_copy_get ... ok test.unit.common.test_swob.TestRequest.test_datetime_properties test.unit.common.test_swob.TestRequest.test_datetime_properties ... ok test.unit.common.test_swob.TestRequest.test_get_response test.unit.common.test_swob.TestRequest.test_get_response ... ok test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_load test.unit.common.ring.test_builder.TestRingBuilder.test_load ... ok test.unit.common.test_swob.TestRequest.test_message_length_just_content_length test.unit.common.test_swob.TestRequest.test_message_length_just_content_length ... ok test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding ... ok test.unit.common.test_swob.TestRequest.test_not_401 test.unit.common.test_swob.TestRequest.test_not_401 ... ok test.unit.common.test_swob.TestRequest.test_params test.unit.common.test_swob.TestRequest.test_params ... ok test.unit.common.test_swob.TestRequest.test_path test.unit.common.test_swob.TestRequest.test_path ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop test.unit.common.test_swob.TestRequest.test_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_last test.unit.common.test_swob.TestRequest.test_path_info_pop_last ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_none test.unit.common.test_swob.TestRequest.test_path_info_pop_none ... ok test.unit.common.test_swob.TestRequest.test_path_qs test.unit.common.test_swob.TestRequest.test_path_qs ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours ... ok test.unit.common.test_swob.TestRequest.test_path_question_mark test.unit.common.test_swob.TestRequest.test_path_question_mark ... ok test.unit.common.test_swob.TestRequest.test_properties test.unit.common.test_swob.TestRequest.test_properties ... ok test.unit.common.test_swob.TestRequest.test_split_path test.unit.common.test_swob.TestRequest.test_split_path ... ok test.unit.common.test_swob.TestRequest.test_swift_entity_path test.unit.common.test_swob.TestRequest.test_swift_entity_path ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload test.unit.common.ring.test_builder.TestRingBuilder.test_overload ... ok test.unit.common.test_swob.TestRequest.test_timestamp test.unit.common.test_swob.TestRequest.test_timestamp ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance ... ok test.unit.common.test_swob.TestRequest.test_timestamp_invalid test.unit.common.test_swob.TestRequest.test_timestamp_invalid ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade ... ok test.unit.common.test_swob.TestRequest.test_timestamp_missing test.unit.common.test_swob.TestRequest.test_timestamp_missing ... ok test.unit.common.test_swob.TestRequest.test_unicode_params test.unit.common.test_swob.TestRequest.test_unicode_params ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot ... ok test.unit.common.test_swob.TestRequest.test_unicode_path test.unit.common.test_swob.TestRequest.test_unicode_path ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev ... ok test.unit.common.test_swob.TestRequest.test_unicode_query test.unit.common.test_swob.TestRequest.test_unicode_query ... ok test.unit.common.test_swob.TestRequest.test_url test.unit.common.test_swob.TestRequest.test_url ... ok test.unit.common.test_swob.TestRequest.test_url2 test.unit.common.test_swob.TestRequest.test_url2 ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight ... ok test.unit.common.test_swob.TestRequest.test_wsgify test.unit.common.test_swob.TestRequest.test_wsgify ... ok test.unit.common.test_swob.TestRequest.test_wsgify_method test.unit.common.test_swob.TestRequest.test_wsgify_method ... ok test.unit.common.test_swob.TestRequest.test_wsgify_raise test.unit.common.test_swob.TestRequest.test_wsgify_raise ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_pending test.unit.common.test_utils.TestGreenAsyncPile.test_pending ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_conf test.unit.container.test_sharder.TestContainerSharderConf.test_conf ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save test.unit.common.ring.test_builder.TestRingBuilder.test_save ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save_load test.unit.common.ring.test_builder.TestRingBuilder.test_save_load ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_default test.unit.container.test_sharder.TestContainerSharderConf.test_default ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate test.unit.container.test_sharder.TestContainerSharderConf.test_validate ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate test.unit.common.ring.test_builder.TestRingBuilder.test_validate ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go ... ok test.unit.common.test_db.TestHelperFunctions.test_zero_like test.unit.common.test_db.TestHelperFunctions.test_zero_like ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run test.unit.obj.test_replicator.TestObjectReplicator.test_run ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once test.unit.obj.test_replicator.TestObjectReplicator.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test_digest_utils WARNING: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY jojo', (txn: tx7ea780f46e5f4d1aabe7d-00665c9d8a) proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXYjojo a b c d e', (txn: tx86608bb3975a4389b822b-00665c9d8a) proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY a b c d e', (txn: tx8433c740021e4847972ce-00665c9d8a) proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'GET /someurl HTTP/1.0\r\n', (txn: test-bad-req-trans-id) proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:27:54] "GET /someurl HTTP/1.1" 200 160 0.000123 proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:27:54] "GET /otherurl HTTP/1.1" 200 179 0.000081 proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:27:54] "GET /someurl HTTP/1.0" 200 186 0.000105 proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:27:54] "GET /someurl HTTP/1.0" 200 179 0.000141 proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:27:54] "GET /someurl HTTP/1.0" 200 180 0.000159 proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:27:54] "GET /someurl HTTP/1.0" 200 180 0.000075 proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True proxy INFO: (3543398) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543398) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:27:54] "GET /someurl HTTP/1.0" 200 180 0.000078 proxy INFO: wsgi exiting proxy INFO: (3543398) wsgi exited, is_accepting=True Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor test.unit.common.test_splice.TestTee.test_fileno test.unit.common.test_splice.TestTee.test_fileno ... ok test.unit.common.test_splice.TestTee.test_flags_list test.unit.common.test_splice.TestTee.test_flags_list ... ok test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe ... ok test.unit.common.test_splice.TestTee.test_unavailable test.unit.common.test_splice.TestTee.test_unavailable ... ok test.unit.common.test_splice.TestTee.test_unavailable_in_libc test.unit.common.test_splice.TestTee.test_unavailable_in_libc ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst test.unit.common.test_utils.TestEventletRateLimiter.test_burst ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_init test.unit.common.test_utils.TestEventletRateLimiter.test_init ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking test.unit.common.test_utils.TestPipeMutex.test_blocking ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool ... ok test.unit.common.test_utils.TestPipeMutex.test_nonblocking test.unit.common.test_utils.TestPipeMutex.test_nonblocking ... ok test.unit.common.test_utils.TestPipeMutex.test_recursive test.unit.common.test_utils.TestPipeMutex.test_recursive ... ok test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire ... ok test.unit.common.test_utils.TestPipeMutex.test_too_many_releases test.unit.common.test_utils.TestPipeMutex.test_too_many_releases ... ok test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default ... ok test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 370, in reap_container headers, objects = direct_get_container( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 171, in fake_direct_get_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.10.10.1:6202/sda3 test-reaper ERROR: ERROR: invalid storage policy index: 2 test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail ... ok test.unit.account.test_reaper.TestReaper.test_reap_delay test.unit.account.test_reaper.TestReaper.test_reap_delay ... ok test.unit.account.test_reaper.TestReaper.test_reap_device test.unit.account.test_reaper.TestReaper.test_reap_device ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts ... ok test.unit.account.test_reaper.TestReaper.test_reap_object test.unit.account.test_reaper.TestReaper.test_reap_object ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_fail test.unit.account.test_reaper.TestReaper.test_reap_object_fail ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_object_timeout ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set ... ok test.unit.account.test_reaper.TestReaper.test_reset_stats test.unit.account.test_reaper.TestReaper.test_reset_stats ... ok test.unit.account.test_reaper.TestReaper.test_run_forever test.unit.account.test_reaper.TestReaper.test_run_forever ... ok test.unit.account.test_reaper.TestReaper.test_run_once test.unit.account.test_reaper.TestReaper.test_run_once ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys ... ok Finished. Marked 0 objects for deletion. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long ... ok Reloaded swift-proxy-server test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE ... ok =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 5 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 199, in HEAD return self._handle_acl(app, 'DELETE') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 141, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'X-Amz-Acl': 'public-read', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Grant AllUsers READ permission on the bucket /bucket test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter ... ok test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00070 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpjbiqw534/sda/accounts/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.01054 seconds (94.86281/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00070 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00076 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00056 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sda/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'ed2c8e53-698a-4b50-a15f-f124e1743b55-sda', '1717345674.58514', '1717345674.58215', '0', '') test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp86sdznap/sda/accounts/0/cca/7e85f39cfbd92b74b0b21a19f4188cca/7e85f39cfbd92b74b0b21a19f4188cca.db REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', '5c0a66d6-7e38-4696-a0f2-ef67da823100-sda', '1717345674.60401', '1717345674.60028', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('5c0a66d6-7e38-4696-a0f2-ef67da823100-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', '5c0a66d6-7e38-4696-a0f2-ef67da823100-sda', '1717345674.60401', '1717345674.60028', '0', '') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('5c0a66d6-7e38-4696-a0f2-ef67da823100-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09031 seconds (11.07324/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (1, '594dd821c10f85cf100ac9760aa9b4ed', '95229c53-8627-4d2d-809a-52ea32f0d8e8-sda', '1717345674.72800', '1717345674.719', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, rsync_then_merge, ('95229c53-8627-4d2d-809a-52ea32f0d8e8-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (11, 'b39f0f53b684abd465d5ee424584597b', '7e3def85-6e40-45d5-b8b2-0c59b71c5581-sda', '1717345674.82931', '1717345674.82109', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_items, ([{'ROWID': 1, 'name': '/a/c_0', 'put_timestamp': '1717345674.86796', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c_1', 'put_timestamp': '1717345674.86983', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c_2', 'put_timestamp': '1717345674.87019', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c_3', 'put_timestamp': '1717345674.87133', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c_4', 'put_timestamp': '1717345674.87164', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c_5', 'put_timestamp': '1717345674.87195', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c_6', 'put_timestamp': '1717345674.87225', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c_7', 'put_timestamp': '1717345674.87339', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c_8', 'put_timestamp': '1717345674.8737', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c_9', 'put_timestamp': '1717345674.87399', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c_missing', 'put_timestamp': '1717345674.87435', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}], '7e3def85-6e40-45d5-b8b2-0c59b71c5581-sda') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_syncs, ([],) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items ... ok test ERROR: Bad key for 'container': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: get_keys(): from callback: boom: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_crypto_utils.py", line 95, in callback raise Exception('boom') Exception: boom test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Missing key for 'container': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'container' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Did not get a keys dict: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ TypeError: list indices must be integers or slices, not str test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record ... ok test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Cipher must be AES_CTR_256 test DEBUG: No decryption is necessary because of override test DEBUG: No decryption is necessary because of override test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_decrypter.py", line 902, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: Error decrypting header X-Object-Sysmeta-Crypto-Etag: Missing crypto meta in value fgBd78GL6kACmfmuAxTz9B7eJCFTfjxotcr7trw/5Qo= test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting object: Bad crypto meta: Missing 'cipher' test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Key must be length 32 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting object: Missing 'body_key' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_percent test.unit.common.test_utils.TestFsHasFreeSpace.test_percent ... ok test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding ... ok test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges ... ok Ring Builder file does not exist: /tmp/tmpa120ugt8/tmpv53lk82g Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.2:6200R127.0.0.2:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.1:6200R127.0.0.1:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:6200R[3::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Ring Builder file does not exist: /tmp/tmp_wp5mabs/tmpbfemraxa Partition power increase in progress. You need to finish the increase first before adding devices. Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sda9_"" with 3.14159265359 weight got id 4 Device 4 already uses 127.0.0.1:6200/sda9. The on-disk ring builder is unchanged. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. /tmp/tmpwfiwhked/tmpwc90km1e, build version 4, id d878e4845a9c4bd99ae7fa3279320669 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmpwfiwhked/tmpwc90km1e.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output_id_assigned Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 Device d4r1z2-127.0.0.5:6004R127.0.0.5:6004/sda5_"" with 100.0 weight got id 4 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d0r2z1-127.0.0.6:6005R127.0.0.6:6005/sdb6_"" with 100.0 weight got id 0 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_sorted_output Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. /build/reproducible-path/swift-2.33.0/test/unit/cli/test_ipv6_output Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 21 1 25 1 Partition Matches 2 1 3 1 6 1 8 1 11 1 13 1 15 1 17 1 33 1 36 1 38 1 42 1 53 1 55 1 58 1 63 1 4 1 9 1 14 1 16 1 18 1 20 1 21 1 25 1 26 1 27 1 29 1 32 1 47 1 48 1 49 1 59 1 1 1 7 1 19 1 23 1 24 1 28 1 31 1 34 1 37 1 40 1 44 1 45 1 46 1 57 1 60 1 62 1 Partition Matches 1 1 6 1 7 1 10 1 13 1 14 1 18 1 31 1 35 1 37 1 46 1 49 1 55 1 57 1 59 1 63 1 3 1 8 1 9 1 15 1 22 1 23 1 25 1 28 1 32 1 38 1 39 1 40 1 44 1 45 1 53 1 60 1 0 1 2 1 4 1 5 1 11 1 19 1 20 1 24 1 30 1 33 1 41 1 47 1 51 1 52 1 54 1 62 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 16 1 9 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 30 1 29 1 Specified builder file "/tmp/tmpvj58162d/tmpk5i4tx2m" is not rebalanced yet. Please rebalance first. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted ... ok proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp935rsbpk/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp935rsbpk/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp935rsbpk/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp935rsbpk/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0214 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0153 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0156 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0158 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx67dc944c494e4f0aa5274-00665c9d8a" "proxy-server 3543404" 0.0034 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:46887/sda1/1/a/c" "tx67dc944c494e4f0aa5274-00665c9d8a" "container-server 3543404" 0.0025 "-" 3543404 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:37753/sdb1/1/a/c" "tx67dc944c494e4f0aa5274-00665c9d8a" "container-server 3543404" 0.0022 "-" 3543404 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx67dc944c494e4f0aa5274-00665c9d8a" "proxy-server 3543404" 0.0510 "-" 3543404 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx67dc944c494e4f0aa5274-00665c9d8a" "proxy-server 3543404" 0.0824 "-" 3543404 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/54 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx67dc944c494e4f0aa5274-00665c9d8a - 0.1020 - - 1717345674.168346405 1717345674.270382881 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txffd8db16d1b34d87810eb-00665c9d8a" "proxy-server 3543404" 0.0021 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:46887/sda1/1/a1/c1" "txffd8db16d1b34d87810eb-00665c9d8a" "container-server 3543404" 0.0043 "-" 3543404 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:37753/sdb1/1/a1/c1" "txffd8db16d1b34d87810eb-00665c9d8a" "container-server 3543404" 0.0026 "-" 3543404 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txffd8db16d1b34d87810eb-00665c9d8a" "proxy-server 3543404" 0.0371 "-" 3543404 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txffd8db16d1b34d87810eb-00665c9d8a" "proxy-server 3543404" 0.0541 "-" 3543404 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/54 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txffd8db16d1b34d87810eb-00665c9d8a - 0.0684 - - 1717345674.273662806 1717345674.342096806 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3224cb0a72c0411994fcb-00665c9d8a" "proxy-server 3543404" 0.0137 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:37753/sdb1/2/a/c1" "tx3224cb0a72c0411994fcb-00665c9d8a" "container-server 3543404" 0.0024 "-" 3543404 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:46887/sda1/2/a/c1" "tx3224cb0a72c0411994fcb-00665c9d8a" "container-server 3543404" 0.0121 "-" 3543404 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx3224cb0a72c0411994fcb-00665c9d8a" "proxy-server 3543404" 0.0406 "-" 3543404 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx3224cb0a72c0411994fcb-00665c9d8a" "proxy-server 3543404" 0.0621 "-" 3543404 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/54 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx3224cb0a72c0411994fcb-00665c9d8a - 0.0914 - - 1717345674.345136881 1717345674.436512232 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb46067b59feb4779baf7a-00665c9d8a" "proxy-server 3543404" 0.0101 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:46887/sda1/1/a/c2" "txb46067b59feb4779baf7a-00665c9d8a" "container-server 3543404" 0.0022 "-" 3543404 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:37753/sdb1/1/a/c2" "txb46067b59feb4779baf7a-00665c9d8a" "container-server 3543404" 0.0212 "-" 3543404 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txb46067b59feb4779baf7a-00665c9d8a" "proxy-server 3543404" 0.0510 "-" 3543404 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:54 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txb46067b59feb4779baf7a-00665c9d8a" "proxy-server 3543404" 0.0779 "-" 3543404 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/54 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txb46067b59feb4779baf7a-00665c9d8a - 0.1006 - - 1717345674.438844204 1717345674.539395809 2 test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, rejecting request with error: 401 Unauthorized test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/service.py", line 35, in GET resp = req.get_response(self.app, query={'format': 'json'}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware user quit No .MockServer object at 0x7f75d9e6ab50> running .MockServerFactory.MockServer object at 0x7f75d9bb7f10> (1) appears to have stopped No .MockServerFactory.MockServer object at 0x7f75d9bb73d0> running No .MockServerFactory.MockServer object at 0x7f75d9bb7650> running Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f75d9bb7f10> to die; giving up Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f75d9a780d0> to die; killing Signal .MockServerFactory.MockServer object at 0x7f75d9a780d0> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f75d9bb7cd0> to die; killing Signal .MockServerFactory.MockServer object at 0x7f75d9bb7cd0> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f75d9bb7c10> to die; killing Signal .MockServerFactory.MockServer object at 0x7f75d9bb7c10> pid: None signal: 9 WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify memory limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10127878189086914 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10260891914367676 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10299444198608398 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10343647003173828 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.1038825511932373 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10431146621704102 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10907483100891113 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10956144332885742 test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0002710819244385 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0008563995361328 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0012099742889404 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0015435218811035 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0019726753234863 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0023157596588135 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0026237964630127 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0030243396759033 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout connecting to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10841131210327148 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 0.00012040138244628906: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test.unit.common.test_memcached.TestMemcached.test_error_disabled test.unit.common.test_memcached.TestMemcached.test_error_disabled ... ok test.unit.common.test_memcached.TestMemcached.test_error_limiting test.unit.common.test_memcached.TestMemcached.test_error_limiting ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid ... ok test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_01 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_02 fake-swift ERROR: Got status 404 listing container /AUTH_test/c fake-swift ERROR: Too many bytes for /v1/AUTH_test/mancon/manifest; truncating in /v1/AUTH_test/c/seg_03 with 5 bytes left fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, max LO GET time of 86400s exceeded fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had a21075a36eeddd084e17611a238c7101, but object MD5 was actually 3902ee7e149eb8313a34757e89e21af6 fake-swift ERROR: Bad response length for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/c/seg_01 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp4424dgst/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp4424dgst/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp4424dgst/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp4424dgst/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0183 "-" 3543392 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0244 "-" 3543392 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0169 "-" 3543392 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0135 "-" 3543392 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txfdb79c760b204cb38785e-00665c9d8e" "proxy-server 3543392" 0.0023 "-" 3543392 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:37893/sdb1/0/a/c" "txfdb79c760b204cb38785e-00665c9d8e" "container-server 3543392" 0.0021 "-" 3543392 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:60715/sda1/0/a/c" "txfdb79c760b204cb38785e-00665c9d8e" "container-server 3543392" 0.0019 "-" 3543392 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txfdb79c760b204cb38785e-00665c9d8e" "proxy-server 3543392" 0.0331 "-" 3543392 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txfdb79c760b204cb38785e-00665c9d8e" "proxy-server 3543392" 0.0536 "-" 3543392 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/58 PUT /v1/a/c HTTP/1.0 201 - - t - - - txfdb79c760b204cb38785e-00665c9d8e - 0.0685 - - 1717345678.295158625 1717345678.363639832 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "HEAD /sda1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx0236128fc3234687a8eb8-00665c9d8e" "proxy-server 3543392" 0.0023 "-" 3543392 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:37893/sdb1/2/a1/c1" "tx0236128fc3234687a8eb8-00665c9d8e" "container-server 3543392" 0.0031 "-" 3543392 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:60715/sda1/2/a1/c1" "tx0236128fc3234687a8eb8-00665c9d8e" "container-server 3543392" 0.0031 "-" 3543392 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx0236128fc3234687a8eb8-00665c9d8e" "proxy-server 3543392" 0.0282 "-" 3543392 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx0236128fc3234687a8eb8-00665c9d8e" "proxy-server 3543392" 0.0460 "-" 3543392 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/58 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx0236128fc3234687a8eb8-00665c9d8e - 0.0593 - - 1717345678.366147518 1717345678.425437689 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe318c37f0f9d4a19a86cb-00665c9d8e" "proxy-server 3543392" 0.0144 "-" 3543392 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:60715/sda1/0/a/c1" "txe318c37f0f9d4a19a86cb-00665c9d8e" "container-server 3543392" 0.0036 "-" 3543392 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:37893/sdb1/0/a/c1" "txe318c37f0f9d4a19a86cb-00665c9d8e" "container-server 3543392" 0.0110 "-" 3543392 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe318c37f0f9d4a19a86cb-00665c9d8e" "proxy-server 3543392" 0.0515 "-" 3543392 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe318c37f0f9d4a19a86cb-00665c9d8e" "proxy-server 3543392" 0.0378 "-" 3543392 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/58 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txe318c37f0f9d4a19a86cb-00665c9d8e - 0.0799 - - 1717345678.427843571 1717345678.507713795 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3dc7d4184bc94589b1620-00665c9d8e" "proxy-server 3543392" 0.0103 "-" 3543392 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:37893/sdb1/2/a/c2" "tx3dc7d4184bc94589b1620-00665c9d8e" "container-server 3543392" 0.0019 "-" 3543392 2 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container ... ok test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.test_locale.test_locale.TestTranslations.test_translations test.unit.test_locale.test_locale.TestTranslations.test_translations ... ok test INFO: Begin account audit "once" mode test ERROR: Audit Failed for /tmp/tmpb13pksxa/sda/accounts/0/0/0/test.db: The total container_count for the account a (3) does not match the sum of container_count across policies (2) test INFO: Account audit "once" mode completed: 0.00s test INFO: Disabling account_sdb test INFO: Disabling container_sdb test INFO: Disabling object_sdb test INFO: Enabling account_sdb test INFO: Enabling container_sdb test INFO: Enabling object_sdb {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f886490d360>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware test ERROR: Error retrieving recon data: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 228, in raise_Exception raise Exception Exception test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test ERROR: Error parsing recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 225, in raise_ValueError raise ValueError ValueError test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star test.unit.common.test_swob.TestConditionalIfMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: nojunk None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Fri, 01 Apr 2014 12:00:00 GMT', 'X-Amz-Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000'} test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000'} test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000'} test DEBUG: checking permission: bucket None POST {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000'} test DEBUG: bucket None POST test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version ... ok test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST ... ok test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name ... ok test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file ... ok test.unit.common.test_utils.TestStringAlong.test_happy test.unit.common.test_utils.TestStringAlong.test_happy ... ok test.unit.common.test_utils.TestStringAlong.test_unhappy test.unit.common.test_utils.TestStringAlong.test_unhappy ... ok test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init test.unit.obj.test_diskfile.TestDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking ... ok test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:60715/sda1/2/a/c2" "tx3dc7d4184bc94589b1620-00665c9d8e" "container-server 3543392" 0.0101 "-" 3543392 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3dc7d4184bc94589b1620-00665c9d8e" "proxy-server 3543392" 0.0357 "-" 3543392 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3dc7d4184bc94589b1620-00665c9d8e" "proxy-server 3543392" 0.0534 "-" 3543392 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/27/58 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx3dc7d4184bc94589b1620-00665c9d8e - 0.0767 - - 1717345678.510460377 1717345678.587113619 2 test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object ... ok test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestTpoolSize.test_default_config test.unit.obj.test_server.TestTpoolSize.test_default_config ... ok test.unit.obj.test_server.TestTpoolSize.test_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting ... ok test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted ... ok test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing ... ok test.unit.obj.test_updater.TestObjectUpdater.test_conf_params test.unit.obj.test_updater.TestObjectUpdater.test_conf_params ... ok test.unit.obj.test_updater.TestObjectUpdater.test_creation test.unit.obj.test_updater.TestObjectUpdater.test_creation ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission ... ok test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0310 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0142 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0237 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0254 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0219 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0123 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0210 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0152 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0001 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0172 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0154 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0186 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0030 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0119 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0114 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0191 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0205 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0017 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0139 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0148 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0274 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0231 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0159 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0035 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0189 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0166 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0024 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0265 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0733 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0160 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0116 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0140 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0151 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0153 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0259 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0174 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0120 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0120 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0426 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0034 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0036 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0197 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:55 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0138 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0262 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0099 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0178 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0158 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0024 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0268 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0062 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0125 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0107 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0131 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0147 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0200 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0137 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0153 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0163 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0113 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0112 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0099 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0027 "-" 3543414 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0130 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0170 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0140 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0137 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0182 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0028 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0113 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0115 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0122 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0111 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0175 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0039 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0120 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0289 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0226 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0203 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0190 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0029 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0197 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0040 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0139 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0237 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0018 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0164 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0157 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0003 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0255 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0083 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0331 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0151 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0105 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0111 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0172 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0121 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0154 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0101 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0115 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0041 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0141 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0208 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0095 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0148 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0104 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0146 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0128 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0015 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0052 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0003 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0189 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0089 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0217 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0038 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0108 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0183 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0035 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0021 "-" 3543384 - test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestAccountController.test_PUT_auto_create test.unit.account.test_server.TestAccountController.test_PUT_auto_create ... ok test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_not_found test.unit.account.test_server.TestAccountController.test_PUT_not_found ... ok test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestAccountController.test__call__raise_timeout test.unit.account.test_server.TestAccountController.test__call__raise_timeout ... ok test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD ... ok test.unit.account.test_server.TestAccountController.test_correct_allowed_method test.unit.account.test_server.TestAccountController.test_correct_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies ... ok test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' - rejecting request fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_json test.unit.common.middleware.test_xprofile.TestStats2.test_to_json ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.test_swob.TestRange.test_range test.unit.common.test_swob.TestRange.test_range ... ok test.unit.common.test_swob.TestRange.test_range_invalid_syntax test.unit.common.test_swob.TestRange.test_range_invalid_syntax ... ok test.unit.common.test_swob.TestRange.test_ranges_for_large_length test.unit.common.test_swob.TestRange.test_ranges_for_large_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length test.unit.common.test_swob.TestRange.test_ranges_for_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_edges test.unit.common.test_swob.TestRange.test_ranges_for_length_edges ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_multi test.unit.common.test_swob.TestRange.test_ranges_for_length_multi ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many ... ok test.unit.common.test_swob.TestRange.test_str test.unit.common.test_swob.TestRange.test_str ... ok test.unit.common.test_swob.TestRange.test_upsidedown_range test.unit.common.test_swob.TestRange.test_upsidedown_range ... ok test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation ... ok test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats ... ok test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates test.unit.common.test_utils.TestStatsdLogging.test_sample_rates ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor ... ok test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 ... ok test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation ... ok test.unit.common.test_utils.TestStatsdLogging.test_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_timing_stats ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals ... ok test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_chexor test.unit.container.test_backend.TestContainerBroker.test_chexor ... ok test.unit.container.test_backend.TestContainerBroker.test_create_broker test.unit.container.test_backend.TestContainerBroker.test_create_broker ... ok test.unit.container.test_backend.TestContainerBroker.test_creation test.unit.container.test_backend.TestContainerBroker.test_creation ... ok test.unit.container.test_backend.TestContainerBroker.test_db_file test.unit.container.test_backend.TestContainerBroker.test_db_file ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_object test.unit.container.test_backend.TestContainerBroker.test_delete_object ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBroker.test_empty test.unit.container.test_backend.TestContainerBroker.test_empty ... ok test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_enable_sharding test.unit.container.test_backend.TestContainerBroker.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBroker.test_exception test.unit.container.test_backend.TestContainerBroker.test_exception ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_brokers test.unit.container.test_backend.TestContainerBroker.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBroker.test_get_db_state test.unit.container.test_backend.TestContainerBroker.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info test.unit.container.test_backend.TestContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBroker.test_get_items_since test.unit.container.test_backend.TestContainerBroker.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces test.unit.container.test_backend.TestContainerBroker.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects test.unit.container.test_backend.TestContainerBroker.test_get_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBroker.test_get_policy_info test.unit.container.test_backend.TestContainerBroker.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_replication_info test.unit.container.test_backend.TestContainerBroker.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_report_info test.unit.container.test_backend.TestContainerBroker.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBroker.test_is_deleted test.unit.container.test_backend.TestContainerBroker.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_last_modified_time test.unit.container.test_backend.TestContainerBroker.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items test.unit.container.test_backend.TestContainerBroker.test_merge_items ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBroker.test_newid test.unit.container.test_backend.TestContainerBroker.test_newid ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBroker.test_path test.unit.container.test_backend.TestContainerBroker.test_path ... ok test.unit.container.test_backend.TestContainerBroker.test_pending_file_name test.unit.container.test_backend.TestContainerBroker.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object test.unit.container.test_backend.TestContainerBroker.test_put_object ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim test.unit.container.test_backend.TestContainerBroker.test_reclaim ... ok test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/27/55 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0017 - - 1717345675.306472301 1717345675.308202267 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/27/55 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0017 - - 1717345675.309338331 1717345675.311034203 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'Content-Md5': 'Je0by0I7C3IA9IX8X/ccjg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '25ed1bcb423b0b7200f485fc5ff71c8e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'AUTH_a.example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized ... ok test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.0.0.0:1000/sda: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 490, in reap_object direct_delete_object( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 160, in fake_direct_delete_object raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.0.0.0:1000/sda test-reaper ERROR: Timeout Exception with 10.0.0.1:1001/sdb test-reaper ERROR: Timeout Exception with 10.0.0.2:1002/sdc test-reaper ERROR: Timeout Exception with 10.0.0.3:1003/sdd test-reaper DEBUG: Begin devices pass: thisdeviceisbad test-reaper ERROR: Exception in top-level account reaper loop: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 135, in run_once for device in os.listdir(self.devices): ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'thisdeviceisbad' test-reaper INFO: Devices pass completed: 0.00s test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 ... ok test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test1 STDOUT: test2 STDERR: test4 test5 STDERR: test6 test8 STDOUT: a#012b#012c STDOUT: d test.unit.common.test_utils.TestUtils.test_LoggerFileObject test.unit.common.test_utils.TestUtils.test_LoggerFileObject ... ok proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpif98vh9o/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpif98vh9o/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpif98vh9o/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpif98vh9o/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0127 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0153 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0157 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0181 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb8d5f391222d416896b7f-00665c9d94" "proxy-server 3543404" 0.0020 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:53633/sdb1/1/a/c" "txb8d5f391222d416896b7f-00665c9d94" "container-server 3543404" 0.0030 "-" 3543404 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:35437/sda1/1/a/c" "txb8d5f391222d416896b7f-00665c9d94" "container-server 3543404" 0.0016 "-" 3543404 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txb8d5f391222d416896b7f-00665c9d94" "proxy-server 3543404" 0.0408 "-" 3543404 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txb8d5f391222d416896b7f-00665c9d94" "proxy-server 3543404" 0.0242 "-" 3543404 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/04 PUT /v1/a/c HTTP/1.0 201 - - t - - - txb8d5f391222d416896b7f-00665c9d94 - 0.0541 - - 1717345684.150130987 1717345684.204217196 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txe59818573fb14a96af951-00665c9d94" "proxy-server 3543404" 0.0018 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35437/sda1/1/a1/c1" "txe59818573fb14a96af951-00665c9d94" "container-server 3543404" 0.0020 "-" 3543404 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:53633/sdb1/1/a1/c1" "txe59818573fb14a96af951-00665c9d94" "container-server 3543404" 0.0018 "-" 3543404 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe59818573fb14a96af951-00665c9d94" "proxy-server 3543404" 0.0429 "-" 3543404 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe59818573fb14a96af951-00665c9d94" "proxy-server 3543404" 0.0289 "-" 3543404 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/04 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txe59818573fb14a96af951-00665c9d94 - 0.0556 - - 1717345684.206290722 1717345684.261888504 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txde4f9b254b56457687aa3-00665c9d94" "proxy-server 3543404" 0.0117 "-" 3543404 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:53633/sdb1/2/a/c1" "txde4f9b254b56457687aa3-00665c9d94" "container-server 3543404" 0.0146 "-" 3543404 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:35437/sda1/2/a/c1" "txde4f9b254b56457687aa3-00665c9d94" "container-server 3543404" 0.0017 "-" 3543404 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txde4f9b254b56457687aa3-00665c9d94" "proxy-server 3543404" 0.0347 "-" 3543404 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txde4f9b254b56457687aa3-00665c9d94" "proxy-server 3543404" 0.0507 "-" 3543404 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/04 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txde4f9b254b56457687aa3-00665c9d94 - 0.0725 - - 1717345684.264208794 1717345684.336734295 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx90236a63bfa54e21a251d-00665c9d94" "proxy-server 3543404" 0.0099 "-" 3543404 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:35437/sda1/1/a/c2" "tx90236a63bfa54e21a251d-00665c9d94" "container-server 3543404" 0.0100 "-" 3543404 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:53633/sdb1/1/a/c2" "tx90236a63bfa54e21a251d-00665c9d94" "container-server 3543404" 0.0020 "-" 3543404 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx90236a63bfa54e21a251d-00665c9d94" "proxy-server 3543404" 0.0412 "-" 3543404 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:04 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx90236a63bfa54e21a251d-00665c9d94" "proxy-server 3543404" 0.0621 "-" 3543404 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/04 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx90236a63bfa54e21a251d-00665c9d94 - 0.0825 - - 1717345684.339017391 1717345684.421530008 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission ... ok test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.4:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method get: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config ... ok test.unit.common.test_memcached.TestMemcached.test_error_raising test.unit.common.test_memcached.TestMemcached.test_error_raising ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns test.unit.common.test_memcached.TestMemcached.test_get_conns ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname ... ok test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request ... ok I like ham STDERR: --- Logging error --- STDERR: Traceback (most recent call last): STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 845, in emit raise socket.error(errno.ENOTCONN, "This is an ex-syslog") STDERR: OSError: [Errno 107] This is an ex-syslog STDERR: Call stack: STDERR: File "", line 198, in _run_module_as_main STDERR: File "", line 88, in _run_code STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 88, in main() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 82, in main program.TestProgram( STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 197, in __init__ self.runTests() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 247, in runTests self.result = testRunner.run(self.test) STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 51, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/case.py", line 678, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/case.py", line 623, in run self._callTestMethod(testMethod) STDERR: File "/usr/lib/python3.11/unittest/case.py", line 579, in _callTestMethod if method() is not None: STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 877, in test_LoggerFileObject_recursion logger.info("I like ham") STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1489, in info self._log(INFO, msg, args, **kwargs) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1634, in _log self.handle(record) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1644, in handle self.callHandlers(record) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1706, in callHandlers hdlr.handle(record) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 978, in handle self.emit(record) STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 847, in emit self.handleError(record) STDERR: Message: 'I like ham' Arguments: () test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion ... ok test.unit.common.test_utils.TestUtils.test_NullLogger test.unit.common.test_utils.TestUtils.test_NullLogger ... ok test.unit.common.test_utils.TestUtils.test_TRUE_VALUES test.unit.common.test_utils.TestUtils.test_TRUE_VALUES ... ok test.unit.common.test_utils.TestUtils.test_backwards test.unit.common.test_utils.TestUtils.test_backwards ... ok test.unit.common.test_utils.TestUtils.test_cache_from_env test.unit.common.test_utils.TestUtils.test_cache_from_env ... ok test.unit.common.test_utils.TestUtils.test_cap_length test.unit.common.test_utils.TestUtils.test_cap_length ... ok test.unit.common.test_utils.TestUtils.test_capture_stdio test.unit.common.test_utils.TestUtils.test_capture_stdio ... ok test.unit.common.test_utils.TestUtils.test_clean_content_type test.unit.common.test_utils.TestUtils.test_clean_content_type ... ok test.unit.common.test_utils.TestUtils.test_clean_logger_exception test.unit.common.test_utils.TestUtils.test_clean_logger_exception ... ok test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene ... ok test.unit.common.test_utils.TestUtils.test_config_auto_int_value test.unit.common.test_utils.TestUtils.test_config_auto_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_fallocate_value test.unit.common.test_utils.TestUtils.test_config_fallocate_value ... ok test.unit.common.test_utils.TestUtils.test_config_float_value test.unit.common.test_utils.TestUtils.test_config_float_value ... ok test.unit.common.test_utils.TestUtils.test_config_percent_value test.unit.common.test_utils.TestUtils.test_config_percent_value ... ok test.unit.common.test_utils.TestUtils.test_config_positive_int_value test.unit.common.test_utils.TestUtils.test_config_positive_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_request_node_count_value test.unit.common.test_utils.TestUtils.test_config_request_node_count_value ... ok test.unit.common.test_utils.TestUtils.test_config_true_value test.unit.common.test_utils.TestUtils.test_config_true_value ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close test.unit.common.test_utils.TestUtils.test_drain_and_close ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges test.unit.common.test_utils.TestUtils.test_drop_privileges ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache test.unit.common.test_utils.TestUtils.test_dump_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner ... ok test.unit.common.test_utils.TestUtils.test_extract_swift_bytes test.unit.common.test_utils.TestUtils.test_extract_swift_bytes ... ok test.unit.common.test_utils.TestUtils.test_find_namespace test.unit.common.test_utils.TestUtils.test_find_namespace ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_large_body test.unit.common.test_utils.TestUtils.test_friendly_close_large_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_small_body test.unit.common.test_utils.TestUtils.test_friendly_close_small_body ... ok test.unit.common.test_utils.TestUtils.test_fsync_dir test.unit.common.test_utils.TestUtils.test_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_generate_trans_id test.unit.common.test_utils.TestUtils.test_generate_trans_id ... ok test.unit.common.test_utils.TestUtils.test_get_db_files test.unit.common.test_utils.TestUtils.test_get_db_files ... ok test.unit.common.test_utils.TestUtils.test_get_log_line test.unit.common.test_utils.TestUtils.test_get_log_line ... ok test.unit.common.test_utils.TestUtils.test_get_logger test.unit.common.test_utils.TestUtils.test_get_logger ... ok test.unit.common.test_utils.TestUtils.test_get_logger_console test.unit.common.test_utils.TestUtils.test_get_logger_console ... ok test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route ... ok test.unit.common.test_utils.TestUtils.test_get_partition_for_hash test.unit.common.test_utils.TestUtils.test_get_partition_for_hash ... ok test.unit.common.test_utils.TestUtils.test_get_partition_from_path test.unit.common.test_utils.TestUtils.test_get_partition_from_path ... ok test.unit.common.test_utils.TestUtils.test_get_policy_index test.unit.common.test_utils.TestUtils.test_get_policy_index ... ok test.unit.common.test_utils.TestUtils.test_get_redirect_data test.unit.common.test_utils.TestUtils.test_get_redirect_data ... ok test.unit.common.test_utils.TestUtils.test_get_trans_id_time test.unit.common.test_utils.TestUtils.test_get_trans_id_time ... ok test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str ... ok test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string ... ok test.unit.common.test_utils.TestUtils.test_hash_path test.unit.common.test_utils.TestUtils.test_hash_path ... ok test.unit.common.test_utils.TestUtils.test_human_readable test.unit.common.test_utils.TestUtils.test_human_readable ... ok test.unit.common.test_utils.TestUtils.test_is_file_older test.unit.common.test_utils.TestUtils.test_is_file_older ... ok test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_error test.unit.common.test_utils.TestUtils.test_ismount_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_root test.unit.common.test_utils.TestUtils.test_ismount_path_is_root ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_dev test.unit.common.test_utils.TestUtils.test_ismount_successes_dev ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_ino test.unit.common.test_utils.TestUtils.test_ismount_successes_ino ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists ... ok test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists ... ok test ERROR: ERROR: swift.cache could not be found in env! test ERROR: ERROR: swift.cache could not be found in env! test WARNING: Unable to perform fsync() on directory /tmp/tmp7vrt7zn_: Bad file descriptor Unable to locate some_not_real_function in libc. Leaving as a no-op. test.unit.common.test_utils.TestUtils.test_load_libc_function test.unit.common.test_utils.TestUtils.test_load_libc_function ... ok test.unit.common.test_utils.TestUtils.test_load_pkg_resource test.unit.common.test_utils.TestUtils.test_load_pkg_resource ... skipped 'pkg_resources loading is only available on python 3.7 and earlier' test.unit.common.test_utils.TestUtils.test_load_recon_cache test.unit.common.test_utils.TestUtils.test_load_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_lock_file test.unit.common.test_utils.TestUtils.test_lock_file ... ok test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink ... ok test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail ... ok test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open ... ok test.unit.common.test_utils.TestUtils.test_lock_path test.unit.common.test_utils.TestUtils.test_lock_path ... ok test.unit.common.test_utils.TestUtils.test_lock_path_class test.unit.common.test_utils.TestUtils.test_lock_path_class ... ok test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit ... ok test.unit.common.test_utils.TestUtils.test_lock_path_name test.unit.common.test_utils.TestUtils.test_lock_path_name ... ok test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps ... ok test.unit.common.test_utils.TestUtils.test_log_string_formatter test.unit.common.test_utils.TestUtils.test_log_string_formatter ... ok test.unit.common.test_utils.TestUtils.test_majority_size test.unit.common.test_utils.TestUtils.test_majority_size ... ok test.unit.common.test_utils.TestUtils.test_make_db_file_path test.unit.common.test_utils.TestUtils.test_make_db_file_path ... ok test.unit.common.test_utils.TestUtils.test_makedirs_count test.unit.common.test_utils.TestUtils.test_makedirs_count ... ok test.unit.common.test_utils.TestUtils.test_md5_with_data test.unit.common.test_utils.TestUtils.test_md5_with_data ... ok test.unit.common.test_utils.TestUtils.test_md5_without_data test.unit.common.test_utils.TestUtils.test_md5_without_data ... ok test.unit.common.test_utils.TestUtils.test_mkdirs test.unit.common.test_utils.TestUtils.test_mkdirs ... ok test.unit.common.test_utils.TestUtils.test_monkey_patch test.unit.common.test_utils.TestUtils.test_monkey_patch ... ok test.unit.common.test_utils.TestUtils.test_nested_prefixlogger test.unit.common.test_utils.TestUtils.test_nested_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_node_to_string test.unit.common.test_utils.TestUtils.test_node_to_string ... ok test.unit.common.test_utils.TestUtils.test_non_negative_float test.unit.common.test_utils.TestUtils.test_non_negative_float ... ok test.unit.common.test_utils.TestUtils.test_non_negative_int test.unit.common.test_utils.TestUtils.test_non_negative_int ... ok test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision ... ok test.unit.common.test_utils.TestUtils.test_normalize_timestamp test.unit.common.test_utils.TestUtils.test_normalize_timestamp ... ok test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_content_type test.unit.common.test_utils.TestUtils.test_parse_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_db_filename test.unit.common.test_utils.TestUtils.test_parse_db_filename ... ok test.unit.common.test_utils.TestUtils.test_parse_options test.unit.common.test_utils.TestUtils.test_parse_options ... ok test.unit.common.test_utils.TestUtils.test_parse_options_errors test.unit.common.test_utils.TestUtils.test_parse_options_errors ... ok test.unit.common.test_utils.TestUtils.test_parse_override_options test.unit.common.test_utils.TestUtils.test_parse_override_options ... ok test.unit.common.test_utils.TestUtils.test_prefixlogger test.unit.common.test_utils.TestUtils.test_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_quorum_size test.unit.common.test_utils.TestUtils.test_quorum_size ... ok test.unit.common.test_utils.TestUtils.test_quote_bytes test.unit.common.test_utils.TestUtils.test_quote_bytes ... ok test.unit.common.test_utils.TestUtils.test_quote_unicode test.unit.common.test_utils.TestUtils.test_quote_unicode ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep ... ok test.unit.common.test_utils.TestUtils.test_readconf test.unit.common.test_utils.TestUtils.test_readconf ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir test.unit.common.test_utils.TestUtils.test_readconf_dir ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files ... ok test.unit.common.test_utils.TestUtils.test_readconf_raw test.unit.common.test_utils.TestUtils.test_readconf_raw ... ok test.unit.common.test_utils.TestUtils.test_remove_directory test.unit.common.test_utils.TestUtils.test_remove_directory ... ok test.unit.common.test_utils.TestUtils.test_remove_file test.unit.common.test_utils.TestUtils.test_remove_file ... ok test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false ... ok test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_replace_partition_in_path test.unit.common.test_utils.TestUtils.test_replace_partition_in_path ... ok test.unit.common.test_utils.TestUtils.test_round_robin_iter test.unit.common.test_utils.TestUtils.test_round_robin_iter ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip ... ok test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation ... ok test.unit.common.test_utils.TestUtils.test_safe_json_loads test.unit.common.test_utils.TestUtils.test_safe_json_loads ... ok test.unit.common.test_utils.TestUtils.test_search_tree test.unit.common.test_utils.TestUtils.test_search_tree ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match ... ok test.unit.common.test_utils.TestUtils.test_select_node_ip test.unit.common.test_utils.TestUtils.test_select_node_ip ... ok test.unit.common.test_utils.TestUtils.test_split_path test.unit.common.test_utils.TestUtils.test_split_path ... ok test.unit.common.test_utils.TestUtils.test_storage_directory test.unit.common.test_utils.TestUtils.test_storage_directory ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer test.unit.common.test_utils.TestUtils.test_str_anonymizer ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness ... ok test.unit.common.test_utils.TestUtils.test_str_format_time test.unit.common.test_utils.TestUtils.test_str_format_time ... ok test.unit.common.test_utils.TestUtils.test_streq_const_time test.unit.common.test_utils.TestUtils.test_streq_const_time ... ok test.unit.common.test_utils.TestUtils.test_strict_b64decode test.unit.common.test_utils.TestUtils.test_strict_b64decode ... ok test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter test.unit.common.test_utils.TestUtils.test_swift_log_formatter ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length ... ok test.unit.common.test_utils.TestUtils.test_systemd_notify test.unit.common.test_utils.TestUtils.test_systemd_notify ... ok test.unit.common.test_utils.TestUtils.test_validate_device_partition test.unit.common.test_utils.TestUtils.test_validate_device_partition ... ok test.unit.common.test_utils.TestUtils.test_validate_hash_conf test.unit.common.test_utils.TestUtils.test_validate_hash_conf ... ok test.unit.common.test_utils.TestUtils.test_validate_sync_to test.unit.common.test_utils.TestUtils.test_validate_sync_to ... ok test.unit.common.test_utils.TestUtils.test_write_file test.unit.common.test_utils.TestUtils.test_write_file ... ok test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 ... ok test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_base.TestGetterSource.test_close test.unit.proxy.controllers.test_base.TestGetterSource.test_close ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_init test.unit.proxy.controllers.test_base.TestGetterSource.test_init ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_sort test.unit.proxy.controllers.test_base.TestGetterSource.test_sort ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET ... ok test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test INFO: - - 02/Jun/2024/16/28/01 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345681.107504368 1717345681.107817888 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0014 - - 1717345681.107428074 1717345681.108785391 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717345681.111168861 1717345681.111624956 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0016 - - 1717345681.111059666 1717345681.112708807 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.115867138 1717345681.116240978 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0013 - - 1717345681.115816355 1717345681.117136240 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0040 FA - 1717345681.120210886 1717345681.124173880 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0051 - - 1717345681.120101213 1717345681.125168085 0 test INFO: - - 02/Jun/2024/16/28/01 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.128010988 1717345681.128443956 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0017 - - 1717345681.127947330 1717345681.129654169 0 test INFO: - - 02/Jun/2024/16/28/01 HEAD /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.132892132 1717345681.133270741 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0006 SSC - 1717345681.134085417 1717345681.134639025 0 test INFO: - - 02/Jun/2024/16/28/01 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.135783911 1717345681.136145115 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0047 - - 1717345681.132647753 1717345681.137329817 - test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345681.140021324 1717345681.140351772 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345681.140949488 1717345681.141410828 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0006 FA - 1717345681.142404318 1717345681.142963648 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0045 - - 1717345681.139751434 1717345681.144234657 - test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345681.147176743 1717345681.147501230 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345681.148602724 1717345681.149144650 0 test INFO: - - 02/Jun/2024/16/28/01 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.150233746 1717345681.150591850 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0049 - - 1717345681.146927357 1717345681.151850462 - test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.155091524 1717345681.155506134 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717345681.156141281 1717345681.156609058 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717345681.157437801 1717345681.157839775 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0042 - - 1717345681.154833078 1717345681.158990622 - test INFO: - - 02/Jun/2024/16/28/01 DELETE /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345681.162265778 1717345681.162545919 0 test INFO: - - 02/Jun/2024/16/28/01 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0004 SSC - 1717345681.163227797 1717345681.163625479 0 test INFO: - - 02/Jun/2024/16/28/01 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717345681.164435148 1717345681.164698839 0 test INFO: - - 02/Jun/2024/16/28/01 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0036 - - 1717345681.162010431 1717345681.165592909 - test WARNING: boom test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test ERROR: ERROR: Unable to access foo/bar: permission_denied test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpxl5w71zb/devices/sda1 (pid: 3543408) test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpxl5w71zb/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345697.00000 test INFO: Object update sweep completed on /tmp/tmpxl5w71zb/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_1fo3120/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmp_1fo3120/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345697.00000 test INFO: Object update sweep completed on /tmp/tmp_1fo3120/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7km1sra6/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid ... ok test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0153 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0018 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0113 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0103 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0092 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0107 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0090 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0098 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0104 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0137 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0032 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0027 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0171 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0186 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0030 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0003 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0159 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0040 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0167 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0185 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0127 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0119 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0163 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0138 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0105 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0158 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0179 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0230 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0231 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0145 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0037 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0030 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0218 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0032 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0148 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0134 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0126 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0134 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0170 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0140 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0102 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0165 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0110 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0197 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0265 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0130 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0144 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0058 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0129 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0137 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0127 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0127 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0100 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0110 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0125 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0124 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0122 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0115 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0100 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0124 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0027 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0049 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0161 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0026 "-" 3543414 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync ... ok test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0141 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0864 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0157 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0164 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0188 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0156 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0067 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0014 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.0169 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0138 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0026 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0018 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0139 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0119 "-" 3543384 - test DEBUG: - - - [02/Jun/2024:16:27:57 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3543384 - test DEBUG: - - - [02/Jun/2024:16:27:57 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3543384 - test DEBUG: - - - [02/Jun/2024:16:27:57 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0006 "-" 3543384 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:27:57 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0055 "-" 3543384 - test DEBUG: - - - [02/Jun/2024:16:27:57 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0005 "-" 3543384 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0136 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0158 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0002 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0003 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0179 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0135 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0144 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0178 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0024 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0188 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0033 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0032 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0112 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0142 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0032 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0030 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0033 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0190 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3543384 0 test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0104 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3543384 - test.unit.account.test_server.TestAccountController.test_empty_policy_stats test.unit.account.test_server.TestAccountController.test_empty_policy_stats ... ok test.unit.account.test_server.TestAccountController.test_init test.unit.account.test_server.TestAccountController.test_init ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestAccountController.test_list_allowed_methods test.unit.account.test_server.TestAccountController.test_list_allowed_methods ... ok test.unit.account.test_server.TestAccountController.test_log_line_format test.unit.account.test_server.TestAccountController.test_log_line_format ... ok test.unit.account.test_server.TestAccountController.test_markers_with_reserved test.unit.account.test_server.TestAccountController.test_markers_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestAccountController.test_not_allowed_method test.unit.account.test_server.TestAccountController.test_not_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_null_markers test.unit.account.test_server.TestAccountController.test_null_markers ... ok test.unit.account.test_server.TestAccountController.test_params_format test.unit.account.test_server.TestAccountController.test_params_format ... ok test.unit.account.test_server.TestAccountController.test_params_utf8 test.unit.account.test_server.TestAccountController.test_params_utf8 ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_non_default test.unit.account.test_server.TestAccountController.test_policy_stats_non_default ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_prefix_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestAccountController.test_serv_reserv test.unit.account.test_server.TestAccountController.test_serv_reserv ... ok test.unit.account.test_server.TestAccountController.test_through_call test.unit.account.test_server.TestAccountController.test_through_call ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path test.unit.account.test_server.TestAccountController.test_through_call_invalid_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name ... ok test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/27/57 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0014 - - 1717345677.638987064 1717345677.640408039 0 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/27/57 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0012 - - 1717345677.641700268 1717345677.642862320 1 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Use tempauth middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Use of the auth_admin_prefix, auth_host, auth_port, auth_protocol, identity_uri, admin_token, admin_user, admin_password, and admin_tenant_name configuration options was deprecated in the Mitaka release in favor of an auth_plugin and its related options. This class may be removed in a future release. Configuring admin URI using auth fragments was deprecated in the Kilo release, and will be removed in the Newton release, use 'identity_uri' instead. Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint test DEBUG: Calling S3Api Middleware Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_container test.unit.common.middleware.test_bulk.TestUntar.test_bad_container ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar ... ok test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking ... ok test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_call test.unit.common.middleware.test_bulk.TestUntar.test_extract_call ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works ... ok test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body ... ok test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata ... ok test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_client_exception test.unit.common.test_exceptions.TestExceptions.test_client_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_exception test.unit.common.test_exceptions.TestExceptions.test_replication_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_filters test.unit.common.test_utils.TestAuditLocationGenerator.test_filters ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive ... ok test.unit.common.test_utils.TestLRUCache.test_maxsize test.unit.common.test_utils.TestLRUCache.test_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_maxtime test.unit.common.test_utils.TestLRUCache.test_maxtime ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxsize test.unit.common.test_utils.TestLRUCache.test_set_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxtime test.unit.common.test_utils.TestLRUCache.test_set_maxtime ... ok test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser ... ok test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set ... ok test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_delta test.unit.common.utils.test_timestamp.TestTimestamp.test_delta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_false test.unit.common.utils.test_timestamp.TestTimestamp.test_false ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_float test.unit.common.utils.test_timestamp.TestTimestamp.test_float ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_int test.unit.common.utils.test_timestamp.TestTimestamp.test_int ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_raw test.unit.common.utils.test_timestamp.TestTimestamp.test_raw ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_true test.unit.common.utils.test_timestamp.TestTimestamp.test_true ... ok test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts ... ok test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_remove_objects test.unit.container.test_backend.TestContainerBroker.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object ... ok test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/dontchange', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.aexample.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/ä½\xa0好', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/FOO_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce440>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49672e7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49664e710>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/PREFIX_uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49664e710>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce440>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce7a0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com:8080', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce440>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb49664e710>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage1.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce440>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage2.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce440>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage3.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fb495fce440>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put ... ok test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test.unit.common.test_memcached.TestMemcached.test_incr test.unit.common.test_memcached.TestMemcached.test_incr ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails ... ok test.unit.common.test_memcached.TestMemcached.test_incr_add_expires test.unit.common.test_memcached.TestMemcached.test_incr_add_expires ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response ... ok test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold ... ok test.unit.common.test_memcached.TestMemcached.test_logger_kwarg test.unit.common.test_memcached.TestMemcached.test_logger_kwarg ... ok test.unit.common.test_memcached.TestMemcached.test_multi test.unit.common.test_memcached.TestMemcached.test_multi ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.test_memcached.TestMemcached.test_multi_delete test.unit.common.test_memcached.TestMemcached.test_multi_delete ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_retry test.unit.common.test_memcached.TestMemcached.test_retry ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object ... ok test.unit.common.test_memcached.TestMemcached.test_serialization test.unit.common.test_memcached.TestMemcached.test_serialization ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success ... ok test.unit.common.test_memcached.TestMemcached.test_set_error test.unit.common.test_memcached.TestMemcached.test_set_error ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest ... ok test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_get_json test.unit.common.test_memcached.TestMemcached.test_set_get_json ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough ... ok test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon test.unit.common.middleware.test_tempauth.TestAuth.test_anon ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates ... ok test.unit.common.test_utils.TestParseContentRange.test_bad test.unit.common.test_utils.TestParseContentRange.test_bad ... ok test.unit.common.test_utils.TestParseContentRange.test_good test.unit.common.test_utils.TestParseContentRange.test_good ... ok test.unit.common.test_utils.TestSpliterator.test_big_input_string test.unit.common.test_utils.TestSpliterator.test_big_input_string ... ok test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries ... ok test.unit.common.test_utils.TestSpliterator.test_closing test.unit.common.test_utils.TestSpliterator.test_closing ... ok test.unit.common.test_utils.TestSpliterator.test_no_empty_strings test.unit.common.test_utils.TestSpliterator.test_no_empty_strings ... ok test.unit.common.test_utils.TestSpliterator.test_overlap test.unit.common.test_utils.TestSpliterator.test_overlap ... ok test.unit.common.test_utils.TestSpliterator.test_running_out test.unit.common.test_utils.TestSpliterator.test_running_out ... ok test.unit.common.test_utils.TestSpliterator.test_string test.unit.common.test_utils.TestSpliterator.test_string ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_options_call test.unit.common.middleware.test_tempauth.TestAuth.test_options_call ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors ... ok test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted ... ok test.unit.common.ring.test_utils.TestUtils.test_dispersion_report test.unit.common.ring.test_utils.TestUtils.test_dispersion_report ... ok test.unit.common.ring.test_utils.TestUtils.test_is_local_device test.unit.common.ring.test_utils.TestUtils.test_is_local_device ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table ... ok test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname ... ok test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_args test.unit.common.ring.test_utils.TestUtils.test_parse_args ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_value test.unit.common.ring.test_utils.TestUtils.test_parse_search_value ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_replication_defaults test.unit.common.ring.test_utils.TestUtils.test_replication_defaults ... ok test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args test.unit.common.ring.test_utils.TestUtils.test_validate_args ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param ... ok test.unit.common.test_utils.TestFallocate.test_arg_bounds test.unit.common.test_utils.TestFallocate.test_arg_bounds ... ok test.unit.common.test_utils.TestFallocate.test_fallocate test.unit.common.test_utils.TestFallocate.test_fallocate ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_offset test.unit.common.test_utils.TestFallocate.test_fallocate_offset ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors ... ok test.unit.common.test_utils.TestFallocate.test_no_fallocates_available test.unit.common.test_utils.TestFallocate.test_no_fallocates_available ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset ... ok test.unit.common.test_utils.TestPunchHole.test_arg_bounds test.unit.common.test_utils.TestPunchHole.test_arg_bounds ... ok test.unit.common.test_utils.TestPunchHole.test_error test.unit.common.test_utils.TestPunchHole.test_error ... ok test.unit.common.test_utils.TestPunchHole.test_no_fallocate test.unit.common.test_utils.TestPunchHole.test_no_fallocate ... ok test.unit.common.test_utils.TestPunchHole.test_punch_hole test.unit.common.test_utils.TestPunchHole.test_punch_hole ... ok test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate ... ok test INFO: Begin container audit "once" mode test DEBUG: Audit passed for /tmp/tmpf6hvltyx/sda/containers/0/0/0/test.db test INFO: Container audit "once" mode completed: 0.00s test ERROR: Problem finding shard upper in '/tmp/tmp3ot22wi5/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp3ot22wi5/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpsihiloct/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpsihiloct/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs ... ok [2024-06-02 16:27:54] Checking auditor stats =============================================================================== test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpl1il2fzp/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpl1il2fzp/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpsbv21z5g/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test INFO: Since Wed Dec 31 12:00:00 1969: container audits: 0 passed audit, 1 failed audit test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test ERROR: ERROR Could not get container info /tmp/tmpsbv21z5g/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test DEBUG: Audit passed for test DEBUG: Audit passed for test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpubeziqpo/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test.unit.common.test_db_auditor.TestAuditor.test_run_forever test.unit.common.test_db_auditor.TestAuditor.test_run_forever ... ok test.unit.common.test_db_auditor.TestAuditor.test_run_once test.unit.common.test_db_auditor.TestAuditor.test_run_once ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish ... ok test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers ... ok test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency test.unit.container.test_reconciler.TestReconciler.test_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency_config test.unit.container.test_reconciler.TestReconciler.test_concurrency_config ... ok test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi ... ok test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required ... ok test ERROR: Invalid swift_bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4161, in override_bytes_from_content_type listing_dict['bytes'] = int(swift_bytes) ^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: 'hey' test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: Setting object count to 1032 (txn: txf88d1dd0485943adabc04-00665c9d96) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: Setting object count to 48 (txn: tx51ea056096e74c42978e8-00665c9d96) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: Setting object count to 1064 (txn: tx4adf6a4f7a044ad198463-00665c9d96) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txc60b15486d38457d88334-00665c9d96) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '462'} test DEBUG: Grant test:full_control FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff ... ok fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded test WARNING: Error sending UDP message to ('some.host.com', 8125): [Errno 1] test errno 1 test ERROR: Problem finding shard upper in '/tmp/tmp6oklqrjk/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp6oklqrjk/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpud_pk_7v/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpud_pk_7v/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBroker.test_set_db_states test.unit.container.test_backend.TestContainerBroker.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBroker.test_sync_merging test.unit.container.test_backend.TestContainerBroker.test_sync_merging ... ok test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one ... ok test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found ... ok test.unit.obj.test_auditor.TestAuditor.test_object_run_logging test.unit.obj.test_auditor.TestAuditor.test_object_run_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found ... ok test.unit.obj.test_auditor.TestAuditor.test_run_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found ... ok test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_sleeper test.unit.obj.test_auditor.TestAuditor.test_sleeper ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads ... ok test DEBUG: Update redirected for /a/c/o /tmp/tmp7km1sra6/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_new test DEBUG: Update sent for /.shards_a/c_shard_new/o /tmp/tmp7km1sra6/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 test INFO: Object update sweep completed on /tmp/tmp7km1sra6/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpzqanmm9h/devices/sda1 (pid: 3543408) test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpzqanmm9h/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 test INFO: Object update sweep completed on /tmp/tmpzqanmm9h/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpzqanmm9h/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpzqanmm9h/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_1 test DEBUG: Update sent for /.shards_a/c_shard_1/o /tmp/tmpzqanmm9h/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 test INFO: Object update sweep completed on /tmp/tmpzqanmm9h/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpn_dh4bmz/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_older/o /tmp/tmpn_dh4bmz/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_new test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_new/o /tmp/tmpn_dh4bmz/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_newer test INFO: Object update sweep completed on /tmp/tmpn_dh4bmz/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpn_dh4bmz/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /.shards_a/c_shard_newer/o /tmp/tmpn_dh4bmz/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 test INFO: Object update sweep completed on /tmp/tmpn_dh4bmz/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfynfn7st/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_2 test INFO: Object update sweep completed on /tmp/tmpfynfn7st/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfynfn7st/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_2/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to None test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_3 test INFO: Object update sweep completed on /tmp/tmpfynfn7st/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfynfn7st/devices/sda1 (pid: 3543408) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_3/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 to None test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop ... ok test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0003 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0002 "-" 3543384 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0177 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0091 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0110 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0028 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0029 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0083 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0115 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0189 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0117 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0032 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0161 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0020 "-" 3543384 0 test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0126 "-" 3543384 1 test INFO: - - - [02/Jun/2024:16:27:58 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0111 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0140 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0218 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0101 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0118 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0031 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0130 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0027 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0025 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0004 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0002 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0049 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0226 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0171 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3543384 1 test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0112 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0122 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0028 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0110 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0054 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0138 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0092 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0105 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0100 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0121 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0034 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0144 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0020 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0107 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0105 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0116 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0104 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0029 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0105 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0014 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0091 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0085 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0087 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0021 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0023 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0022 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3543384 - test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 ... ok test.unit.account.test_server.TestAccountController.test_utf8_metadata test.unit.account.test_server.TestAccountController.test_utf8_metadata ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id ... ok test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 ... ok test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info ... ok test.unit.common.middleware.test_acl.TestACL.test_clean_acl test.unit.common.middleware.test_acl.TestACL.test_clean_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_parse_acl test.unit.common.middleware.test_acl.TestACL.test_parse_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_chrome test.unit.common.middleware.test_formpost.TestFormPost.test_chrome ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_explorer test.unit.common.middleware.test_formpost.TestFormPost.test_explorer ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_firefox test.unit.common.middleware.test_formpost.TestFormPost.test_firefox ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_container test.unit.common.middleware.test_formpost.TestFormPost.test_no_container ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_safari test.unit.common.middleware.test_formpost.TestFormPost.test_safari ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:full_control OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account ... ok test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0645 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0126 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0137 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0134 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0131 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0075 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0094 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0132 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0100 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0129 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0173 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0071 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0101 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0110 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0116 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0146 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0020 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0161 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0199 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0177 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0018 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0121 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0210 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0003 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0006 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0002 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0002 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0132 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0148 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0015 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0129 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0086 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0108 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0099 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0176 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0144 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0086 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0063 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0070 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0109 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0064 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0105 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0062 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0071 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0157 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0121 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0018 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:57 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0167 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0106 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0514 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0152 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0032 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0169 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543414 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled ... ok test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware test DEBUG: User tenant_id has system reader authorizing test DEBUG: tenant mismatch: AUTH_tenant_id2 != tenant_id test DEBUG: allow user with role(s) admin,swiftoperator as account admin test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00042 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkprbv_3p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00811 seconds (123.33497/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test DEBUG: Successfully deleted db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test DEBUG: Not deleting db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (2/3 success) test DEBUG: Not deleting db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (requires sharding, state unsharded) test DEBUG: Not deleting db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345692.77402.db (requires sharding, state sharding) test DEBUG: Not deleting db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345692.77402.db (2/3 success) test DEBUG: Successfully deleted db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345692.77402.db test DEBUG: Successfully deleted db /tmp/tmptfz2h5zq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '745665e398dba74a93be99dbd53e3ee0', 'a8098386-a400-496a-ab05-528a0061f71e-sda', '1717345692.87714', '1717345692.00000', '0', '', '1717345692.00000', 51, 0) test DEBUG: /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o0', 'created_at': '1717345693.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o1', 'created_at': '1717345694.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o2', 'created_at': '1717345695.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o3', 'created_at': '1717345696.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o4', 'created_at': '1717345697.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o5', 'created_at': '1717345698.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o6', 'created_at': '1717345699.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o7', 'created_at': '1717345700.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o8', 'created_at': '1717345701.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o9', 'created_at': '1717345702.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a8098386-a400-496a-ab05-528a0061f71e-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 11, 'name': 'o10', 'created_at': '1717345703.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 12, 'name': 'o11', 'created_at': '1717345704.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 13, 'name': 'o12', 'created_at': '1717345705.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 14, 'name': 'o13', 'created_at': '1717345706.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 15, 'name': 'o14', 'created_at': '1717345707.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 16, 'name': 'o15', 'created_at': '1717345708.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 17, 'name': 'o16', 'created_at': '1717345709.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 18, 'name': 'o17', 'created_at': '1717345710.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 19, 'name': 'o18', 'created_at': '1717345711.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 20, 'name': 'o19', 'created_at': '1717345712.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a8098386-a400-496a-ab05-528a0061f71e-sda') test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_reserved test.unit.common.test_utils.TestFallocateReserve.test_all_reserved ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_disabled test.unit.common.test_utils.TestFallocateReserve.test_disabled ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space test.unit.common.test_utils.TestFallocateReserve.test_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works ... ok test.unit.common.test_utils.TestShardName.test test.unit.common.test_utils.TestShardName.test ... ok test.unit.common.test_utils.TestShardName.test_bad_create test.unit.common.test_utils.TestShardName.test_bad_create ... ok test.unit.common.test_utils.TestShardName.test_bad_parse test.unit.common.test_utils.TestShardName.test_bad_parse ... ok test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names ... ok test.unit.common.test_utils.TestShardName.test_root_has_hyphens test.unit.common.test_utils.TestShardName.test_root_has_hyphens ... ok test.unit.obj.test_diskfile.TestDiskFile.test_close_error test.unit.obj.test_diskfile.TestDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit test.unit.obj.test_diskfile.TestDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_delete test.unit.obj.test_diskfile.TestDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init test.unit.obj.test_diskfile.TestECDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx4c26555dc13b4adc9cd38-00665c9d96) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Setting object count to 1065 (txn: tx775b4c6b9f504dc083956-00665c9da2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Setting object count to 1065 (txn: txc562b7503a2e4df3b160b-00665c9da5) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txcf31ca1ffea844f5b0b15-00665c9da9) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: txcf31ca1ffea844f5b0b15-00665c9da9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: txfce25317d5784b7bae510-00665c9dac) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file ... ok test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 4.744529724121094e-05: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0, expired ttl=1.23 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 493, in _incr_or_decr return int(line[0].strip()) ^^^^^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: b'UNEXPECTED' test WARNING: Item size larger than warning threshold: 80 (80) >= 80 (80) test WARNING: Item size larger than warning threshold: 81 (81) >= 80 (80) test WARNING: Item size larger than warning threshold: 99 (99) >= 80 (80) test WARNING: Item size larger than warning threshold: 100 (100) >= 80 (80) test WARNING: Item size larger than warning threshold: 0 (0) >= 0 (0) test WARNING: Item size larger than warning threshold: 1 (1) >= 0 (0) test WARNING: Item size larger than warning threshold: 50 (50) >= 0 (0) test WARNING: Item size larger than warning threshold: 99 (99) >= 0 (0) test WARNING: Item size larger than warning threshold: 100 (100) >= 0 (0) test WARNING: Item size larger than warning threshold: 2048576 (2Mi) >= 1000000 (977Ki) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix multi_key, method get_multi, time_spent 6.341934204101562e-05, incomplete read test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0, failed to connect test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 460, in get sock.sendall(b'get ' + cmd.hash_key + b'\r\n') File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1282, in handle_get raise Exception('get failed') Exception: get failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method get, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix incr_key, method incr, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 531, in incr new_val = self._add(fp, sock, cmd, add_val, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 496, in _add sock.sendall(b' '.join([ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1224, in handle_add raise Exception('add failed') Exception: add failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix nvratelimit/v2/wf, method incr, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix set_key, method set, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 427, in set sock.sendall(set_msg(cmd.hash_key, flags, timeout, value)) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1252, in handle_set raise Exception('set failed') Exception: set failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test NOTICE: Closing unnecessary sock for port 6006 (child pid 88) test NOTICE: Closing unnecessary sock for port 6006 (child pid 89) test NOTICE: Closing unnecessary sock for port 6006 (child pid 90) test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test NOTICE: Started child 0 (PID 94) for port 6009 test NOTICE: Started child 1 (PID 95) for port 6009 test NOTICE: Started child 2 (PID 96) for port 6009 test NOTICE: Started child 1 (PID 97) for port 6009 test NOTICE: Child 2 (PID 3543380, port 6007) exiting normally test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test ERROR: Problem finding shard upper in '/tmp/tmpr66n0fvy/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpr66n0fvy/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpg_icdzrf/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpg_icdzrf/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging ... ok test.unit.container.test_updater.TestContainerUpdater.test_conf_params test.unit.container.test_updater.TestContainerUpdater.test_conf_params ... ok test.unit.container.test_updater.TestContainerUpdater.test_creation test.unit.container.test_updater.TestContainerUpdater.test_creation ... ok test.unit.container.test_updater.TestContainerUpdater.test_error_in_process test.unit.container.test_updater.TestContainerUpdater.test_error_in_process ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container ... ok test ERROR: Failed to unlink '/tmp/tmpz_wbikw3/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpz_wbikw3/containers/part/suffix/hash/container_1717345709.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpz_wbikw3/containers/part/suffix/hash/container_1717345709.00000.db': ['/tmp/tmpz_wbikw3/containers/part/suffix/hash/container.db', '/tmp/tmpz_wbikw3/containers/part/suffix/hash/container_1717345710.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpz_wbikw3/containers/part/suffix/hash/container_1717345710.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp_2gerrzs/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp_2gerrzs/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp_2gerrzs/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp_2gerrzs/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0213 "-" 3543418 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0179 "-" 3543418 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0185 "-" 3543418 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0138 "-" 3543418 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa1c5f9003ef74d1a89008-00665c9dae" "proxy-server 3543418" 0.0026 "-" 3543418 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:54519/sda1/1/a/c" "txa1c5f9003ef74d1a89008-00665c9dae" "container-server 3543418" 0.0018 "-" 3543418 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:35673/sdb1/1/a/c" "txa1c5f9003ef74d1a89008-00665c9dae" "container-server 3543418" 0.0016 "-" 3543418 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa1c5f9003ef74d1a89008-00665c9dae" "proxy-server 3543418" 0.0228 "-" 3543418 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa1c5f9003ef74d1a89008-00665c9dae" "proxy-server 3543418" 0.0394 "-" 3543418 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a/c HTTP/1.0 201 - - t - - - txa1c5f9003ef74d1a89008-00665c9dae - 0.0536 - - 1717345710.381566525 1717345710.435170889 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx3131bbf8f6c94186a2f28-00665c9dae" "proxy-server 3543418" 0.0019 "-" 3543418 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35673/sdb1/1/a1/c1" "tx3131bbf8f6c94186a2f28-00665c9dae" "container-server 3543418" 0.0020 "-" 3543418 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:54519/sda1/1/a1/c1" "tx3131bbf8f6c94186a2f28-00665c9dae" "container-server 3543418" 0.0026 "-" 3543418 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx3131bbf8f6c94186a2f28-00665c9dae" "proxy-server 3543418" 0.0469 "-" 3543418 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx3131bbf8f6c94186a2f28-00665c9dae" "proxy-server 3543418" 0.0340 "-" 3543418 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx3131bbf8f6c94186a2f28-00665c9dae - 0.0607 - - 1717345710.437441826 1717345710.498146772 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx2f04b82899524f4aa931d-00665c9dae" "proxy-server 3543418" 0.0092 "-" 3543418 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:54519/sda1/2/a/c1" "tx2f04b82899524f4aa931d-00665c9dae" "container-server 3543418" 0.0018 "-" 3543418 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:35673/sdb1/2/a/c1" "tx2f04b82899524f4aa931d-00665c9dae" "container-server 3543418" 0.0080 "-" 3543418 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx2f04b82899524f4aa931d-00665c9dae" "proxy-server 3543418" 0.0418 "-" 3543418 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx2f04b82899524f4aa931d-00665c9dae" "proxy-server 3543418" 0.0287 "-" 3543418 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx2f04b82899524f4aa931d-00665c9dae - 0.0635 - - 1717345710.500572443 1717345710.564033985 1 test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit "once" mode test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpztiig5aq/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpztiig5aq/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit "once" mode completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716678000 test DEBUG: looking for objects in 1716678000 test DEBUG: checking container 1716674400 test DEBUG: looking for objects in 1716674400 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'something', 'last_modified': '1970-01-01T00:00:00.000000', 'hash': '0.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test ERROR: Error listing containers in account .misplaced_objects (Unexpected response: 503 Service Unavailable) test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: ppi_skip: Container policy (0) in the middle of a part power increase (PPI) test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'ppi_skip': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'bogus', 'last_modified': '1970-01-01T01:00:18.841870', 'hash': '3618.841870', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': '00000bogus', 'last_modified': '1970-01-01T01:00:00.000000', 'hash': '3600.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'invalid_record': 1, 'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse ... ok test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process ... ok test.unit.container.test_reconciler.TestReconciler.test_no_concurrency test.unit.container.test_reconciler.TestReconciler.test_no_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_object_delete test.unit.container.test_reconciler.TestReconciler.test_object_delete ... ok test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move test.unit.container.test_reconciler.TestReconciler.test_object_move ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved ... ok test ERROR: Problem finding shard upper in '/tmp/tmpffqfolue/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpffqfolue/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpj93fi1j7/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpj93fi1j7/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '446'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '450'} test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta ... ok test ERROR: Failed to unlink '/tmp/tmpp9_xefve/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpp9_xefve/containers/part/suffix/hash/container_1717345709.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpp9_xefve/containers/part/suffix/hash/container_1717345709.00000.db': ['/tmp/tmpp9_xefve/containers/part/suffix/hash/container.db', '/tmp/tmpp9_xefve/containers/part/suffix/hash/container_1717345710.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpp9_xefve/containers/part/suffix/hash/container_1717345710.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txdc610b1b5ff24ca1b6067-00665c9daf) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sdx (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sdz (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sdy (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sdq (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sdo (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sdb (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sdv (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx3d5f0e3b68134a40b366e-00665c9db0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_sync test.unit.obj.test_ssync.TestSsyncReplication.test_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state ... ok test INFO: - - - [02/Jun/2024:16:27:58 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0034 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0191 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0088 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0117 "-" 3543384 - test INFO: - - - [02/Jun/2024:16:27:58 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0088 "-" 3543384 - test ERROR: Error decrypting container listing: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: get_keys(): unknown key id: unknown_key test ERROR: Error decrypting container listing: unknown_key test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txc4601e3896424e6d8dee0-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx386a592f134c4d8d8e667-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx8df51cb04ba24af8821ff-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx57bd093971cb447687c51-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx35a58ec98f27423ba40fa-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txedb68d2331fc45f58b56c-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx4931a34442ea449fb579a-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx22a107236a6549e9a47a5-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx3f74123d50464e4d825f7-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx18934ddc8cc74b01ab4ea-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx75d89d9204814d61a03e8-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa9642cea6b31479cb84a1-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txfefa7e273fdb4261b49bb-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txeaa3415344a647c3975f1-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txe7258654755b4228a6ead-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx43000b242abf4a24a3495-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx0324fe6bd91f4eae95efb-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx48f0edd5af9947a2bef15-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx479e7099c3f24f6ab197e-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx2fb72e00892e45ee9ac51-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txafa21571a262468381283-00665c9db3) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx19cd64a7f0894c2fb2f2a-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx5fcedb5b81d1414f9dfe0-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx07e91e8d720f4fb4ad22a-00665c9db3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: tx9697a5018d724b4bbe41e-00665c9db3) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: tx340c4290c76443de8a1ba-00665c9db3) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: tx48212c76266146be9802d-00665c9db3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test INFO: Object update sweep completed on /tmp/tmpfynfn7st/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfynfn7st/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmpfynfn7st/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345696.00000 test INFO: Object update sweep completed on /tmp/tmpfynfn7st/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7lrbpbr7/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /.shards_a/shard_c/o /tmp/tmp7lrbpbr7/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.00000 test INFO: Object update sweep completed on /tmp/tmp7lrbpbr7/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7lrbpbr7/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmp7lrbpbr7/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345699.00000 test INFO: Object update sweep completed on /tmp/tmp7lrbpbr7/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7lrbpbr7/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmp7lrbpbr7/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345700.00000 test INFO: Object update sweep completed on /tmp/tmp7lrbpbr7/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7lrbpbr7/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmp7lrbpbr7/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345701.00000 test INFO: Object update sweep completed on /tmp/tmp7lrbpbr7/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7lrbpbr7/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmp7lrbpbr7/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345702.00000 test INFO: Object update sweep completed on /tmp/tmp7lrbpbr7/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7lrbpbr7/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmp7lrbpbr7/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345703.00000 test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine ... ok test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txeac80ee3c2bb47c0abf40-00665c9dae" "proxy-server 3543418" 0.0089 "-" 3543418 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:54519/sda1/1/a/c2" "txeac80ee3c2bb47c0abf40-00665c9dae" "container-server 3543418" 0.0113 "-" 3543418 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:35673/sdb1/1/a/c2" "txeac80ee3c2bb47c0abf40-00665c9dae" "container-server 3543418" 0.0019 "-" 3543418 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txeac80ee3c2bb47c0abf40-00665c9dae" "proxy-server 3543418" 0.0364 "-" 3543418 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txeac80ee3c2bb47c0abf40-00665c9dae" "proxy-server 3543418" 0.0529 "-" 3543418 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txeac80ee3c2bb47c0abf40-00665c9dae - 0.0732 - - 1717345710.566439629 1717345710.639643669 2 test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': 'o20', 'created_at': '1717345713.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 22, 'name': 'o21', 'created_at': '1717345714.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 23, 'name': 'o22', 'created_at': '1717345715.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 24, 'name': 'o23', 'created_at': '1717345716.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 25, 'name': 'o24', 'created_at': '1717345717.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 26, 'name': 'o25', 'created_at': '1717345718.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 27, 'name': 'o26', 'created_at': '1717345719.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 28, 'name': 'o27', 'created_at': '1717345720.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 29, 'name': 'o28', 'created_at': '1717345721.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 30, 'name': 'o29', 'created_at': '1717345722.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a8098386-a400-496a-ab05-528a0061f71e-sda') test DEBUG: /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 30 (2.4818s) test DEBUG: Synchronization for /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db has fallen more than 30 rows behind; moving on and will try again next pass. REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '745665e398dba74a93be99dbd53e3ee0', 'a8098386-a400-496a-ab05-528a0061f71e-sda', '1717345692.87714', '1717345692.00000', '0', '', '1717345692.00000', 51, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a8098386-a400-496a-ab05-528a0061f71e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.58009 seconds (0.38758/s) test INFO: Removed 0 dbs test INFO: 1 successes, 1 failures test INFO: diff:1 diff_capped:1 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '745665e398dba74a93be99dbd53e3ee0', 'a8098386-a400-496a-ab05-528a0061f71e-sda', '1717345692.87714', '1717345692.00000', '0', '', '1717345692.00000', 51, 0) test DEBUG: /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row 30 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 31, 'name': 'o30', 'created_at': '1717345723.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 32, 'name': 'o31', 'created_at': '1717345724.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 33, 'name': 'o32', 'created_at': '1717345725.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 34, 'name': 'o33', 'created_at': '1717345726.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 35, 'name': 'o34', 'created_at': '1717345727.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 36, 'name': 'o35', 'created_at': '1717345728.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 37, 'name': 'o36', 'created_at': '1717345729.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 38, 'name': 'o37', 'created_at': '1717345730.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 39, 'name': 'o38', 'created_at': '1717345731.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 40, 'name': 'o39', 'created_at': '1717345732.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a8098386-a400-496a-ab05-528a0061f71e-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 41, 'name': 'o40', 'created_at': '1717345733.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 42, 'name': 'o41', 'created_at': '1717345734.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 43, 'name': 'o42', 'created_at': '1717345735.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 44, 'name': 'o43', 'created_at': '1717345736.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 45, 'name': 'o44', 'created_at': '1717345737.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 46, 'name': 'o45', 'created_at': '1717345738.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 47, 'name': 'o46', 'created_at': '1717345739.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 48, 'name': 'o47', 'created_at': '1717345740.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 49, 'name': 'o48', 'created_at': '1717345741.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 50, 'name': 'o49', 'created_at': '1717345742.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a8098386-a400-496a-ab05-528a0061f71e-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 51, 'name': 'o101', 'created_at': '1717345843.00000', 'size': 0, 'content_type': 'content-type-new', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a8098386-a400-496a-ab05-528a0061f71e-sda') test DEBUG: /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 51 (0.0280423s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '745665e398dba74a93be99dbd53e3ee0', 'a8098386-a400-496a-ab05-528a0061f71e-sda', '1717345692.87714', '1717345692.00000', '0', '', '1717345692.00000', 51, 0) test DEBUG: /tmp/tmpqqsctre2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04968 seconds (20.12743/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:0 ts_repl:0 test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part ... ok test.unit.container.test_replicator.TestReplicatorSync.test_local_ids test.unit.container.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:write OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other OWNER test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users ... ok test INFO: - - - [02/Jun/2024:16:27:58 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0188 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0226 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0069 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0105 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0133 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0140 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0083 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0018 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.0165 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:27:59 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0130 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0438 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0121 "-" 3543414 - test DEBUG: - - - [02/Jun/2024:16:28:00 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0007 "-" 3543414 - test DEBUG: - - - [02/Jun/2024:16:28:00 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0006 "-" 3543414 - test DEBUG: - - - [02/Jun/2024:16:28:00 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0006 "-" 3543414 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:28:00 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0055 "-" 3543414 - test DEBUG: - - - [02/Jun/2024:16:28:00 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0006 "-" 3543414 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0134 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0151 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0002 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0141 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0145 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0137 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0140 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0018 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0103 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0030 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0030 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0029 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0121 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0099 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0095 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0032 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0029 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0033 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0030 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0040 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0158 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3543414 1 test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0097 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0152 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:00 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission ... ok test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket ... ok test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1717340400 test DEBUG: looking for objects in 1717340400 test DEBUG: checking container 1717336800 test DEBUG: looking for objects in 1717336800 test DEBUG: checking container 1717333200 test DEBUG: looking for objects in 1717333200 test DEBUG: checking container 1717329600 test DEBUG: looking for objects in 1717329600 test DEBUG: checking container 1717326000 test DEBUG: looking for objects in 1717326000 test DEBUG: checking container 1717322400 test DEBUG: looking for objects in 1717322400 test DEBUG: checking container 1717318800 test DEBUG: looking for objects in 1717318800 test DEBUG: checking container 1717315200 test DEBUG: looking for objects in 1717315200 test DEBUG: checking container 1717311600 test DEBUG: looking for objects in 1717311600 test DEBUG: checking container 1716739200 test DEBUG: looking for objects in 1716739200 test DEBUG: checking container 1716735600 test DEBUG: looking for objects in 1716735600 test DEBUG: checking container 1716732000 test DEBUG: looking for objects in 1716732000 test DEBUG: checking container 1716728400 test DEBUG: looking for objects in 1716728400 test DEBUG: checking container 1716724800 test DEBUG: looking for objects in 1716724800 test DEBUG: checking container 1716721200 test DEBUG: looking for objects in 1716721200 test DEBUG: checking container 1716717600 test DEBUG: looking for objects in 1716717600 test DEBUG: checking container 1716714000 test DEBUG: looking for objects in 1716714000 test DEBUG: checking container 1716710400 test DEBUG: looking for objects in 1716710400 test DEBUG: checking container 1716706800 test DEBUG: looking for objects in 1716706800 test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345679.976560) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345679.976560) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345679.976560) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345679.976560) from storage policy 1 test DEBUG: delete_attempt: '/AUTH_bob/c/o1' (1717345679.976560) in policy_index 1 will be deleted from policy_index 0 (1717345679.97656_0000000000000002) test INFO: delete_success: '/AUTH_bob/c/o1' (1717345679.976560) deleted from policy_index 0 (1717345679.97656_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (1717345679.976560) from policy_index 1 (1717345679.97656_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (1717345679.976560) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (1717345679.976560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1717345679.976560) from the queue (1717345679.97656_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'delete_attempt': 1, 'delete_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3679.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent ... ok test ERROR: Failed to unlink '/tmp/tmpzzyhpcpd/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpzzyhpcpd/containers/part/suffix/hash/container_1717345706.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpzzyhpcpd/containers/part/suffix/hash/container_1717345706.00000.db': ['/tmp/tmpzzyhpcpd/containers/part/suffix/hash/container.db', '/tmp/tmpzzyhpcpd/containers/part/suffix/hash/container_1717345707.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpzzyhpcpd/containers/part/suffix/hash/container_1717345707.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test INFO: Begin container update single threaded sweep test ERROR: Error processing container /tmp/tmpegdyag34/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db: Boom!: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/updater.py", line 225, in container_sweep self.process_container(dbfile) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Boom! test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: ERROR: Failed to get paths to drive partitions: permission_denied test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpfn_avlh8/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpfn_avlh8/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpfn_avlh8/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test.unit.container.test_updater.TestContainerUpdater.test_run_once test.unit.container.test_updater.TestContainerUpdater.test_run_once ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument ... ok test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe ... ok test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '456'} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '448'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx751ac34ba83a49ddb698d-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx2a2fd61a364243fd9d9d9-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: txf4b41cf8ecf34fe18799f-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx86d94307e3d5414cbbb70-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx531cac88d60041a0b8cc4-00665c9db3) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: txe3a20028be60431492fac-00665c9db3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx9a2271e4aa6b49f7b22ce-00665c9db4) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txab515ef461654981b1753-00665c9db4) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txec9f2516833f4c2e94c1a-00665c9db4) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd3f8429dc9da4a099882f-00665c9db4) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx6af8a89959cb483aa4288-00665c9db4) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx841e1bab7c41458bb2cf9-00665c9db4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: tx4e23d914d8ee456f85cad-00665c9db4) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx4e23d914d8ee456f85cad-00665c9db4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp0a6oxc5z/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp0a6oxc5z/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp0a6oxc5z/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp0a6oxc5z/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0136 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0107 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0113 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0142 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx448d6c1762e14854a1635-00665c9db4" "proxy-server 3543384" 0.0020 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:59931/sda1/1/a/c" "tx448d6c1762e14854a1635-00665c9db4" "container-server 3543384" 0.0018 "-" 3543384 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:38511/sdb1/1/a/c" "tx448d6c1762e14854a1635-00665c9db4" "container-server 3543384" 0.0017 "-" 3543384 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx448d6c1762e14854a1635-00665c9db4" "proxy-server 3543384" 0.0247 "-" 3543384 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx448d6c1762e14854a1635-00665c9db4" "proxy-server 3543384" 0.0413 "-" 3543384 0 test.unit.proxy.test_mem_server.TestAccountController.test_DELETE test.unit.proxy.test_mem_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET test.unit.proxy.test_mem_server.TestAccountController.test_GET ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD test.unit.proxy.test_mem_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT test.unit.proxy.test_mem_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sdp (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sdt (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sdu (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sds (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sdw (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sdr (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server ERROR: Object returning 503 for [] (txn: tx3d5f0e3b68134a40b366e-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) proxy-server ERROR: Object returning 503 for [] (txn: txd2bdb5b8a9d1452fa2dc4-00665c9db0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: txfce25317d5784b7bae510-00665c9dac) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txfce25317d5784b7bae510-00665c9dac) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: txfce25317d5784b7bae510-00665c9dac) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx66b0196614ed443ea0207-00665c9dac) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx66b0196614ed443ea0207-00665c9dac) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx66b0196614ed443ea0207-00665c9dac) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: tx66b0196614ed443ea0207-00665c9dac) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx4979ed9d0b60480781b86-00665c9dac) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4979ed9d0b60480781b86-00665c9dac) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4979ed9d0b60480781b86-00665c9dac) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx4979ed9d0b60480781b86-00665c9dac) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: Setting object count to 3120 (txn: tx1c88b8ffdf664fb5a6fef-00665c9daf) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: Setting object count to 3120 (txn: tx7605401294454583a8558-00665c9daf) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: Setting object count to 3120 (txn: tx85182766f661498fb8aff-00665c9daf) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: Setting object count to 3120 (txn: tx5c34fac3c06748cba4530-00665c9db0) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: Setting object count to 3120 (txn: txb641261b3df743b4bd16a-00665c9db0) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache ... ok test INFO: Object update sweep completed on /tmp/tmp7lrbpbr7/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpytz8ftzw/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmpytz8ftzw/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.00000 test INFO: Object update sweep completed on /tmp/tmpytz8ftzw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpytz8ftzw/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmpytz8ftzw/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345700.00000 test INFO: Object update sweep completed on /tmp/tmpytz8ftzw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp0cyy89v5/devices/sda1 (pid: 3543408) test ERROR: ERROR Pickle problem, quarantining /tmp/tmp0cyy89v5/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345697.00000: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/updater.py", line 431, in _load_update return pickle.load(open(update_path, 'rb')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _pickle.UnpicklingError: invalid load key, '\xff'. test INFO: Object update sweep completed on /tmp/tmp0cyy89v5/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpmlh2mcpl/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c1/o02 /tmp/tmpmlh2mcpl/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345697.34597 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmpmlh2mcpl/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717345697.34803 test INFO: Object update sweep completed on /tmp/tmpmlh2mcpl/devices/sda1 in 0.21s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.21s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpwj_aa439/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c1/o00 /tmp/tmpwj_aa439/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717345697.56270 test DEBUG: Update sent for /a/c1/o03 /tmp/tmpwj_aa439/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345697.56333 test DEBUG: Update sent for /a/c1/o01 /tmp/tmpwj_aa439/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345697.56296 test INFO: Object update sweep completed on /tmp/tmpwj_aa439/devices/sda1 in 0.10s seconds:, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.10s, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp6yeub_ax/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c1/o04 /tmp/tmp6yeub_ax/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345697.67026 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp6yeub_ax/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717345697.66951 test DEBUG: Update sent for /a/c1/o03 /tmp/tmp6yeub_ax/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345697.67006 test DEBUG: Update sent for /a/c1/o02 /tmp/tmp6yeub_ax/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345697.66989 test INFO: Object update sweep completed on /tmp/tmp6yeub_ax/devices/sda1 in 0.01s seconds:, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpkqipq9f8/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c1/o02 /tmp/tmpkqipq9f8/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345697.68115 test DEBUG: Update sent for /a/c1/o01 /tmp/tmpkqipq9f8/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345697.68096 test INFO: Object update sweep completed on /tmp/tmpkqipq9f8/devices/sda1 in 0.11s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains (pid: 3543408) test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals ... ok test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdb test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00064 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00041 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00043 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmplux5alcq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '5c949802-da12-4fa3-9b16-6a586de68494-sda', '1717345700.41509', '1717344000.00000', '0', '', '1717344000.00000', 0, 1) test DEBUG: /tmp/tmplux5alcq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5c949802-da12-4fa3-9b16-6a586de68494-sda') test DEBUG: /tmp/tmplux5alcq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0130298s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '5c949802-da12-4fa3-9b16-6a586de68494-sda', '1717345700.41509', '1717344000.00000', '0', '', '1717344000.00000', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5c949802-da12-4fa3-9b16-6a586de68494-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmplux5alcq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13231 seconds (7.55782/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmplux5alcq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '86570890-926e-4ac2-9621-9b3cd6727cb5-sda', '1717345700.56463', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('86570890-926e-4ac2-9621-9b3cd6727cb5-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '86570890-926e-4ac2-9621-9b3cd6727cb5-sda', '1717345700.56463', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('86570890-926e-4ac2-9621-9b3cd6727cb5-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpo_8l9jxq/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '38f4b6fc-fa63-49ce-8c09-9285d37c1e0e-sda', '1717345700.67066', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717345700.67948"]}', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmpo_8l9jxq/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '38f4b6fc-fa63-49ce-8c09-9285d37c1e0e-sda') test DEBUG: /tmp/tmpo_8l9jxq/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0130787s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '38f4b6fc-fa63-49ce-8c09-9285d37c1e0e-sda', '1717345700.67066', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717345700.67948"]}', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('38f4b6fc-fa63-49ce-8c09-9285d37c1e0e-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpo_8l9jxq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12902 seconds (7.75088/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpo_8l9jxq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'bc9ccfbb-b981-4fb5-a74b-a92c324840fa-sda', '1717345700.84252', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('bc9ccfbb-b981-4fb5-a74b-a92c324840fa-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'bc9ccfbb-b981-4fb5-a74b-a92c324840fa-sda', '1717345700.84252', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('bc9ccfbb-b981-4fb5-a74b-a92c324840fa-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_5f9bfds/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard ... ok test INFO: - - - [02/Jun/2024:16:28:00 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0003 "-" 3543414 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0149 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0016 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0094 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0089 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0027 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0148 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0029 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0121 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0099 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0111 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0025 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0134 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/c_zero" 201 - "-" "-" "-" 0.0018 "-" 3543414 0 test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0083 "-" 3543414 1 test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0101 "-" 3543414 2 test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0103 "-" 3543414 3 test INFO: - - - [02/Jun/2024:16:28:02 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0095 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0112 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0027 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0118 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0109 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0149 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0151 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0024 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0019 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0004 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0001 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0002 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0002 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0002 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0153 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0051 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0105 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3543414 3 test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0096 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0132 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0085 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0143 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0021 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0094 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0107 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0107 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0162 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0032 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0112 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0022 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0093 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0086 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:02 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0090 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0106 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0027 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0122 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0013 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0082 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0120 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0144 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0028 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0026 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0024 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3543414 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata ... ok test.unit.cli.test_info.TestPrintObj.test_missing_etag test.unit.cli.test_info.TestPrintObj.test_missing_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObj.test_print_obj_valid test.unit.cli.test_info.TestPrintObj.test_print_obj_valid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission ... ok test ERROR: Failed to unlink '/tmp/tmptz2w7lxn/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmptz2w7lxn/containers/part/suffix/hash/container_1717345719.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmptz2w7lxn/containers/part/suffix/hash/container_1717345719.00000.db': ['/tmp/tmptz2w7lxn/containers/part/suffix/hash/container.db', '/tmp/tmptz2w7lxn/containers/part/suffix/hash/container_1717345720.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmptz2w7lxn/containers/part/suffix/hash/container_1717345720.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: Setting object count to 1032 (txn: tx1301d9ac41fe4b2e9ee84-00665c9dbc) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: Setting object count to 48 (txn: txaee92a2af5df4a6e82765-00665c9dbc) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: Setting object count to 1064 (txn: txc027dfeedeb34444ac9bf-00665c9dbc) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: txdd4f73c51f674d9497b10-00665c9dbc) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range test.unit.common.middleware.test_helpers.TestFakeSwift.test_range ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: public None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: authenticated None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users ... ok proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/36 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx448d6c1762e14854a1635-00665c9db4 - 0.0548 - - 1717345716.264395237 1717345716.319171667 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txe4883fa720774d199b54e-00665c9db4" "proxy-server 3543384" 0.0023 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:38511/sdb1/1/a1/c1" "txe4883fa720774d199b54e-00665c9db4" "container-server 3543384" 0.0020 "-" 3543384 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:59931/sda1/1/a1/c1" "txe4883fa720774d199b54e-00665c9db4" "container-server 3543384" 0.0016 "-" 3543384 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe4883fa720774d199b54e-00665c9db4" "proxy-server 3543384" 0.0350 "-" 3543384 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe4883fa720774d199b54e-00665c9db4" "proxy-server 3543384" 0.0230 "-" 3543384 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/36 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txe4883fa720774d199b54e-00665c9db4 - 0.0504 - - 1717345716.321865320 1717345716.372252464 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb3db9cee964441ddac77b-00665c9db4" "proxy-server 3543384" 0.0083 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:38511/sdb1/2/a/c1" "txb3db9cee964441ddac77b-00665c9db4" "container-server 3543384" 0.0021 "-" 3543384 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59931/sda1/2/a/c1" "txb3db9cee964441ddac77b-00665c9db4" "container-server 3543384" 0.0081 "-" 3543384 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb3db9cee964441ddac77b-00665c9db4" "proxy-server 3543384" 0.0515 "-" 3543384 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb3db9cee964441ddac77b-00665c9db4" "proxy-server 3543384" 0.0374 "-" 3543384 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/36 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txb3db9cee964441ddac77b-00665c9db4 - 0.0731 - - 1717345716.375062943 1717345716.448145151 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx63d8469744134675afb4e-00665c9db4" "proxy-server 3543384" 0.0089 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:38511/sdb1/1/a/c2" "tx63d8469744134675afb4e-00665c9db4" "container-server 3543384" 0.0082 "-" 3543384 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:59931/sda1/1/a/c2" "tx63d8469744134675afb4e-00665c9db4" "container-server 3543384" 0.0018 "-" 3543384 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx63d8469744134675afb4e-00665c9db4" "proxy-server 3543384" 0.0541 "-" 3543384 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:36 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx63d8469744134675afb4e-00665c9db4" "proxy-server 3543384" 0.0407 "-" 3543384 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/36 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx63d8469744134675afb4e-00665c9db4 - 0.0765 - - 1717345716.450583220 1717345716.527066708 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpjjy_wdla/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpjjy_wdla/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpjjy_wdla/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpjjy_wdla/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0122 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0099 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0110 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0114 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd72cdc8b710647699190d-00665c9db5" "proxy-server 3543384" 0.0018 "-" 3543384 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:55723/sda1/1/a/c" "txd72cdc8b710647699190d-00665c9db5" "container-server 3543384" 0.0015 "-" 3543384 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:56495/sdb1/1/a/c" "txd72cdc8b710647699190d-00665c9db5" "container-server 3543384" 0.0013 "-" 3543384 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txd72cdc8b710647699190d-00665c9db5" "proxy-server 3543384" 0.0332 "-" 3543384 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txd72cdc8b710647699190d-00665c9db5" "proxy-server 3543384" 0.0218 "-" 3543384 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/37 PUT /v1/a/c HTTP/1.0 201 - - t - - - txd72cdc8b710647699190d-00665c9db5 - 0.0463 - - 1717345717.251135588 1717345717.297474146 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txf67e60adf83e45cd8f797-00665c9db5" "proxy-server 3543384" 0.0019 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:56495/sdb1/1/a1/c1" "txf67e60adf83e45cd8f797-00665c9db5" "container-server 3543384" 0.0015 "-" 3543384 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:55723/sda1/1/a1/c1" "txf67e60adf83e45cd8f797-00665c9db5" "container-server 3543384" 0.0015 "-" 3543384 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txf67e60adf83e45cd8f797-00665c9db5" "proxy-server 3543384" 0.0208 "-" 3543384 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txf67e60adf83e45cd8f797-00665c9db5" "proxy-server 3543384" 0.0338 "-" 3543384 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/37 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txf67e60adf83e45cd8f797-00665c9db5 - 0.0444 - - 1717345717.299437761 1717345717.343867540 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx40887f4ef404411987167-00665c9db5" "proxy-server 3543384" 0.0066 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:55723/sda1/2/a/c1" "tx40887f4ef404411987167-00665c9db5" "container-server 3543384" 0.0015 "-" 3543384 1 setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: found_object: '/AUTH_bob/c/o1' (3680.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383930) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383930) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (36123.383930) moved from policy_index 1 to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 (0000036123.38393_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (36123.383930) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/36000/1:/AUTH_bob/c/o1' (36123.383930) from the queue (0000036123.38393_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.123460) in policy_index 1 will be moved to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.123460) moved from policy_index 1 to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_failed: '/AUTH_bob/c/o1' (3600.123456) was not cleaned up in storage_policy 1 (Unexpected response: 503 Service Unavailable) test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: unavailable_destination: '/AUTH_bob/c/o1' (3600.123456) unable to determine the destination timestamp, if any test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_destination': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716678000 test DEBUG: looking for objects in 1716678000 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1716680400.088561) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1716680400.088561) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1716680400.088561) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1716680400.088561) from storage policy 1 test CRITICAL: lost_source: '/AUTH_jeb/c/o1' (1716680400.08856) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_jeb/c/o1' (1716680400.088561) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/1716678000/1:/AUTH_jeb/c/o1' (1716680400.088561) from the queue (1716680400.08856_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1717345680.101296) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1717345680.101296) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1717345680.101296) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1717345680.101296) from storage policy 1 test WARNING: unavailable_source: '/AUTH_jeb/c/o1' (1717345680.10130) in policy_index 1 responded 404 (0000000000.00000) test INFO: retry: '/AUTH_jeb/c/o1' (1717345680.101296) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmprrex01zp/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 1.19s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmprrex01zp/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/c /tmp/tmprrex01zp/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test WARNING: /tmp/tmpu0iiuf4o/tmp_test_container_updater/devices/sda1 is not mounted test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Failed to get container info (Lock timeout: 10 seconds: /tmp/tmpm_b9c_no/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db); skipping. test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmp9bh1q750/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmp9bh1q750/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmp9bh1q750/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.28s, 1 successes, 0 failures, 0 with no changes test.unit.container.test_updater.TestContainerUpdater.test_shard_container test.unit.container.test_updater.TestContainerUpdater.test_shard_container ... ok test.unit.container.test_updater.TestContainerUpdater.test_unicode test.unit.container.test_updater.TestContainerUpdater.test_unicode ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add test.unit.obj.test_replicator.TestReplicatorStats.test_add ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon ... ok test.unit.obj.test_updater.TestSweepStats.test_copy test.unit.obj.test_updater.TestSweepStats.test_copy ... ok test.unit.obj.test_updater.TestSweepStats.test_reset test.unit.obj.test_updater.TestSweepStats.test_reset ... ok test.unit.obj.test_updater.TestSweepStats.test_since test.unit.obj.test_updater.TestSweepStats.test_since ... ok test.unit.obj.test_updater.TestSweepStats.test_str test.unit.obj.test_updater.TestSweepStats.test_str ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:56495/sdb1/2/a/c1" "tx40887f4ef404411987167-00665c9db5" "container-server 3543384" 0.0073 "-" 3543384 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx40887f4ef404411987167-00665c9db5" "proxy-server 3543384" 0.0238 "-" 3543384 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx40887f4ef404411987167-00665c9db5" "proxy-server 3543384" 0.0369 "-" 3543384 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/37 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx40887f4ef404411987167-00665c9db5 - 0.0527 - - 1717345717.345826387 1717345717.398563623 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txaae206eb6c204bf982523-00665c9db5" "proxy-server 3543384" 0.0067 "-" 3543384 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:55723/sda1/1/a/c2" "txaae206eb6c204bf982523-00665c9db5" "container-server 3543384" 0.0017 "-" 3543384 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:56495/sdb1/1/a/c2" "txaae206eb6c204bf982523-00665c9db5" "container-server 3543384" 0.0139 "-" 3543384 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txaae206eb6c204bf982523-00665c9db5" "proxy-server 3543384" 0.0415 "-" 3543384 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:37 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txaae206eb6c204bf982523-00665c9db5" "proxy-server 3543384" 0.0307 "-" 3543384 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/37 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txaae206eb6c204bf982523-00665c9db5 - 0.0584 - - 1717345717.400535583 1717345717.458940268 2 The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error /tmp/tmpwcv4pun9/tmpnm5oa2ec, build version 4, id bcb6c44308e148cf81fd4a1e5fb79770 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmpwcv4pun9/tmpnm5oa2ec.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 Reassigned 192 (300.00%) partitions. Balance is now 5.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance test.unit.cli.test_ringbuilder.TestCommands.test_rebalance ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read ... ok test INFO: Object update single-threaded sweep completed: 0.11s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp2liwwzjd/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c1/o03 /tmp/tmp2liwwzjd/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345697.79660 test DEBUG: Update sent for /a/c1/o05 /tmp/tmp2liwwzjd/devices/sda1/async_pending/dc4/017ff741f591e7220e0836a0c4fcedc4-1717345697.79695 test DEBUG: Update sent for /a/c1/o01 /tmp/tmp2liwwzjd/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717345697.79622 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp2liwwzjd/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717345697.79603 test DEBUG: Update sent for /a/c1/o08 /tmp/tmp2liwwzjd/devices/sda1/async_pending/5b8/4e594cf882aac2f7f8f2f4cdade285b8-1717345697.79749 test DEBUG: Update sent for /a/c1/o04 /tmp/tmp2liwwzjd/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345697.79677 test DEBUG: Update sent for /a/c1/o02 /tmp/tmp2liwwzjd/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345697.79642 test DEBUG: Update sent for /a/c1/o06 /tmp/tmp2liwwzjd/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717345697.79713 test DEBUG: Update sent for /a/c1/o09 /tmp/tmp2liwwzjd/devices/sda1/async_pending/bee/db77f283b4b1bb5e5b6ce501ca337bee-1717345697.79767 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmp2liwwzjd/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717345697.79814 test DEBUG: Update sent for /a/c1/o07 /tmp/tmp2liwwzjd/devices/sda1/async_pending/fc8/26ac197975ac1169d403eeb7ae8bbfc8-1717345697.79732 test INFO: Object update sweep completed on /tmp/tmp2liwwzjd/devices/sda1 in 0.21s seconds:, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.21s, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpt43wfbck/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c1/o03 /tmp/tmpt43wfbck/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717345698.01176 test DEBUG: Update sent for /a/c1/o04 /tmp/tmpt43wfbck/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717345698.01202 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpt43wfbck/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717345698.01154 test DEBUG: Update sent for /a/c1/o06 /tmp/tmpt43wfbck/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717345698.01239 test DEBUG: Update sent for /a/c1/o00 /tmp/tmpt43wfbck/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717345698.01113 test INFO: Object update sweep completed on /tmp/tmpt43wfbck/devices/sda1 in 0.01s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test DEBUG: Update failed for /a/c/o /tmp/tmpned4fmha/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.04606 test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test DEBUG: Error code 500 is returned from remote server 127.0.0.1:39995/sda1 test DEBUG: Error code 500 is returned from remote server 127.0.0.1:39995/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpned4fmha/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.04606 test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.02s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test DEBUG: Error code 404 is returned from remote server 127.0.0.1:39995/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpned4fmha/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.04606 test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.02s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test INFO: Timeout waiting on remote server 127.0.0.1:39995/sda1: 99 seconds test DEBUG: Update failed for /a/c/o /tmp/tmpned4fmha/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.04606 test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test.unit.obj.test_updater.TestObjectUpdater.test_run_once test.unit.obj.test_updater.TestObjectUpdater.test_run_once ... ok test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bool test.unit.obj.test_updater.TestRateLimiterBucket.test_bool ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_len test.unit.obj.test_updater.TestRateLimiterBucket.test_len ... ok test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req ... ok test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx5e5ba7fc4d714ceb9f8d7-00665c9db0) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txad3a0417a2cf4a998affc-00665c9db0) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx2bbab877afab49d8bef32-00665c9db0) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txe7c62ec7c7204654acad3-00665c9db0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe7c62ec7c7204654acad3-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe7c62ec7c7204654acad3-00665c9db0) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txe7c62ec7c7204654acad3-00665c9db0) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txace76d61a959441eb2faa-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txace76d61a959441eb2faa-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txace76d61a959441eb2faa-00665c9db0) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txace76d61a959441eb2faa-00665c9db0) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx8073f07a0d88459d8fe45-00665c9db0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx8073f07a0d88459d8fe45-00665c9db0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx8073f07a0d88459d8fe45-00665c9db0) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx8073f07a0d88459d8fe45-00665c9db0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: Setting object count to 1068 (txn: tx351f517d22994459a1eab-00665c9db3) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9efe796808c04c969ea9d-00665c9db3) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server ERROR: Object returning 503 for [] (txn: tx530ac14e26e647fcbd1eb-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range ... ok REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'bd3e08b0-a39c-4f5e-8f99-d214be4f8c42-sda', '1717345700.93260', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717345700.94044"]}', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmp_5f9bfds/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'bd3e08b0-a39c-4f5e-8f99-d214be4f8c42-sda') test DEBUG: /tmp/tmp_5f9bfds/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0137689s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'bd3e08b0-a39c-4f5e-8f99-d214be4f8c42-sda', '1717345700.93260', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717345700.94044"]}', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('bd3e08b0-a39c-4f5e-8f99-d214be4f8c42-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmp_5f9bfds/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 8.40534 seconds (0.11897/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp_5f9bfds/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', '9d4f308f-9783-4590-b02e-004167440752-sda', '1717345709.02435', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9d4f308f-9783-4590-b02e-004167440752-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', '9d4f308f-9783-4590-b02e-004167440752-sda', '1717345709.02435', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9d4f308f-9783-4590-b02e-004167440752-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmprr1wvisi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '5a17036a5fd12729647527c6421e0c1a', 'a895a43a-334d-40d7-8677-04b051e7416f-sda', '1717345709.46364', '1717345709.00000', '0', '', '1717345709.00000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-nulo', 'created_at': '1717345711.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o-unu', 'created_at': '1717345712.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'a895a43a-334d-40d7-8677-04b051e7416f-sda') test DEBUG: /tmp/tmprr1wvisi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0283308s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '5a17036a5fd12729647527c6421e0c1a', 'a895a43a-334d-40d7-8677-04b051e7416f-sda', '1717345709.46364', '1717345709.00000', '0', '', '1717345709.00000', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a895a43a-334d-40d7-8677-04b051e7416f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmprr1wvisi/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14715 seconds (6.79595/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmprr1wvisi/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7e08901763eb1d11569c376bbb14fda8', '0849b812-c619-48bd-a758-08479e53f512-sda', '1717345709.63177', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0849b812-c619-48bd-a758-08479e53f512-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7e08901763eb1d11569c376bbb14fda8', '0849b812-c619-48bd-a758-08479e53f512-sda', '1717345709.63177', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0849b812-c619-48bd-a758-08479e53f512-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmprr1wvisi/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd0f55abf9a3495d0d1f1eb3ac4003401', '96e14821-079e-4cbe-8513-92b9bd8beaa1-sdb', '1717345709.46364', '1717345710.00000', '0', '', '1717345710.00000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row -1 REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-nulo', 'created_at': '1717345713.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o-unu', 'created_at': '1717345714.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '96e14821-079e-4cbe-8513-92b9bd8beaa1-sdb') test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.0109975s) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': 'a895a43a-334d-40d7-8677-04b051e7416f-sda', 'sync_point': 2}],) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd0f55abf9a3495d0d1f1eb3ac4003401', '96e14821-079e-4cbe-8513-92b9bd8beaa1-sdb', '1717345709.46364', '1717345710.00000', '0', '', '1717345710.00000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, starting at row -1 test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize ... ok test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 200 - - - - 8 - - First:%201%0ASecond:%202 0.0001 - - 1717345683.722165346 1717345683.722260237 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 500 - - - - - - - - 0.0000 - - 1717345683.724677324 1717345683.724688530 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345683.726056099 1717345683.726134777 - test INFO: GET /v1/a/c 205 205 test INFO: GET /v1/a/c 209 205 test INFO: GET /v1/a/c 210 205 test INFO: 1.2.3.4 1.2.3.4 02/Jun/2024/16/28/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345683.736363411 1717345683.736438751 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717345683.737809181 1717345683.737842321 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717345683.739267588 1717345683.739287138 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345683.740624905 1717345683.740688562 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345683.742050648 1717345683.742106676 - test INFO: - - 02/Jun/2024/16/28/03 GET / HTTP/1.0 200 - - - - 15 - - - 0.0006 - - 1717345683.742018461 1717345683.742569685 - test INFO: - - 26/Apr/1970/17/46/41 GET / HTTP/1.0 200 - - - - 8 - - - 1.0000 - - 10000000.000000000 10000001.000000000 - test INFO: GET /v1/a/c 503 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c True 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c parsing%20ok 503 test INFO: GET /v1/a/c 500 500 test INFO: GET /v1/a/c 500 568 test INFO: GET /v1/a/c 499 503 test INFO: GET /v1/a/c 499 503 test INFO: GET / 500 200 test INFO: 2001:db8:85a3:8d3:1319:8a2e:370:7348 2001:db8:85a3:8d3:1319:8a2e:370:7348 02/Jun/2024/16/28/10 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345690.995229721 1717345690.995312691 - test INFO: - - 02/Jun/2024/16/28/10 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345690.998506546 1717345690.998562098 - test INFO: - - 02/Jun/2024/16/28/10 GET / HTTP/1.0 200 - - b05bf940-0464-4c... - 8 - - - 0.0001 - - 1717345690.999689102 1717345690.999744654 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345691.000710964 1717345691.000765324 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - b05bf940... - 8 - - - 0.0001 - - 1717345691.002073050 1717345691.002149820 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717345691.003462076 1717345691.003524303 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717345691.004750013 1717345691.004827499 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345691.006179333 1717345691.006247759 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - ... - 8 - - - 0.0001 - - 1717345691.007584095 1717345691.007657766 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717345691.009330750 1717345691.009397745 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717345691.010765553 1717345691.010869265 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one%2Cand%20two 1717345691.012537956 1717345691.012604952 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345691.014152288 1717345691.014224291 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345691.015557528 1717345691.015662909 - test INFO: - - 02/Jun/2024/16/28/11 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one 1717345691.017258406 1717345691.017330647 - test INFO: template which can be edited in config: HTTP/1.0 / GET {SMD5}c65475e457fea0951fbb9ec9596b2177 - 1.0000 26/Apr/1970/17/46/40 10000001.000000000 0.5 example.com test INFO: HTTP/1.0 /bucket/path/to/key GET AUTH_test bucket path/to/key test INFO: - - 02/Jun/2024/16/28/16 GET /%3Fx%3D3 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345696.549336910 1717345696.549420118 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body ... ok test INFO: - - - [02/Jun/2024:16:28:03 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:17 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0001 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:18 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0296 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:18 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0075 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:18 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0073 "-" 3543414 - test INFO: - - - [02/Jun/2024:16:28:18 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0063 "-" 3543414 - test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_encrypter.py", line 913, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:27:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/Ω /tmp/tmpx9l2vjg6/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.03s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: [worker 1/2 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 1/2 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/2 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 2/2 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/3 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 2/3 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 3/3 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 3/3 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3543380] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3543380] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=8804] Running object replicator in script mode. test-replicator DEBUG: [worker 1/3 pid=8804] debug message test-replicator INFO: [worker 1/3 pid=8804] info message test-replicator WARNING: [worker 1/3 pid=8804] warning message test-replicator ERROR: [worker 1/3 pid=8804] error message test-replicator INFO: [worker 1/3 pid=8804] Object replication complete (once). (0.00 minutes) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txef9a586c1f6e448481076-00665c9dad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: ValueError("upper ('a') must be greater than or equal to lower ('z')") (txn: txaf7b403463cc4d2db86be-00665c9dad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: KeyError('name') (txn: tx992bfd800acc48b794992-00665c9dad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type None (txn: tx5e12b29d2c7e4f049357d-00665c9dad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: ValueError('not a list') (txn: txbc7630e932dd4c5c83878-00665c9dad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Failed to get container listing from /v1/a/c/o: 404 (txn: txfb88309fa1194932b7a2a-00665c9dad) test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type ... ok test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test INFO: Timeout connecting to remote server 127.0.0.1:39995/sda1: 9 seconds test DEBUG: Update failed for /a/c/o /tmp/tmpned4fmha/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.04606 test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpned4fmha/devices/sda1 (pid: 3543408) test DEBUG: Update sent for /a/c/o /tmp/tmpned4fmha/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717345698.04606 test INFO: Object update sweep completed on /tmp/tmpned4fmha/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test WARNING: Skipping: test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Object update sweep starting on /tmp/tmpbqbyxuvr/devices/sda1 (pid: 3543408) test INFO: Object update sweep progress on /tmp/tmpbqbyxuvr/devices/sda1: 16.00s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update sweep progress on /tmp/tmpbqbyxuvr/devices/sda1: 32.00s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update sweep completed on /tmp/tmpbqbyxuvr/devices/sda1 in 44.00s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test INFO: Object update sweep starting on /tmp/tmpzdizot10/devices/sda1 (pid: 3543408) test INFO: Object update sweep completed on /tmp/tmpzdizot10/devices/sda1 in 0.13s seconds:, 6 successes, 0 failures, 0 quarantines, 6 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3543408) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Base DELETE returning 503 for (204, 204, 404) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5038761ca0ef4f98bd91b-00665c9da2) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5038761ca0ef4f98bd91b-00665c9da2) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5038761ca0ef4f98bd91b-00665c9da2) test ERROR: Container HEAD returning 503 for [] (txn: tx5038761ca0ef4f98bd91b-00665c9da2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app ... ok test ERROR: unhandled_error: unable to upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 635, in ensure_object_in_right_location self.swift.upload_object( File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 844, in upload_object self.handle_request('PUT', path, headers, acceptable_statuses, fobj, File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 264, in handle_request resp = self.make_request(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'unhandled_error': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test WARNING: copy_failed: upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 returned Unexpected response: 503 Service Unavailable test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1716739199.000000) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1716739199.000000) from storage policy 1 test CRITICAL: lost_source: '/AUTH_bob/c/o1' (1716739199.00000) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1716739199.000000) from the queue (1716739199.00000_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.234570) in policy_index 1 will be moved to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.234570) moved from policy_index 1 to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3600.123456) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3600.123456) from the queue (0000003600.12346_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345680.175802) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345680.175802) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345680.175802) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345680.175802) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717345680.17580) in policy_index 1 responded 200 (1717345679.17580) test INFO: retry: '/AUTH_bob/c/o1' (1717345680.175802) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 1 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 should be in policy_index 1 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 0 test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces ... ok test.unit.container.test_reconciler.TestReconciler.test_processes_config test.unit.container.test_reconciler.TestReconciler.test_processes_config ... ok test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone ... ok test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy ... ok swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. Reassigned 768 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) Reassigned 192 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (0:24:00 remaining) No partitions could be reassigned. There is no need to do so at this time Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 1000.0 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) ------------------------------------------------------------------------------- An error has occurred during ring validation. Common causes of failure are rings that are empty or do not have enough devices to accommodate the replica count. Original exception message: There are no devices in this ring, or all devices have been deleted ------------------------------------------------------------------------------- Partition power increase in progress. You need to finish the increase first before rebalancing. Reassigned 0 (0.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 20 (31.25%) partitions. Balance is now 24.48. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 24.48 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 33.33 ------------------------------------------------------------------------------- NOTE: Dispersion of 33.333333 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 16.67 ------------------------------------------------------------------------------- NOTE: Dispersion of 16.666667 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. test.unit.cli.test_ringbuilder.TestCommands.test_remove_device test.unit.cli.test_ringbuilder.TestCommands.test_remove_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching ... ok test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Setting object count to 1068 (txn: tx9efe796808c04c969ea9d-00665c9db3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: tx1e461d4cae5f45bbbb873-00665c9db6) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: tx9d7fb7b3ed794eddadacd-00665c9db9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: txbbc5b416b857445db9295-00665c9dbc) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdd4f73c51f674d9497b10-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdd4f73c51f674d9497b10-00665c9dbc) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txdd4f73c51f674d9497b10-00665c9dbc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdd4f73c51f674d9497b10-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdd4f73c51f674d9497b10-00665c9dbc) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txdd4f73c51f674d9497b10-00665c9dbc) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc5877421ed4c4dacbc4ff-00665c9dbc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Setting object count to 1065 (txn: tx4c30dd544c784f27abde4-00665c9dbe) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Setting object count to 1065 (txn: txbd86836cdf2e4516b4c3d-00665c9dc1) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx9ebd2c3da74141aeac002-00665c9dc4) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: tx9ebd2c3da74141aeac002-00665c9dc4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx1de56c1dfa864489950c9-00665c9dc6) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server ERROR: Object returning 503 for [] (txn: tx2280a29b24ee483c8e57e-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa4faf875e1a8479b96e3a-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from primary (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txecee36349d414004b463c-00665c9db0) proxy-server ERROR: Object returning 503 for [] (txn: txecee36349d414004b463c-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-nulo', 'created_at': '1717345713.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o-unu', 'created_at': '1717345714.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '96e14821-079e-4cbe-8513-92b9bd8beaa1-sdb') test DEBUG: Replicating db /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7e08901763eb1d11569c376bbb14fda8', '78d0843b-0a97-4824-8da9-e413bfb1c60b-', '1717345709.63177', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.2:1002/sdc, nothing to do REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '7e08901763eb1d11569c376bbb14fda8', '78d0843b-0a97-4824-8da9-e413bfb1c60b-', '1717345709.63177', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.0:1000/sda, nothing to do test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0316329s) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': 'a895a43a-334d-40d7-8677-04b051e7416f-sda', 'sync_point': 2}],) test DEBUG: Adding 1 objects to the reconciler at /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.16308 seconds (12.26363/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:2 diff_capped:0 empty:0 hashmatch:0 no_change:2 remote_merge:0 rsync:0 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '4c3f52f9118ca6d99a88ad36c012e2cf', '78d0843b-0a97-4824-8da9-e413bfb1c60b-', '1717345709.63177', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row 1 REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '1:/a/c/o-unu', 'created_at': '1717345714.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717345714.00000', 'deleted': 0, 'storage_policy_index': 0}], '78d0843b-0a97-4824-8da9-e413bfb1c60b-') test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.0135999s) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': '0849b812-c619-48bd-a758-08479e53f512-sda', 'sync_point': 1}],) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '4c3f52f9118ca6d99a88ad36c012e2cf', '78d0843b-0a97-4824-8da9-e413bfb1c60b-', '1717345709.63177', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, starting at row 1 REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '1:/a/c/o-unu', 'created_at': '1717345714.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717345714.00000', 'deleted': 0, 'storage_policy_index': 0}], '78d0843b-0a97-4824-8da9-e413bfb1c60b-') test DEBUG: /tmp/tmprr1wvisi/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0133369s) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': '0849b812-c619-48bd-a758-08479e53f512-sda', 'sync_point': 1}],) test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7c360650d95e5ccd4d5aae1ef3fde428', 'f6156abc-f870-41d1-8a5f-19a9853aaea8-', '1717345710.31754', '1717345710.31193', '0', '', '1717345710.31193', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f6156abc-f870-41d1-8a5f-19a9853aaea8-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.32861.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7c360650d95e5ccd4d5aae1ef3fde428', 'f6156abc-f870-41d1-8a5f-19a9853aaea8-', '1717345710.31754', '1717345710.31193', '0', '', '1717345710.31193', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345710.32843', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345710.32843', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345710.32843', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345710.32883', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345710.32883', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.32861', 'epoch': '1717345710.32861', 'reported': 0, 'tombstones': -1}], 'f6156abc-f870-41d1-8a5f-19a9853aaea8-') test DEBUG: /tmp/tmpsliu7jvk/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.32861.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpsliu7jvk/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.32861.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'aefe49294140f5edb63a2ae051915215', 'ae9f7d0b-b85b-4527-8559-af59bb688f3c-', '1717345710.45933', '1717345710.45432', '0', '', '1717345710.45432', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345710.54831', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345710.54831', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345710.54831', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345710.56124', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345710.56124', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.47294', 'epoch': '1717345710.47294', 'reported': 0, 'tombstones': -1}], 'ae9f7d0b-b85b-4527-8559-af59bb688f3c-') test DEBUG: /tmp/tmpdaqr4jd4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.47294.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpdaqr4jd4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.47294.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'aefe49294140f5edb63a2ae051915215', 'ae9f7d0b-b85b-4527-8559-af59bb688f3c-', '1717345710.45933', '1717345710.53726', '0', '', '1717345710.45432', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding ... ok test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 HEAD /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 POST /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 DELETE /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 OPTIONS /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/42 GET - HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET / HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /bad HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /baddy/mc_badderson HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1 HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1/ HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.377984047 1717345707.378066778 - test INFO: - - 02/Jun/2024/16/28/27 GET /foo HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.378796101 1717345707.378898859 - test INFO: - - 02/Jun/2024/16/28/27 GET /foo/bar HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.379523277 1717345707.379598141 - test INFO: - - 02/Jun/2024/16/28/27 GET /v1 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.380292892 1717345707.380367756 - test INFO: - - 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - 17 - - - 0.0001 SOS - 1717345707.386466980 1717345707.386610270 - test INFO: - - 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - - - - - 0.0001 - - 1717345707.388750076 1717345707.388830185 - test INFO: - - 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - 3 - - - 0.0001 - - 1717345707.390893459 1717345707.390992880 - test INFO: - - 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - 12 - - - 0.0001 - - 1717345707.393104553 1717345707.393178701 - test INFO: - - 02/Jun/2024/16/28/27 PUT /v1/a/c/o HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.407561302 1717345707.407628059 1 test INFO: - - 02/Jun/2024/16/28/27 GET /v1/a/c HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.409306049 1717345707.409450769 1 test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.411418915 1717345707.411537409 - test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/28/27 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717345707.412995577 1717345707.413071632 - test INFO: - - 02/Jun/2024/16/28/27 GET /v1/something HTTP/1.0 200 - - - - 8 - - - 0.0002 - - 1717345707.415079594 1717345707.415238142 - test INFO: - - 02/Jun/2024/16/28/27 GET / HTTP/1.0 499 - - - - 4 - - - 0.0001 - - 1717345707.418498993 1717345707.418594837 - test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas ... ok test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy ... ok test.unit.common.test_constraints.TestConstraints.test_check_account_format test.unit.common.test_constraints.TestConstraints.test_check_account_format ... ok test.unit.common.test_constraints.TestConstraints.test_check_container_format test.unit.common.test_constraints.TestConstraints.test_check_container_format ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount ... ok test.unit.common.test_constraints.TestConstraints.test_check_float test.unit.common.test_constraints.TestConstraints.test_check_float ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_count test.unit.common.test_constraints.TestConstraints.test_check_metadata_count ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_good test.unit.common.test_constraints.TestConstraints.test_check_metadata_good ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_size test.unit.common.test_constraints.TestConstraints.test_check_metadata_size ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8 test.unit.common.test_constraints.TestConstraints.test_check_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical ... ok test.unit.common.test_constraints.TestConstraints.test_valid_api_version test.unit.common.test_constraints.TestConstraints.test_valid_api_version ... ok test.unit.common.test_constraints.TestConstraints.test_valid_timestamp test.unit.common.test_constraints.TestConstraints.test_valid_timestamp ... ok test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta ... ok test.unit.common.test_constraints.TestConstraints.test_validate_constraints test.unit.common.test_constraints.TestConstraints.test_validate_constraints ... ok test.unit.common.test_db.TestChexor.test_chexor test.unit.common.test_db.TestChexor.test_chexor ... ok test.unit.common.test_db.TestChexor.test_invalid_old_hash test.unit.common.test_db.TestChexor.test_invalid_old_hash ... ok test.unit.common.test_db.TestChexor.test_no_name test.unit.common.test_db.TestChexor.test_no_name ... ok test.unit.common.test_db.TestChexor.test_normal_case test.unit.common.test_db.TestChexor.test_normal_case ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_creation test.unit.common.test_db_replicator.TestDBReplicator.test_creation ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type 'object' (txn: txef4e1257dce548969d4c0-00665c9dad) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp5pgyvvxe/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp5pgyvvxe/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp5pgyvvxe/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp5pgyvvxe/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:29 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0833 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.2354 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0160 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0129 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx09559f650ec644d982d3c-00665c9dae" "proxy-server 3543380" 0.0019 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:36241/sda1/3/a/c" "tx09559f650ec644d982d3c-00665c9dae" "container-server 3543380" 0.0020 "-" 3543380 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:38807/sdb1/3/a/c" "tx09559f650ec644d982d3c-00665c9dae" "container-server 3543380" 0.0016 "-" 3543380 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx09559f650ec644d982d3c-00665c9dae" "proxy-server 3543380" 0.0394 "-" 3543380 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx09559f650ec644d982d3c-00665c9dae" "proxy-server 3543380" 0.0260 "-" 3543380 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx09559f650ec644d982d3c-00665c9dae - 0.0525 - - 1717345710.047194004 1717345710.099660873 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx10bf75a1c2cd47459b2d0-00665c9dae" "proxy-server 3543380" 0.0021 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:36241/sda1/0/a1/c1" "tx10bf75a1c2cd47459b2d0-00665c9dae" "container-server 3543380" 0.0024 "-" 3543380 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:38807/sdb1/0/a1/c1" "tx10bf75a1c2cd47459b2d0-00665c9dae" "container-server 3543380" 0.0015 "-" 3543380 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx10bf75a1c2cd47459b2d0-00665c9dae" "proxy-server 3543380" 0.0260 "-" 3543380 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx10bf75a1c2cd47459b2d0-00665c9dae" "proxy-server 3543380" 0.0415 "-" 3543380 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx10bf75a1c2cd47459b2d0-00665c9dae - 0.0537 - - 1717345710.101792336 1717345710.155458927 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1f3adc5a6bf647a9a928f-00665c9dae" "proxy-server 3543380" 0.0090 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:38807/sdb1/3/a/c1" "tx1f3adc5a6bf647a9a928f-00665c9dae" "container-server 3543380" 0.0106 "-" 3543380 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:36241/sda1/3/a/c1" "tx1f3adc5a6bf647a9a928f-00665c9dae" "container-server 3543380" 0.0021 "-" 3543380 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1f3adc5a6bf647a9a928f-00665c9dae" "proxy-server 3543380" 0.0356 "-" 3543380 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1f3adc5a6bf647a9a928f-00665c9dae" "proxy-server 3543380" 0.0537 "-" 3543380 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx1f3adc5a6bf647a9a928f-00665c9dae - 0.0732 - - 1717345710.157457829 1717345710.230670929 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx43d7b3d476b446a2963ba-00665c9dae" "proxy-server 3543380" 0.0101 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:36241/sda1/0/a/c2" "tx43d7b3d476b446a2963ba-00665c9dae" "container-server 3543380" 0.0101 "-" 3543380 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:38807/sdb1/0/a/c2" "tx43d7b3d476b446a2963ba-00665c9dae" "container-server 3543380" 0.0017 "-" 3543380 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx43d7b3d476b446a2963ba-00665c9dae" "proxy-server 3543380" 0.0351 "-" 3543380 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:30 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx43d7b3d476b446a2963ba-00665c9dae" "proxy-server 3543380" 0.0536 "-" 3543380 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/30 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx43d7b3d476b446a2963ba-00665c9dae - 0.0770 - - 1717345710.233413935 1717345710.310441017 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpmzr0c8ac/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpmzr0c8ac/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpmzr0c8ac/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpmzr0c8ac/tmp_test_proxy_server_chunked/container-sync-realms.conf' test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test INFO: - - 02/Jun/2024/16/28/27 POST /v1/a/c HTTP/1.0 200 - - - 11 8 - - Host:%20localhost:80 0.0000 - - 1717345707.421043634 1717345707.421082258 - test INFO: - - 02/Jun/2024/16/28/36 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0001 - - 1717345716.817221403 1717345716.817277670 0 test INFO: - - 02/Jun/2024/16/28/36 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717345716.818554163 1717345716.818588972 -1 test INFO: - - 02/Jun/2024/16/28/36 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717345716.820161819 1717345716.820196152 - test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2 new rows) test-replicator ERROR: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (negative max_row_delta: -1) test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Failed to delete db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py ERROR reading HTTP response from {'replication_ip': '127.0.0.1', 'replication_port': 80, 'device': 'sdb1'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 170, in replicate self.request('REPLICATE', self.path, body, File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 353, in other_req raise Exception('blah') Exception: blah test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_get_info test.unit.proxy.controllers.test_base.TestFuncs.test_get_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission ... ok test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 will be moved to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 0 to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 0 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 0 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/0:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c ت/o1 ت' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c ت/o1 ت' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c ت/o1 ت' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345698.013379) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345698.013379) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345698.013379) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345698.013379) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717345698.01338) in policy_index 1 responded 503 (0000000000.00000) test INFO: retry: '/AUTH_bob/c/o1' (1717345698.013379) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717345698.022318) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717345698.022318) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717345698.022318) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717345698.022318) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717345698.02232) in policy_index 1 responded 404 (1717345698.02232_0000000000000002) test INFO: retry: '/AUTH_bob/c/o1' (1717345698.022318) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: unavailable_container: '/AUTH_bob/c/o1' (3618.841870) unable to determine the destination policy_index test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_container': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put test.unit.obj.test_server.TestObjectServer.test_expect_on_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_not_found test.unit.obj.test_server.TestObjectServer.test_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_options_info test.unit.proxy.controllers.test_info.TestInfoController.test_options_info ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body ... ok test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Setting object count to 1058 (txn: txbbc5b416b857445db9295-00665c9dbc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Setting object count to 1024 (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Setting object count to 1024 (txn: tx0e9088d70eac4ae79adb6-00665c9dbf) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Setting object count to 1072 (txn: txaffa585b09564608aa3a1-00665c9dc2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple ... ok d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" with 3.14159265359 weight got id 4 d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Partition power increase in progress. You need to finish the increase first before removing devices. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 test.test.com 6200 r.test.com 7000 sda3 3.14 2 33.67 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data No matching devices found swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info test.unit.cli.test_ringbuilder.TestCommands.test_set_info ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0120 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0107 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0108 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0106 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd87d03424dd8450993432-00665c9dc7" "proxy-server 3543380" 0.0022 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:34057/sda1/3/a/c" "txd87d03424dd8450993432-00665c9dc7" "container-server 3543380" 0.0019 "-" 3543380 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:40653/sdb1/3/a/c" "txd87d03424dd8450993432-00665c9dc7" "container-server 3543380" 0.0018 "-" 3543380 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txd87d03424dd8450993432-00665c9dc7" "proxy-server 3543380" 0.0368 "-" 3543380 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txd87d03424dd8450993432-00665c9dc7" "proxy-server 3543380" 0.0224 "-" 3543380 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/55 PUT /v1/a/c HTTP/1.0 201 - - t - - - txd87d03424dd8450993432-00665c9dc7 - 0.0533 - - 1717345735.152089357 1717345735.205411434 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txbbe2a310798747e7a1fc9-00665c9dc7" "proxy-server 3543380" 0.0023 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:40653/sdb1/0/a1/c1" "txbbe2a310798747e7a1fc9-00665c9dc7" "container-server 3543380" 0.0021 "-" 3543380 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:34057/sda1/0/a1/c1" "txbbe2a310798747e7a1fc9-00665c9dc7" "container-server 3543380" 0.0018 "-" 3543380 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbbe2a310798747e7a1fc9-00665c9dc7" "proxy-server 3543380" 0.0211 "-" 3543380 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbbe2a310798747e7a1fc9-00665c9dc7" "proxy-server 3543380" 0.0364 "-" 3543380 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/55 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txbbe2a310798747e7a1fc9-00665c9dc7 - 0.0501 - - 1717345735.208013773 1717345735.258149385 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txea1fac92cfaa4682a0bfc-00665c9dc7" "proxy-server 3543380" 0.0098 "-" 3543380 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:34057/sda1/3/a/c1" "txea1fac92cfaa4682a0bfc-00665c9dc7" "container-server 3543380" 0.0024 "-" 3543380 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:40653/sdb1/3/a/c1" "txea1fac92cfaa4682a0bfc-00665c9dc7" "container-server 3543380" 0.0081 "-" 3543380 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txea1fac92cfaa4682a0bfc-00665c9dc7" "proxy-server 3543380" 0.0353 "-" 3543380 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txea1fac92cfaa4682a0bfc-00665c9dc7" "proxy-server 3543380" 0.0506 "-" 3543380 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/55 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txea1fac92cfaa4682a0bfc-00665c9dc7 - 0.0730 - - 1717345735.260825396 1717345735.333856821 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa292518f99304973a31be-00665c9dc7" "proxy-server 3543380" 0.0077 "-" 3543380 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:40653/sdb1/0/a/c2" "txa292518f99304973a31be-00665c9dc7" "container-server 3543380" 0.0021 "-" 3543380 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:34057/sda1/0/a/c2" "txa292518f99304973a31be-00665c9dc7" "container-server 3543380" 0.0068 "-" 3543380 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa292518f99304973a31be-00665c9dc7" "proxy-server 3543380" 0.0552 "-" 3543380 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:55 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa292518f99304973a31be-00665c9dc7" "proxy-server 3543380" 0.0391 "-" 3543380 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/55 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txa292518f99304973a31be-00665c9dc7 - 0.0770 - - 1717345735.336593390 1717345735.413617849 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant AllUsers WRITE permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options test.unit.proxy.controllers.test_base.TestFuncs.test_options ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists ... ok test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/0 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 1, 'weight': 10.0, 'zone': 1, 'ip': '1.1.1.1', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.1', 'replication_port': 6200, 'region': 1}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 675, in _replicate_object success = self._repl_to_node(node, broker, partition, info, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 481, in _repl_to_node return self._handle_sync_response(node, response, info, broker, http, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 486, in _handle_sync_response if response.status == HTTP_NOT_FOUND: # completely missing, rsync ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 713, in status raise self._status Exception: ugh test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 2, 'weight': 10.0, 'zone': 2, 'ip': '1.1.1.2', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.2', 'replication_port': 6200, 'region': 2}: Timeout (Nones) test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (1/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator ERROR: ERROR Remote drive not mounted {'id': 3, 'weight': 10.0, 'zone': 3, 'ip': '1.1.1.3', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.3', 'replication_port': 6200, 'region': 1} test-replicator ERROR: ERROR Remote drive not mounted {'id': 4, 'weight': 10.0, 'zone': 4, 'ip': '1.1.1.4', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.4', 'replication_port': 6200, 'region': 2} test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.5:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (3/5 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator ERROR: Unable to decode remote metadata 'metadata' test-replicator ERROR: Quarantining DB test-replicator ERROR: ERROR Failed to get my own IPs? test-replicator ERROR: Can't find itself 127.0.0.1 with port 1000 in ring file, not replicating test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00032 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00072 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: Boom! test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00054 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 1 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: /srv/node/sdb is not mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_stats test.unit.common.test_db_replicator.TestDBReplicator.test_stats ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync test.unit.common.test_db_replicator.TestDBReplicator.test_usync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_increment test.unit.common.test_error_limiter.TestErrorLimiter.test_increment ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345710.54831', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345710.54831', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345710.54831', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345710.56124', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345710.56124', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.47294', 'epoch': '1717345710.47294', 'reported': 0, 'tombstones': -1}], 'ae9f7d0b-b85b-4527-8559-af59bb688f3c-') test DEBUG: /tmp/tmpdaqr4jd4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.47294.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpdaqr4jd4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.47294.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'aefe49294140f5edb63a2ae051915215', '2886efe1-b934-405b-b9ce-5e2abea42870-', '1717345710.70725', '1717345710.70173', '0', '', '1717345710.70173', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345710.81315', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345710.81315', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345710.81315', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345710.82513', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345710.82513', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.71957', 'epoch': '1717345710.71957', 'reported': 0, 'tombstones': -1}], '2886efe1-b934-405b-b9ce-5e2abea42870-') test DEBUG: /tmp/tmp4sm6mnwz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.71957.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp4sm6mnwz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.71957.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'aefe49294140f5edb63a2ae051915215', '2886efe1-b934-405b-b9ce-5e2abea42870-', '1717345710.70725', '1717345710.79729', '0', '', '1717345710.70173', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345710.81315', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345710.81315', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345710.81315', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345710.82513', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345710.82513', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.71957', 'epoch': '1717345710.71957', 'reported': 0, 'tombstones': -1}], '2886efe1-b934-405b-b9ce-5e2abea42870-') test DEBUG: /tmp/tmp4sm6mnwz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.71957.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp4sm6mnwz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.71957.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '9a9ce5a8f20254a3cb748839c187f8fc', '0117a795-9535-411e-b84d-a37e17fc18b2-', '1717345710.98049', '1717345710.97585', '0', '', '1717345710.97585', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.06768', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.06768', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345711.06768', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.07552', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.07552', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.99044', 'epoch': '1717345710.99044', 'reported': 0, 'tombstones': -1}], '0117a795-9535-411e-b84d-a37e17fc18b2-') test DEBUG: /tmp/tmp_4z25o2z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.99044.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp_4z25o2z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.99044.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '9a9ce5a8f20254a3cb748839c187f8fc', '0117a795-9535-411e-b84d-a37e17fc18b2-', '1717345710.98049', '1717345711.05369', '0', '', '1717345710.97585', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.06768', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.06768', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345711.06768', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.07552', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.07552', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345710.99044', 'epoch': '1717345710.99044', 'reported': 0, 'tombstones': -1}], '0117a795-9535-411e-b84d-a37e17fc18b2-') test DEBUG: /tmp/tmp_4z25o2z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.99044.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp_4z25o2z/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345710.99044.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '49962d3298b4824e6ccabae25abe188c', '3aa0912a-b50e-432b-afb3-fe64db5d72da-', '1717345711.29009', '1717345711.28067', '0', '', '1717345711.28067', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.38944', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.38944', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345711.38944', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.42443', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.42443', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.29910', 'epoch': '1717345711.29910', 'reported': 0, 'tombstones': -1}], '3aa0912a-b50e-432b-afb3-fe64db5d72da-') test DEBUG: /tmp/tmpprhtziao/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.29910.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpprhtziao/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.29910.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission ... ok test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Setting object count to 1072 (txn: txb9607da3a1b04ec88d614-00665c9dc5) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Setting object count to 1072 (txn: tx080b17adc5464d1383651-00665c9dc8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx25954291ad544cbda6670-00665c9dcb) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: tx1de56c1dfa864489950c9-00665c9dc6) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: tx1de56c1dfa864489950c9-00665c9dc6) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: tx1de56c1dfa864489950c9-00665c9dc6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: txdf533762464148139d7f4-00665c9dc6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdf533762464148139d7f4-00665c9dc6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdf533762464148139d7f4-00665c9dc6) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: txdf533762464148139d7f4-00665c9dc6) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx6e1bc373881f4a6287635-00665c9dc6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6e1bc373881f4a6287635-00665c9dc6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6e1bc373881f4a6287635-00665c9dc6) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx6e1bc373881f4a6287635-00665c9dc6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: Setting object count to 3120 (txn: txd5950e39be964fd9a8674-00665c9dc9) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: Setting object count to 3120 (txn: tx7c26aa165d334027ba53a-00665c9dca) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: Setting object count to 3120 (txn: txf2508f9ff4a64fc9a5521-00665c9dca) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: Setting object count to 3120 (txn: tx655a7d85c45c42fe91452-00665c9dca) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx0f0cb507ad494d04ad735-00665c9dca) test DEBUG: Setting object count to 3120 (txn: tx0f0cb507ad494d04ad735-00665c9dca) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"something" Device d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" is now d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"something" Device d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" is now d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"something" Device d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" is now d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"something" Device 1 already uses 127.0.0.2:6201/sda2. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" is now d4r2z3-test.test2.com:9000Rr.test2.com:9000/sda300_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.2.1:9000R127.0.2.1:9000/sda100_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.1.1:8000/sda10_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[4001:0:1234::c1c0:abcd:876]:9000R[4::10]:9000/sda300_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:8000/sda30_"other meta data" Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The minimum number of hours before a partition can be reassigned is now set to 24 test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload test.unit.cli.test_ringbuilder.TestCommands.test_set_overload ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region test.unit.cli.test_ringbuilder.TestCommands.test_set_region ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight test.unit.cli.test_ringbuilder.TestCommands.test_set_weight ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices ... ok test ERROR: Unhandled Exception trying to reconcile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 852, in run_once self.reconcile() File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 831, in reconcile for container in self._iter_containers(): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 748, in _iter_containers one_page = list(itertools.islice( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 331, in _iter_items resp = self.make_request( ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: Reconciler Stats: {} test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.11:1011/sdl, error: ERROR Insufficient Storage (txn: txba7f934e15c84486ab435-00665c9db2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok proxy-server ERROR: Object returning 503 for [] (txn: tx15ea64890b3d4c5a9b52d-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Ignoring 404 from handoff (txn: txdbea27968ded4e3481339-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfedab62d3e4e406983d3f-00665c9db9) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfedab62d3e4e406983d3f-00665c9db9) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfedab62d3e4e406983d3f-00665c9db9) test ERROR: Container HEAD returning 503 for [] (txn: txfedab62d3e4e406983d3f-00665c9db9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe0892c36a9304ae5b2682-00665c9db9) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe0892c36a9304ae5b2682-00665c9db9) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe0892c36a9304ae5b2682-00665c9db9) test ERROR: Container HEAD returning 503 for [] (txn: txe0892c36a9304ae5b2682-00665c9db9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta ... ok test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.05966 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 6 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 0 dbs in 0.00000 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 30 dbs in 246.81358 seconds (0.12155/s) test-replicator INFO: Removed 9 dbs test-replicator INFO: 25 successes, 1 failures test-replicator INFO: diff:5 diff_capped:4 empty:7 hashmatch:8 no_change:6 remote_merge:2 rsync:3 ts_repl:10 proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx77fee1bccb5f478d9836a-00665c9dc2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txc9ea29e85e7147b2891ef-00665c9dc2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpqd54mg6j/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpqd54mg6j/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpqd54mg6j/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpqd54mg6j/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0115 "-" 3543382 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0092 "-" 3543382 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0090 "-" 3543382 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0088 "-" 3543382 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx59504d2804d34faebfcc6-00665c9dc2" "proxy-server 3543382" 0.0018 "-" 3543382 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:52829/sdb1/1/a/c" "tx59504d2804d34faebfcc6-00665c9dc2" "container-server 3543382" 0.0014 "-" 3543382 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:33423/sda1/1/a/c" "tx59504d2804d34faebfcc6-00665c9dc2" "container-server 3543382" 0.0015 "-" 3543382 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx59504d2804d34faebfcc6-00665c9dc2" "proxy-server 3543382" 0.0286 "-" 3543382 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx59504d2804d34faebfcc6-00665c9dc2" "proxy-server 3543382" 0.0184 "-" 3543382 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/50 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx59504d2804d34faebfcc6-00665c9dc2 - 0.0409 - - 1717345730.758123875 1717345730.799006462 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx11c6f2300bce4381bf368-00665c9dc2" "proxy-server 3543382" 0.0017 "-" 3543382 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:33423/sda1/1/a1/c1" "tx11c6f2300bce4381bf368-00665c9dc2" "container-server 3543382" 0.0014 "-" 3543382 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:52829/sdb1/1/a1/c1" "tx11c6f2300bce4381bf368-00665c9dc2" "container-server 3543382" 0.0013 "-" 3543382 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx11c6f2300bce4381bf368-00665c9dc2" "proxy-server 3543382" 0.0346 "-" 3543382 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx11c6f2300bce4381bf368-00665c9dc2" "proxy-server 3543382" 0.0195 "-" 3543382 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/50 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx11c6f2300bce4381bf368-00665c9dc2 - 0.0456 - - 1717345730.800817966 1717345730.846465111 0 test.unit.proxy.test_server.TestAccountController.test_DELETE test.unit.proxy.test_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_server.TestAccountController.test_GET test.unit.proxy.test_server.TestAccountController.test_GET ... ok test.unit.proxy.test_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD test.unit.proxy.test_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_OPTIONS test.unit.proxy.test_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_POST_metadata test.unit.proxy.test_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT test.unit.proxy.test_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_status test.unit.proxy.test_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_server.TestAccountController.test_connection_refused test.unit.proxy.test_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_server.TestAccountController.test_other_socket_error test.unit.proxy.test_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestController.test_account_info_200 test.unit.proxy.test_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_404 test.unit.proxy.test_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_container_count test.unit.proxy.test_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_account test.unit.proxy.test_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_cache test.unit.proxy.test_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_200 test.unit.proxy.test_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_404 test.unit.proxy.test_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_invalid_account test.unit.proxy.test_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_no_cache test.unit.proxy.test_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_make_requests test.unit.proxy.test_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_transfer_headers test.unit.proxy.test_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '49962d3298b4824e6ccabae25abe188c', '3aa0912a-b50e-432b-afb3-fe64db5d72da-', '1717345711.29009', '1717345711.37221', '0', '', '1717345711.28067', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.38944', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.38944', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345711.38944', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.42443', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.42443', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.29910', 'epoch': '1717345711.29910', 'reported': 0, 'tombstones': -1}], '3aa0912a-b50e-432b-afb3-fe64db5d72da-') test DEBUG: /tmp/tmpprhtziao/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.29910.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpprhtziao/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.29910.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '70192325e007408903014e65e682710e', '445bdafc-0ac4-4738-8ba7-4ba703e25739-', '1717345711.58901', '1717345711.58439', '0', '', '1717345711.58439', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.59913', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345711.59913', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345711.59913', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.59954', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.59954', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.59930', 'epoch': '1717345711.59930', 'reported': 0, 'tombstones': -1}], '445bdafc-0ac4-4738-8ba7-4ba703e25739-') test DEBUG: /tmp/tmp1ah46ce7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.59930.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp1ah46ce7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.59930.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '70192325e007408903014e65e682710e', '445bdafc-0ac4-4738-8ba7-4ba703e25739-', '1717345711.58901', '1717345711.68262', '0', '', '1717345711.58439', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.59913', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345711.59913', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345711.59913', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.59954', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.59954', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.59930', 'epoch': '1717345711.59930', 'reported': 0, 'tombstones': -1}], '445bdafc-0ac4-4738-8ba7-4ba703e25739-') test DEBUG: /tmp/tmp1ah46ce7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.59930.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp1ah46ce7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.59930.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '70192325e007408903014e65e682710e', 'b1e41869-6c3a-4e88-ae1b-b429d4e5fb72-', '1717345711.77734', '1717345711.77269', '0', '', '1717345711.77269', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.78573', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345711.78573', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345711.78573', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.78610', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.78610', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.78589', 'epoch': '1717345711.78589', 'reported': 0, 'tombstones': -1}], 'b1e41869-6c3a-4e88-ae1b-b429d4e5fb72-') test DEBUG: /tmp/tmp60lhhq_k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.78589.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp60lhhq_k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.78589.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '70192325e007408903014e65e682710e', 'b1e41869-6c3a-4e88-ae1b-b429d4e5fb72-', '1717345711.77734', '1717345711.83943', '0', '', '1717345711.77269', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.78573', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345711.78573', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345711.78573', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.78610', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.78610', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.78589', 'epoch': '1717345711.78589', 'reported': 0, 'tombstones': -1}], 'b1e41869-6c3a-4e88-ae1b-b429d4e5fb72-') test DEBUG: /tmp/tmp60lhhq_k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.78589.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp60lhhq_k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.78589.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, 'b7746a579a53ee3e4a221064e021cd16', '8877b584-0522-4550-a036-ebed4993ee29-', '1717345711.93892', '1717345711.93559', '0', '', '1717345711.93559', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8877b584-0522-4550-a036-ebed4993ee29-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.97547.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, 'b7746a579a53ee3e4a221064e021cd16', '8877b584-0522-4550-a036-ebed4993ee29-', '1717345711.93892', '1717345711.93559', '0', '', '1717345711.93559', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345711.94576', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.94576', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345711.94576', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345711.97590', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345711.97590', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345711.97547', 'epoch': '1717345711.97547', 'reported': 0, 'tombstones': -1}], '8877b584-0522-4550-a036-ebed4993ee29-') test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding ... ok test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx52acae36186a435f93086-00665c9dca) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx129970caaba44f6ba99b3-00665c9dca) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx8e8e0caa17384b94be839-00665c9dca) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx2b94ed5975d04fe89e20d-00665c9dca) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2b94ed5975d04fe89e20d-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2b94ed5975d04fe89e20d-00665c9dca) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx2b94ed5975d04fe89e20d-00665c9dca) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txc6dd78113a3e48dea2439-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc6dd78113a3e48dea2439-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txc6dd78113a3e48dea2439-00665c9dca) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txc6dd78113a3e48dea2439-00665c9dca) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx3474b4b535974a64ac09c-00665c9dca) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx3474b4b535974a64ac09c-00665c9dca) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx3474b4b535974a64ac09c-00665c9dca) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx3474b4b535974a64ac09c-00665c9dca) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:50 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx78d57ab2b3be474ea96cb-00665c9dc2" "proxy-server 3543382" 0.0061 "-" 3543382 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:33423/sda1/2/a/c1" "tx78d57ab2b3be474ea96cb-00665c9dc2" "container-server 3543382" 0.0021 "-" 3543382 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:52829/sdb1/2/a/c1" "tx78d57ab2b3be474ea96cb-00665c9dc2" "container-server 3543382" 0.0075 "-" 3543382 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx78d57ab2b3be474ea96cb-00665c9dc2" "proxy-server 3543382" 0.0272 "-" 3543382 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx78d57ab2b3be474ea96cb-00665c9dc2" "proxy-server 3543382" 0.0419 "-" 3543382 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/53 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx78d57ab2b3be474ea96cb-00665c9dc2 - 2.8004 - - 1717345730.848305702 1717345733.648690462 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx0d2a29f5833940409fa77-00665c9dc5" "proxy-server 3543382" 0.0082 "-" 3543382 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:52829/sdb1/1/a/c2" "tx0d2a29f5833940409fa77-00665c9dc5" "container-server 3543382" 0.0018 "-" 3543382 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:33423/sda1/1/a/c2" "tx0d2a29f5833940409fa77-00665c9dc5" "container-server 3543382" 0.0072 "-" 3543382 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx0d2a29f5833940409fa77-00665c9dc5" "proxy-server 3543382" 0.0274 "-" 3543382 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:28:53 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx0d2a29f5833940409fa77-00665c9dc5" "proxy-server 3543382" 0.0406 "-" 3543382 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/28/53 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx0d2a29f5833940409fa77-00665c9dc5 - 0.0609 - - 1717345733.651597977 1717345733.712470293 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission ... ok swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. The overload factor is now 19.88% (0.198780) The change will take effect after the next rebalance. Overload must be non-negative. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. 'swedish fish' is not a valid number. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The replica count is now 3.141593. The change will take effect after the next rebalance. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. "test" is not a valid number. Replica count must be at least 1. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 200.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 200.0 d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 200.0 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 200.0 test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone test.unit.cli.test_ringbuilder.TestCommands.test_set_zone ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: Setting object count to 1068 (txn: tx62f5e68823fa4ece8a52c-00665c9dcd) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Setting object count to 1068 (txn: tx3cb082cae1044c4495ddd-00665c9dcd) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txfae2bee4f54c434e85b72-00665c9dd0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfae2bee4f54c434e85b72-00665c9dd0) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txfae2bee4f54c434e85b72-00665c9dd0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfae2bee4f54c434e85b72-00665c9dd0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfae2bee4f54c434e85b72-00665c9dd0) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfae2bee4f54c434e85b72-00665c9dd0) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txfae2bee4f54c434e85b72-00665c9dd0) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: txfae2bee4f54c434e85b72-00665c9dd0) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx6250e79474694e4ca23ba-00665c9dd3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6250e79474694e4ca23ba-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6250e79474694e4ca23ba-00665c9dd3) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx6250e79474694e4ca23ba-00665c9dd3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6250e79474694e4ca23ba-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx6250e79474694e4ca23ba-00665c9dd3) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: tx6250e79474694e4ca23ba-00665c9dd3) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: tx6250e79474694e4ca23ba-00665c9dd3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test WARNING: Skipping /tmp/tmplaep0fiy/devices3/device_is_file/data because [Errno 20] Not a directory: '/tmp/tmplaep0fiy/devices3/device_is_file/data' test WARNING: Skipping /tmp/tmplaep0fiy/devices4/device/data because test WARNING: Skipping device as it is not mounted test WARNING: Skipping /tmp/tmput38pf9s/asdf/data because [Errno 20] Not a directory: '/tmp/tmput38pf9s/asdf/data' test WARNING: Skipping /tmp/tmput38pf9s/asdf/data because [Errno 20] Not a directory: '/tmp/tmput38pf9s/asdf/data' test WARNING: Skipping drive as it is not mounted test WARNING: Skipping asdf as it is not mounted test WARNING: Skipping /tmp/tmpvlh2cwvc/asdf/data because [Errno 20] Not a directory: '/tmp/tmpvlh2cwvc/asdf/data' test DEBUG: set nice priority to 1 test DEBUG: set ionice class 1 priority 0 WARNING: Unable to modify I/O scheduling class and priority of process. Keeping unchanged! Check logs for more info. test ERROR: Unable to modify ionice priority: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/libc.py", line 326, in _ioprio_set io_class = IO_CLASS_ENUM[io_class] ~~~~~~~~~~~~~^^^^^^^^^^ KeyError: 'class_foo' test DEBUG: set ionice class 2 priority 4 test DEBUG: set nice priority to -15 test DEBUG: set ionice class 3 priority 6 sharder-test WARNING: Reclaimable db stuck waiting for shrinking, path: a/c, db: /tmp/tmpqi7q0vvg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717345691.00000, (0, 0) as of 1717345691.00000, active as of 1717345691.00000> to ShardRange<'k' to 't' as of 1717345692.00000, (0, 0) as of 1717345692.00000, active as of 1717345692.00000>, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345696.00000, (0, 0) as of 1717345696.00000, shrinking as of 1717345693.00000> to ShardRange<'k' to 't' as of 1717345696.00000, (0, 0) as of 1717345696.00000, sharding as of 1717345697.00000>, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmp58e5k9qb/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345691.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/4f3/780b9f31c648d0b03a5586cc97ed64f3/780b9f31c648d0b03a5586cc97ed64f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345693.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/6a8/9adbcda6eaec95df24ce9708d38a06a8/9adbcda6eaec95df24ce9708d38a06a8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345692.00000, (0, 0) as of 1717345692.00000, found as of 1717345692.00000> to ShardRange<'k' to 't' as of 1717345693.00000, (0, 0) as of 1717345693.00000, created as of 1717345693.00000>, path: .shards_a/shard_c_1717345693.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/6a8/9adbcda6eaec95df24ce9708d38a06a8/9adbcda6eaec95df24ce9708d38a06a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345695.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/17d/d5576a4debc3b2a12f4a2e641926e17d/d5576a4debc3b2a12f4a2e641926e17d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345694.00000, (0, 0) as of 1717345694.00000, found as of 1717345694.00000> to ShardRange<'k' to 't' as of 1717345695.00000, (0, 0) as of 1717345695.00000, cleaved as of 1717345695.00000>, path: .shards_a/shard_c_1717345695.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/17d/d5576a4debc3b2a12f4a2e641926e17d/d5576a4debc3b2a12f4a2e641926e17d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345697.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ef8/7a77507815fc2028433af9966aba2ef8/7a77507815fc2028433af9966aba2ef8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345696.00000, (0, 0) as of 1717345696.00000, found as of 1717345696.00000> to ShardRange<'k' to 't' as of 1717345697.00000, (0, 0) as of 1717345697.00000, active as of 1717345697.00000>, path: .shards_a/shard_c_1717345697.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ef8/7a77507815fc2028433af9966aba2ef8/7a77507815fc2028433af9966aba2ef8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345699.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7a4/a2a174c333f3de573e9b43defa6747a4/a2a174c333f3de573e9b43defa6747a4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345698.00000, (0, 0) as of 1717345698.00000, found as of 1717345698.00000> to ShardRange<'k' to 't' as of 1717345699.00000, (0, 0) as of 1717345699.00000, shrinking as of 1717345699.00000>, path: .shards_a/shard_c_1717345699.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7a4/a2a174c333f3de573e9b43defa6747a4/a2a174c333f3de573e9b43defa6747a4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345699.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7a4/a2a174c333f3de573e9b43defa6747a4/a2a174c333f3de573e9b43defa6747a4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345701.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/593/de0ce2b3a8319a9a87a9696c7ff4e593/de0ce2b3a8319a9a87a9696c7ff4e593.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345700.00000, (0, 0) as of 1717345700.00000, found as of 1717345700.00000> to ShardRange<'k' to 't' as of 1717345701.00000, (0, 0) as of 1717345701.00000, sharding as of 1717345701.00000>, path: .shards_a/shard_c_1717345701.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/593/de0ce2b3a8319a9a87a9696c7ff4e593/de0ce2b3a8319a9a87a9696c7ff4e593.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345701.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/593/de0ce2b3a8319a9a87a9696c7ff4e593/de0ce2b3a8319a9a87a9696c7ff4e593.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345703.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a3f/415ef52f40b5fbcc1312042247abfa3f/415ef52f40b5fbcc1312042247abfa3f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345702.00000, (0, 0) as of 1717345702.00000, found as of 1717345702.00000> to ShardRange<'k' to 't' as of 1717345703.00000, (0, 0) as of 1717345703.00000, sharded as of 1717345703.00000>, path: .shards_a/shard_c_1717345703.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a3f/415ef52f40b5fbcc1312042247abfa3f/415ef52f40b5fbcc1312042247abfa3f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345703.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a3f/415ef52f40b5fbcc1312042247abfa3f/415ef52f40b5fbcc1312042247abfa3f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345705.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/380/b8a3d4e754b62799c22959b683cac380/b8a3d4e754b62799c22959b683cac380.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345704.00000, (0, 0) as of 1717345704.00000, found as of 1717345704.00000> to ShardRange<'k' to 't' as of 1717345705.00000, (0, 0) as of 1717345705.00000, shrunk as of 1717345705.00000>, path: .shards_a/shard_c_1717345705.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/380/b8a3d4e754b62799c22959b683cac380/b8a3d4e754b62799c22959b683cac380.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345705.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/380/b8a3d4e754b62799c22959b683cac380/b8a3d4e754b62799c22959b683cac380.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345707.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8dc/b3532adedae01ab5d82a69ffb4f4f8dc/b3532adedae01ab5d82a69ffb4f4f8dc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345706.00000, (0, 0) as of 1717345706.00000, created as of 1717345706.00000> to ShardRange<'k' to 't' as of 1717345707.00000, (0, 0) as of 1717345707.00000, found as of 1717345707.00000>, path: .shards_a/shard_c_1717345707.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8dc/b3532adedae01ab5d82a69ffb4f4f8dc/b3532adedae01ab5d82a69ffb4f4f8dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345709.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dc7/ed05ef0b71057372b89c4ecb64b5adc7/ed05ef0b71057372b89c4ecb64b5adc7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345711.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/92b/8da61593aacde62780ff0669cbffc92b/8da61593aacde62780ff0669cbffc92b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345710.00000, (0, 0) as of 1717345710.00000, created as of 1717345710.00000> to ShardRange<'k' to 't' as of 1717345711.00000, (0, 0) as of 1717345711.00000, cleaved as of 1717345711.00000>, path: .shards_a/shard_c_1717345711.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/92b/8da61593aacde62780ff0669cbffc92b/8da61593aacde62780ff0669cbffc92b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345713.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7c3/e1aa54341b3ccedcd11f340eae5bc7c3/e1aa54341b3ccedcd11f340eae5bc7c3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345712.00000, (0, 0) as of 1717345712.00000, created as of 1717345712.00000> to ShardRange<'k' to 't' as of 1717345713.00000, (0, 0) as of 1717345713.00000, active as of 1717345713.00000>, path: .shards_a/shard_c_1717345713.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7c3/e1aa54341b3ccedcd11f340eae5bc7c3/e1aa54341b3ccedcd11f340eae5bc7c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345715.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7e7/4a60d1edfa3d86115c1c61ae06fc87e7/4a60d1edfa3d86115c1c61ae06fc87e7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345714.00000, (0, 0) as of 1717345714.00000, created as of 1717345714.00000> to ShardRange<'k' to 't' as of 1717345715.00000, (0, 0) as of 1717345715.00000, shrinking as of 1717345715.00000>, path: .shards_a/shard_c_1717345715.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7e7/4a60d1edfa3d86115c1c61ae06fc87e7/4a60d1edfa3d86115c1c61ae06fc87e7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345715.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7e7/4a60d1edfa3d86115c1c61ae06fc87e7/4a60d1edfa3d86115c1c61ae06fc87e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345717.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9df/25fa5986850c6a6755b2568d498b19df/25fa5986850c6a6755b2568d498b19df.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345716.00000, (0, 0) as of 1717345716.00000, created as of 1717345716.00000> to ShardRange<'k' to 't' as of 1717345717.00000, (0, 0) as of 1717345717.00000, sharding as of 1717345717.00000>, path: .shards_a/shard_c_1717345717.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9df/25fa5986850c6a6755b2568d498b19df/25fa5986850c6a6755b2568d498b19df.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345717.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9df/25fa5986850c6a6755b2568d498b19df/25fa5986850c6a6755b2568d498b19df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345719.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9b4/26662a13c78cb80827f0e4301c8109b4/26662a13c78cb80827f0e4301c8109b4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345718.00000, (0, 0) as of 1717345718.00000, created as of 1717345718.00000> to ShardRange<'k' to 't' as of 1717345719.00000, (0, 0) as of 1717345719.00000, sharded as of 1717345719.00000>, path: .shards_a/shard_c_1717345719.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9b4/26662a13c78cb80827f0e4301c8109b4/26662a13c78cb80827f0e4301c8109b4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345719.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9b4/26662a13c78cb80827f0e4301c8109b4/26662a13c78cb80827f0e4301c8109b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345721.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3db/9f436570a4420ca7c6988b07c58643db/9f436570a4420ca7c6988b07c58643db.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345720.00000, (0, 0) as of 1717345720.00000, created as of 1717345720.00000> to ShardRange<'k' to 't' as of 1717345721.00000, (0, 0) as of 1717345721.00000, shrunk as of 1717345721.00000>, path: .shards_a/shard_c_1717345721.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3db/9f436570a4420ca7c6988b07c58643db/9f436570a4420ca7c6988b07c58643db.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345721.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3db/9f436570a4420ca7c6988b07c58643db/9f436570a4420ca7c6988b07c58643db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345723.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/0c6/90e978d226c9b745f145c976862cc0c6/90e978d226c9b745f145c976862cc0c6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345722.00000, (0, 0) as of 1717345722.00000, cleaved as of 1717345722.00000> to ShardRange<'k' to 't' as of 1717345723.00000, (0, 0) as of 1717345723.00000, found as of 1717345723.00000>, path: .shards_a/shard_c_1717345723.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/0c6/90e978d226c9b745f145c976862cc0c6/90e978d226c9b745f145c976862cc0c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345725.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f9a/ea99d1cbdeb84ef7089962927024cf9a/ea99d1cbdeb84ef7089962927024cf9a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345724.00000, (0, 0) as of 1717345724.00000, cleaved as of 1717345724.00000> to ShardRange<'k' to 't' as of 1717345725.00000, (0, 0) as of 1717345725.00000, created as of 1717345725.00000>, path: .shards_a/shard_c_1717345725.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f9a/ea99d1cbdeb84ef7089962927024cf9a/ea99d1cbdeb84ef7089962927024cf9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345727.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d54/56f508f8718140269722c70731981d54/56f508f8718140269722c70731981d54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345729.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a21/4fe6b65d4e117a414686bcfc99adea21/4fe6b65d4e117a414686bcfc99adea21.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345728.00000, (0, 0) as of 1717345728.00000, cleaved as of 1717345728.00000> to ShardRange<'k' to 't' as of 1717345729.00000, (0, 0) as of 1717345729.00000, active as of 1717345729.00000>, path: .shards_a/shard_c_1717345729.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a21/4fe6b65d4e117a414686bcfc99adea21/4fe6b65d4e117a414686bcfc99adea21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345731.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a8d/afcb8bd35f573f4749dddce34be49a8d/afcb8bd35f573f4749dddce34be49a8d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345730.00000, (0, 0) as of 1717345730.00000, cleaved as of 1717345730.00000> to ShardRange<'k' to 't' as of 1717345731.00000, (0, 0) as of 1717345731.00000, shrinking as of 1717345731.00000>, path: .shards_a/shard_c_1717345731.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a8d/afcb8bd35f573f4749dddce34be49a8d/afcb8bd35f573f4749dddce34be49a8d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345731.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a8d/afcb8bd35f573f4749dddce34be49a8d/afcb8bd35f573f4749dddce34be49a8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345733.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/34c/ff20252b71895af2397140d0ca39a34c/ff20252b71895af2397140d0ca39a34c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345732.00000, (0, 0) as of 1717345732.00000, cleaved as of 1717345732.00000> to ShardRange<'k' to 't' as of 1717345733.00000, (0, 0) as of 1717345733.00000, sharding as of 1717345733.00000>, path: .shards_a/shard_c_1717345733.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/34c/ff20252b71895af2397140d0ca39a34c/ff20252b71895af2397140d0ca39a34c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345733.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/34c/ff20252b71895af2397140d0ca39a34c/ff20252b71895af2397140d0ca39a34c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345735.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/cec/73b72d88e364d31d869b261e6a892cec/73b72d88e364d31d869b261e6a892cec.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345734.00000, (0, 0) as of 1717345734.00000, cleaved as of 1717345734.00000> to ShardRange<'k' to 't' as of 1717345735.00000, (0, 0) as of 1717345735.00000, sharded as of 1717345735.00000>, path: .shards_a/shard_c_1717345735.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/cec/73b72d88e364d31d869b261e6a892cec/73b72d88e364d31d869b261e6a892cec.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345735.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/cec/73b72d88e364d31d869b261e6a892cec/73b72d88e364d31d869b261e6a892cec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345737.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a04/bed65b2f94f1144417a369de78ef5a04/bed65b2f94f1144417a369de78ef5a04.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345736.00000, (0, 0) as of 1717345736.00000, cleaved as of 1717345736.00000> to ShardRange<'k' to 't' as of 1717345737.00000, (0, 0) as of 1717345737.00000, shrunk as of 1717345737.00000>, path: .shards_a/shard_c_1717345737.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a04/bed65b2f94f1144417a369de78ef5a04/bed65b2f94f1144417a369de78ef5a04.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345737.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a04/bed65b2f94f1144417a369de78ef5a04/bed65b2f94f1144417a369de78ef5a04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345739.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/1ce/9199bf1416a7e2d51750644d4074a1ce/9199bf1416a7e2d51750644d4074a1ce.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345738.00000, (0, 0) as of 1717345738.00000, active as of 1717345738.00000> to ShardRange<'k' to 't' as of 1717345739.00000, (0, 0) as of 1717345739.00000, found as of 1717345739.00000>, path: .shards_a/shard_c_1717345739.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/1ce/9199bf1416a7e2d51750644d4074a1ce/9199bf1416a7e2d51750644d4074a1ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345741.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/2e4/bac6b6474fa8a4c3b9cf6e72734772e4/bac6b6474fa8a4c3b9cf6e72734772e4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345740.00000, (0, 0) as of 1717345740.00000, active as of 1717345740.00000> to ShardRange<'k' to 't' as of 1717345741.00000, (0, 0) as of 1717345741.00000, created as of 1717345741.00000>, path: .shards_a/shard_c_1717345741.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/2e4/bac6b6474fa8a4c3b9cf6e72734772e4/bac6b6474fa8a4c3b9cf6e72734772e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345743.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7b4/29a3576ef4514d2a4d06e64bb9fe97b4/29a3576ef4514d2a4d06e64bb9fe97b4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345742.00000, (0, 0) as of 1717345742.00000, active as of 1717345742.00000> to ShardRange<'k' to 't' as of 1717345743.00000, (0, 0) as of 1717345743.00000, cleaved as of 1717345743.00000>, path: .shards_a/shard_c_1717345743.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7b4/29a3576ef4514d2a4d06e64bb9fe97b4/29a3576ef4514d2a4d06e64bb9fe97b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345745.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/1c6/923911cebdbc943e59f7a835297f81c6/923911cebdbc943e59f7a835297f81c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345747.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/bff/9045fb5f97563bd4bd3218fa309b5bff/9045fb5f97563bd4bd3218fa309b5bff.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345746.00000, (0, 0) as of 1717345746.00000, active as of 1717345746.00000> to ShardRange<'k' to 't' as of 1717345747.00000, (0, 0) as of 1717345747.00000, shrinking as of 1717345747.00000>, path: .shards_a/shard_c_1717345747.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/bff/9045fb5f97563bd4bd3218fa309b5bff/9045fb5f97563bd4bd3218fa309b5bff.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345747.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/bff/9045fb5f97563bd4bd3218fa309b5bff/9045fb5f97563bd4bd3218fa309b5bff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345749.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/5d4/8fef6e33791d0a1ba6414409b71f75d4/8fef6e33791d0a1ba6414409b71f75d4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345748.00000, (0, 0) as of 1717345748.00000, active as of 1717345748.00000> to ShardRange<'k' to 't' as of 1717345749.00000, (0, 0) as of 1717345749.00000, sharding as of 1717345749.00000>, path: .shards_a/shard_c_1717345749.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/5d4/8fef6e33791d0a1ba6414409b71f75d4/8fef6e33791d0a1ba6414409b71f75d4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345749.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/5d4/8fef6e33791d0a1ba6414409b71f75d4/8fef6e33791d0a1ba6414409b71f75d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345751.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ebb/53314e2abdefc0ea04c34db5bd979ebb/53314e2abdefc0ea04c34db5bd979ebb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345750.00000, (0, 0) as of 1717345750.00000, active as of 1717345750.00000> to ShardRange<'k' to 't' as of 1717345751.00000, (0, 0) as of 1717345751.00000, sharded as of 1717345751.00000>, path: .shards_a/shard_c_1717345751.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ebb/53314e2abdefc0ea04c34db5bd979ebb/53314e2abdefc0ea04c34db5bd979ebb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345751.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ebb/53314e2abdefc0ea04c34db5bd979ebb/53314e2abdefc0ea04c34db5bd979ebb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345753.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e23/a902258193589666fba42331e57fae23/a902258193589666fba42331e57fae23.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345752.00000, (0, 0) as of 1717345752.00000, active as of 1717345752.00000> to ShardRange<'k' to 't' as of 1717345753.00000, (0, 0) as of 1717345753.00000, shrunk as of 1717345753.00000>, path: .shards_a/shard_c_1717345753.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e23/a902258193589666fba42331e57fae23/a902258193589666fba42331e57fae23.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345753.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e23/a902258193589666fba42331e57fae23/a902258193589666fba42331e57fae23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345755.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/492/1ca175177a7af7cc236beb1407cf5492/1ca175177a7af7cc236beb1407cf5492.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345754.00000, (0, 0) as of 1717345754.00000, shrinking as of 1717345754.00000> to ShardRange<'k' to 't' as of 1717345755.00000, (0, 0) as of 1717345755.00000, found as of 1717345755.00000>, path: .shards_a/shard_c_1717345755.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/492/1ca175177a7af7cc236beb1407cf5492/1ca175177a7af7cc236beb1407cf5492.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345757.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3ea/44988d6f500602c8ead3a462886ca3ea/44988d6f500602c8ead3a462886ca3ea.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345756.00000, (0, 0) as of 1717345756.00000, shrinking as of 1717345756.00000> to ShardRange<'k' to 't' as of 1717345757.00000, (0, 0) as of 1717345757.00000, created as of 1717345757.00000>, path: .shards_a/shard_c_1717345757.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3ea/44988d6f500602c8ead3a462886ca3ea/44988d6f500602c8ead3a462886ca3ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345759.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/271/470388be1d68e0ba34614900ea6c9271/470388be1d68e0ba34614900ea6c9271.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345758.00000, (0, 0) as of 1717345758.00000, shrinking as of 1717345758.00000> to ShardRange<'k' to 't' as of 1717345759.00000, (0, 0) as of 1717345759.00000, cleaved as of 1717345759.00000>, path: .shards_a/shard_c_1717345759.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/271/470388be1d68e0ba34614900ea6c9271/470388be1d68e0ba34614900ea6c9271.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345761.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/901/c42d2239714002ef4b6528185743b901/c42d2239714002ef4b6528185743b901.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345760.00000, (0, 0) as of 1717345760.00000, shrinking as of 1717345760.00000> to ShardRange<'k' to 't' as of 1717345761.00000, (0, 0) as of 1717345761.00000, active as of 1717345761.00000>, path: .shards_a/shard_c_1717345761.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/901/c42d2239714002ef4b6528185743b901/c42d2239714002ef4b6528185743b901.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345763.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/141/f36774315eb4ece0aebce35b4f631141/f36774315eb4ece0aebce35b4f631141.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345763.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/141/f36774315eb4ece0aebce35b4f631141/f36774315eb4ece0aebce35b4f631141.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345765.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d18/43d5c76dd22a25a0f9b10e1409241d18/43d5c76dd22a25a0f9b10e1409241d18.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345764.00000, (0, 0) as of 1717345764.00000, shrinking as of 1717345764.00000> to ShardRange<'k' to 't' as of 1717345765.00000, (0, 0) as of 1717345765.00000, sharding as of 1717345765.00000>, path: .shards_a/shard_c_1717345765.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d18/43d5c76dd22a25a0f9b10e1409241d18/43d5c76dd22a25a0f9b10e1409241d18.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345765.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d18/43d5c76dd22a25a0f9b10e1409241d18/43d5c76dd22a25a0f9b10e1409241d18.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345767.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3c6/8924034c6144363b31008ca15dfa53c6/8924034c6144363b31008ca15dfa53c6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345766.00000, (0, 0) as of 1717345766.00000, shrinking as of 1717345766.00000> to ShardRange<'k' to 't' as of 1717345767.00000, (0, 0) as of 1717345767.00000, sharded as of 1717345767.00000>, path: .shards_a/shard_c_1717345767.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3c6/8924034c6144363b31008ca15dfa53c6/8924034c6144363b31008ca15dfa53c6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345767.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3c6/8924034c6144363b31008ca15dfa53c6/8924034c6144363b31008ca15dfa53c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345769.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8c4/33835cbcfee6686d0aeaf89a6f3c78c4/33835cbcfee6686d0aeaf89a6f3c78c4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345768.00000, (0, 0) as of 1717345768.00000, shrinking as of 1717345768.00000> to ShardRange<'k' to 't' as of 1717345769.00000, (0, 0) as of 1717345769.00000, shrunk as of 1717345769.00000>, path: .shards_a/shard_c_1717345769.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8c4/33835cbcfee6686d0aeaf89a6f3c78c4/33835cbcfee6686d0aeaf89a6f3c78c4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345769.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8c4/33835cbcfee6686d0aeaf89a6f3c78c4/33835cbcfee6686d0aeaf89a6f3c78c4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345771.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/12e/22be859861582f4d7bef8e6604a2312e/22be859861582f4d7bef8e6604a2312e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345770.00000, (0, 0) as of 1717345770.00000, sharding as of 1717345770.00000> to ShardRange<'k' to 't' as of 1717345771.00000, (0, 0) as of 1717345771.00000, found as of 1717345771.00000>, path: .shards_a/shard_c_1717345771.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/12e/22be859861582f4d7bef8e6604a2312e/22be859861582f4d7bef8e6604a2312e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345773.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9ab/aeb1a069b4146a4cc33ae11235afd9ab/aeb1a069b4146a4cc33ae11235afd9ab.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345772.00000, (0, 0) as of 1717345772.00000, sharding as of 1717345772.00000> to ShardRange<'k' to 't' as of 1717345773.00000, (0, 0) as of 1717345773.00000, created as of 1717345773.00000>, path: .shards_a/shard_c_1717345773.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9ab/aeb1a069b4146a4cc33ae11235afd9ab/aeb1a069b4146a4cc33ae11235afd9ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345775.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/be8/3690d3cc6137fab16cf08aa1540adbe8/3690d3cc6137fab16cf08aa1540adbe8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345774.00000, (0, 0) as of 1717345774.00000, sharding as of 1717345774.00000> to ShardRange<'k' to 't' as of 1717345775.00000, (0, 0) as of 1717345775.00000, cleaved as of 1717345775.00000>, path: .shards_a/shard_c_1717345775.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/be8/3690d3cc6137fab16cf08aa1540adbe8/3690d3cc6137fab16cf08aa1540adbe8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345777.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/fa9/df759b1aa40983fae10a0d91fd6cffa9/df759b1aa40983fae10a0d91fd6cffa9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345776.00000, (0, 0) as of 1717345776.00000, sharding as of 1717345776.00000> to ShardRange<'k' to 't' as of 1717345777.00000, (0, 0) as of 1717345777.00000, active as of 1717345777.00000>, path: .shards_a/shard_c_1717345777.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/fa9/df759b1aa40983fae10a0d91fd6cffa9/df759b1aa40983fae10a0d91fd6cffa9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345779.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/c5c/8124bfa613c56e99125b514fa2374c5c/8124bfa613c56e99125b514fa2374c5c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345778.00000, (0, 0) as of 1717345778.00000, sharding as of 1717345778.00000> to ShardRange<'k' to 't' as of 1717345779.00000, (0, 0) as of 1717345779.00000, shrinking as of 1717345779.00000>, path: .shards_a/shard_c_1717345779.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/c5c/8124bfa613c56e99125b514fa2374c5c/8124bfa613c56e99125b514fa2374c5c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345779.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/c5c/8124bfa613c56e99125b514fa2374c5c/8124bfa613c56e99125b514fa2374c5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345781.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/87e/64588e44b42ddce5b3508633c1f8987e/64588e44b42ddce5b3508633c1f8987e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345781.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/87e/64588e44b42ddce5b3508633c1f8987e/64588e44b42ddce5b3508633c1f8987e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345783.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e85/502cac95861c0ed2c8425faa30524e85/502cac95861c0ed2c8425faa30524e85.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345782.00000, (0, 0) as of 1717345782.00000, sharding as of 1717345782.00000> to ShardRange<'k' to 't' as of 1717345783.00000, (0, 0) as of 1717345783.00000, sharded as of 1717345783.00000>, path: .shards_a/shard_c_1717345783.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e85/502cac95861c0ed2c8425faa30524e85/502cac95861c0ed2c8425faa30524e85.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345783.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e85/502cac95861c0ed2c8425faa30524e85/502cac95861c0ed2c8425faa30524e85.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345785.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/4ac/f21672b09c9576bce986b8e4545024ac/f21672b09c9576bce986b8e4545024ac.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345784.00000, (0, 0) as of 1717345784.00000, sharding as of 1717345784.00000> to ShardRange<'k' to 't' as of 1717345785.00000, (0, 0) as of 1717345785.00000, shrunk as of 1717345785.00000>, path: .shards_a/shard_c_1717345785.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/4ac/f21672b09c9576bce986b8e4545024ac/f21672b09c9576bce986b8e4545024ac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345785.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/4ac/f21672b09c9576bce986b8e4545024ac/f21672b09c9576bce986b8e4545024ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345787.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8dc/5ee760aa88b829a420982fd3725f28dc/5ee760aa88b829a420982fd3725f28dc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345786.00000, (0, 0) as of 1717345786.00000, sharded as of 1717345786.00000> to ShardRange<'k' to 't' as of 1717345787.00000, (0, 0) as of 1717345787.00000, found as of 1717345787.00000>, path: .shards_a/shard_c_1717345787.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/8dc/5ee760aa88b829a420982fd3725f28dc/5ee760aa88b829a420982fd3725f28dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345789.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/919/99b99e3e628ff10981d66c01c829e919/99b99e3e628ff10981d66c01c829e919.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345788.00000, (0, 0) as of 1717345788.00000, sharded as of 1717345788.00000> to ShardRange<'k' to 't' as of 1717345789.00000, (0, 0) as of 1717345789.00000, created as of 1717345789.00000>, path: .shards_a/shard_c_1717345789.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/919/99b99e3e628ff10981d66c01c829e919/99b99e3e628ff10981d66c01c829e919.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345791.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/aa9/1ec725e0789ab40dcf886817980d4aa9/1ec725e0789ab40dcf886817980d4aa9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345790.00000, (0, 0) as of 1717345790.00000, sharded as of 1717345790.00000> to ShardRange<'k' to 't' as of 1717345791.00000, (0, 0) as of 1717345791.00000, cleaved as of 1717345791.00000>, path: .shards_a/shard_c_1717345791.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/aa9/1ec725e0789ab40dcf886817980d4aa9/1ec725e0789ab40dcf886817980d4aa9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345793.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d7e/ca156fc9a681fc35ce924eae5416dd7e/ca156fc9a681fc35ce924eae5416dd7e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345792.00000, (0, 0) as of 1717345792.00000, sharded as of 1717345792.00000> to ShardRange<'k' to 't' as of 1717345793.00000, (0, 0) as of 1717345793.00000, active as of 1717345793.00000>, path: .shards_a/shard_c_1717345793.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d7e/ca156fc9a681fc35ce924eae5416dd7e/ca156fc9a681fc35ce924eae5416dd7e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345795.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d9b/59ee8a339d1448af0e6778a9ba895d9b/59ee8a339d1448af0e6778a9ba895d9b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345794.00000, (0, 0) as of 1717345794.00000, sharded as of 1717345794.00000> to ShardRange<'k' to 't' as of 1717345795.00000, (0, 0) as of 1717345795.00000, shrinking as of 1717345795.00000>, path: .shards_a/shard_c_1717345795.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d9b/59ee8a339d1448af0e6778a9ba895d9b/59ee8a339d1448af0e6778a9ba895d9b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345795.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d9b/59ee8a339d1448af0e6778a9ba895d9b/59ee8a339d1448af0e6778a9ba895d9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345797.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e6f/bc7d2eb35dbe4dab94960c603af10e6f/bc7d2eb35dbe4dab94960c603af10e6f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345796.00000, (0, 0) as of 1717345796.00000, sharded as of 1717345796.00000> to ShardRange<'k' to 't' as of 1717345797.00000, (0, 0) as of 1717345797.00000, sharding as of 1717345797.00000>, path: .shards_a/shard_c_1717345797.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e6f/bc7d2eb35dbe4dab94960c603af10e6f/bc7d2eb35dbe4dab94960c603af10e6f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345797.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e6f/bc7d2eb35dbe4dab94960c603af10e6f/bc7d2eb35dbe4dab94960c603af10e6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345799.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/64e/ff9ba4f2ad46e82d5814d713505df64e/ff9ba4f2ad46e82d5814d713505df64e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345799.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/64e/ff9ba4f2ad46e82d5814d713505df64e/ff9ba4f2ad46e82d5814d713505df64e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345801.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d65/67ee26b400812632f5aeda0080051d65/67ee26b400812632f5aeda0080051d65.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345800.00000, (0, 0) as of 1717345800.00000, sharded as of 1717345800.00000> to ShardRange<'k' to 't' as of 1717345801.00000, (0, 0) as of 1717345801.00000, shrunk as of 1717345801.00000>, path: .shards_a/shard_c_1717345801.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d65/67ee26b400812632f5aeda0080051d65/67ee26b400812632f5aeda0080051d65.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345801.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d65/67ee26b400812632f5aeda0080051d65/67ee26b400812632f5aeda0080051d65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345803.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ad8/3d75ff62ab2f4c76ed62d9b301185ad8/3d75ff62ab2f4c76ed62d9b301185ad8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345802.00000, (0, 0) as of 1717345802.00000, shrunk as of 1717345802.00000> to ShardRange<'k' to 't' as of 1717345803.00000, (0, 0) as of 1717345803.00000, found as of 1717345803.00000>, path: .shards_a/shard_c_1717345803.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ad8/3d75ff62ab2f4c76ed62d9b301185ad8/3d75ff62ab2f4c76ed62d9b301185ad8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345805.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/1c7/deb8b58b83b9ac501b937ca086f431c7/deb8b58b83b9ac501b937ca086f431c7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345804.00000, (0, 0) as of 1717345804.00000, shrunk as of 1717345804.00000> to ShardRange<'k' to 't' as of 1717345805.00000, (0, 0) as of 1717345805.00000, created as of 1717345805.00000>, path: .shards_a/shard_c_1717345805.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/1c7/deb8b58b83b9ac501b937ca086f431c7/deb8b58b83b9ac501b937ca086f431c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345807.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d2d/f38e9cf584f2c2749500915685b26d2d/f38e9cf584f2c2749500915685b26d2d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345806.00000, (0, 0) as of 1717345806.00000, shrunk as of 1717345806.00000> to ShardRange<'k' to 't' as of 1717345807.00000, (0, 0) as of 1717345807.00000, cleaved as of 1717345807.00000>, path: .shards_a/shard_c_1717345807.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d2d/f38e9cf584f2c2749500915685b26d2d/f38e9cf584f2c2749500915685b26d2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345809.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f22/fa303da92183f88da80dc58acb3edf22/fa303da92183f88da80dc58acb3edf22.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345808.00000, (0, 0) as of 1717345808.00000, shrunk as of 1717345808.00000> to ShardRange<'k' to 't' as of 1717345809.00000, (0, 0) as of 1717345809.00000, active as of 1717345809.00000>, path: .shards_a/shard_c_1717345809.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f22/fa303da92183f88da80dc58acb3edf22/fa303da92183f88da80dc58acb3edf22.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345811.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/46c/c44df530d6916232e516db46a1bd046c/c44df530d6916232e516db46a1bd046c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345810.00000, (0, 0) as of 1717345810.00000, shrunk as of 1717345810.00000> to ShardRange<'k' to 't' as of 1717345811.00000, (0, 0) as of 1717345811.00000, shrinking as of 1717345811.00000>, path: .shards_a/shard_c_1717345811.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/46c/c44df530d6916232e516db46a1bd046c/c44df530d6916232e516db46a1bd046c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345811.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/46c/c44df530d6916232e516db46a1bd046c/c44df530d6916232e516db46a1bd046c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345813.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/47b/77e1aec3afa7f749ae760cb09637b47b/77e1aec3afa7f749ae760cb09637b47b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345812.00000, (0, 0) as of 1717345812.00000, shrunk as of 1717345812.00000> to ShardRange<'k' to 't' as of 1717345813.00000, (0, 0) as of 1717345813.00000, sharding as of 1717345813.00000>, path: .shards_a/shard_c_1717345813.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/47b/77e1aec3afa7f749ae760cb09637b47b/77e1aec3afa7f749ae760cb09637b47b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345813.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/47b/77e1aec3afa7f749ae760cb09637b47b/77e1aec3afa7f749ae760cb09637b47b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345815.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ec6/aa44c4192f74af86fbf4b89d4e9c6ec6/aa44c4192f74af86fbf4b89d4e9c6ec6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345814.00000, (0, 0) as of 1717345814.00000, shrunk as of 1717345814.00000> to ShardRange<'k' to 't' as of 1717345815.00000, (0, 0) as of 1717345815.00000, sharded as of 1717345815.00000>, path: .shards_a/shard_c_1717345815.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ec6/aa44c4192f74af86fbf4b89d4e9c6ec6/aa44c4192f74af86fbf4b89d4e9c6ec6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345815.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ec6/aa44c4192f74af86fbf4b89d4e9c6ec6/aa44c4192f74af86fbf4b89d4e9c6ec6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345817.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/064/87f2b0fb974e2db54fb079d18d962064/87f2b0fb974e2db54fb079d18d962064.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345817.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/064/87f2b0fb974e2db54fb079d18d962064/87f2b0fb974e2db54fb079d18d962064.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345818.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/0d0/cb1a2e6ba7d08d7b57a4e79ce96ce0d0/cb1a2e6ba7d08d7b57a4e79ce96ce0d0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345820.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ff1/4fa4e71692b2ccc82711077753312ff1/4fa4e71692b2ccc82711077753312ff1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345822.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/5fc/e404d5e913b48a7a1895f2fc497f95fc/e404d5e913b48a7a1895f2fc497f95fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345824.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/0e9/72aa68b621c74cca5fa0bbfb533420e9/72aa68b621c74cca5fa0bbfb533420e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345826.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/934/f7e5f2dffbc7c5336d1baf1e4e77c934/f7e5f2dffbc7c5336d1baf1e4e77c934.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345828.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f0b/6e3b994656a976b69f762d0949c7bf0b/6e3b994656a976b69f762d0949c7bf0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345830.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dbe/cec7bfc38b55dab0877ad75e46048dbe/cec7bfc38b55dab0877ad75e46048dbe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345832.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/02d/d122358e409306a3fe6f9af25f2db02d/d122358e409306a3fe6f9af25f2db02d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345834.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ea8/0ee6894084802bc579d7c7a131790ea8/0ee6894084802bc579d7c7a131790ea8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345836.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d3d/488f7293ab8be28777c305e53a20dd3d/488f7293ab8be28777c305e53a20dd3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345838.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/39b/025cf07a64c122606d0740228028939b/025cf07a64c122606d0740228028939b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345840.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f2d/92ca7185aeabc98aff771545bad6df2d/92ca7185aeabc98aff771545bad6df2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345842.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7dd/e706813afa9a939408c5ede9913617dd/e706813afa9a939408c5ede9913617dd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345844.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ad1/e8f378a279e2244036d8490796ce2ad1/e8f378a279e2244036d8490796ce2ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345846.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/4f9/ca8c0059e8308777396b773435fd44f9/ca8c0059e8308777396b773435fd44f9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345848.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7ab/0b314fa1b29cdeaaf1d3287838e977ab/0b314fa1b29cdeaaf1d3287838e977ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345850.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a62/adb54d02852b5ce25cccd5be334cda62/adb54d02852b5ce25cccd5be334cda62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345852.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e84/90fd57c60d60a860dd1afa48f6204e84/90fd57c60d60a860dd1afa48f6204e84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345854.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/682/47be01186cfc8b8458b25830c5913682/47be01186cfc8b8458b25830c5913682.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345856.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a3d/bf524be0f42fbe7e4178d24d4415da3d/bf524be0f42fbe7e4178d24d4415da3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345858.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/c19/c622b0eda5ac447c224fcbb18f23cc19/c622b0eda5ac447c224fcbb18f23cc19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345860.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/2c9/94f225e50c3d3bc013a762fe2e1022c9/94f225e50c3d3bc013a762fe2e1022c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345862.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3aa/83ba8c04ba9e4613bed069bc99cec3aa/83ba8c04ba9e4613bed069bc99cec3aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345864.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/df1/abb8bbd452b4808949238fd5604bbdf1/abb8bbd452b4808949238fd5604bbdf1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345866.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/1e9/1be9734108b4af2eedab6bb1d0d2d1e9/1be9734108b4af2eedab6bb1d0d2d1e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345868.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/c09/1d2c7000cd326eb3132a57c063003c09/1d2c7000cd326eb3132a57c063003c09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345870.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/126/fc693c4d591f24e9cccd8bb2f788a126/fc693c4d591f24e9cccd8bb2f788a126.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345872.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/89b/4f1b4a6eb2c50ad6d3f1d02ac744589b/4f1b4a6eb2c50ad6d3f1d02ac744589b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345874.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/4ac/8a1d5e4c6eaa51a9619f2502526604ac/8a1d5e4c6eaa51a9619f2502526604ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345876.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/a25/2a521685dab2915d8f94d1957aa16a25/2a521685dab2915d8f94d1957aa16a25.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345878.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/406/81d33b357b344e611370a652952a3406/81d33b357b344e611370a652952a3406.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345880.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/2af/3a80c3e331948006f086b14984fe32af/3a80c3e331948006f086b14984fe32af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345882.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/680/df66e138663dd1bbdb7658e62dc33680/df66e138663dd1bbdb7658e62dc33680.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345882.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/680/df66e138663dd1bbdb7658e62dc33680/df66e138663dd1bbdb7658e62dc33680.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345884.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/684/756e24fa87c19fdf1d00b679f03bc684/756e24fa87c19fdf1d00b679f03bc684.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345884.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/684/756e24fa87c19fdf1d00b679f03bc684/756e24fa87c19fdf1d00b679f03bc684.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345886.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/acb/03ada642bfb4f80457e93a9dcac95acb/03ada642bfb4f80457e93a9dcac95acb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345886.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/acb/03ada642bfb4f80457e93a9dcac95acb/03ada642bfb4f80457e93a9dcac95acb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345888.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dd3/cd6cb810de80d32dba8e2e11916e3dd3/cd6cb810de80d32dba8e2e11916e3dd3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345888.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dd3/cd6cb810de80d32dba8e2e11916e3dd3/cd6cb810de80d32dba8e2e11916e3dd3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345890.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dcf/4d394e0464b28793c92b4319561efdcf/4d394e0464b28793c92b4319561efdcf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345890.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dcf/4d394e0464b28793c92b4319561efdcf/4d394e0464b28793c92b4319561efdcf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345892.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d5c/236937d7a078dfc6eba9335a7678dd5c/236937d7a078dfc6eba9335a7678dd5c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345892.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d5c/236937d7a078dfc6eba9335a7678dd5c/236937d7a078dfc6eba9335a7678dd5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345894.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/89e/8ea2815e930818fd0abd1b759ba5689e/8ea2815e930818fd0abd1b759ba5689e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345894.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/89e/8ea2815e930818fd0abd1b759ba5689e/8ea2815e930818fd0abd1b759ba5689e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345896.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/b2e/fb3aacab15f1d8d6630a2add16efbb2e/fb3aacab15f1d8d6630a2add16efbb2e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345896.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/b2e/fb3aacab15f1d8d6630a2add16efbb2e/fb3aacab15f1d8d6630a2add16efbb2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345898.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3e7/36526bdf73e4c16a307256de414c43e7/36526bdf73e4c16a307256de414c43e7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345898.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/3e7/36526bdf73e4c16a307256de414c43e7/36526bdf73e4c16a307256de414c43e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345900.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7ae/ba9bed263516a49ff367b989197117ae/ba9bed263516a49ff367b989197117ae.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345900.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/7ae/ba9bed263516a49ff367b989197117ae/ba9bed263516a49ff367b989197117ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345902.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ae3/32cbe17e152b8469da67255fe65ddae3/32cbe17e152b8469da67255fe65ddae3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345902.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/ae3/32cbe17e152b8469da67255fe65ddae3/32cbe17e152b8469da67255fe65ddae3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345904.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/770/dc730df607feec4ef35121e646165770/dc730df607feec4ef35121e646165770.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345904.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/770/dc730df607feec4ef35121e646165770/dc730df607feec4ef35121e646165770.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345906.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/b53/ae4e2beee7b9130beab5cea7d121bb53/ae4e2beee7b9130beab5cea7d121bb53.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345906.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/b53/ae4e2beee7b9130beab5cea7d121bb53/ae4e2beee7b9130beab5cea7d121bb53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345908.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/57d/eb5ba4c07c6e20f9b7676e1113b8257d/eb5ba4c07c6e20f9b7676e1113b8257d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345908.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/57d/eb5ba4c07c6e20f9b7676e1113b8257d/eb5ba4c07c6e20f9b7676e1113b8257d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345910.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/045/ee379e78a3565907c6b7ede9859c0045/ee379e78a3565907c6b7ede9859c0045.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345910.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/045/ee379e78a3565907c6b7ede9859c0045/ee379e78a3565907c6b7ede9859c0045.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345912.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/72a/0d0bb69f9b6c62830241362e56a1572a/0d0bb69f9b6c62830241362e56a1572a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345912.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/72a/0d0bb69f9b6c62830241362e56a1572a/0d0bb69f9b6c62830241362e56a1572a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345914.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9bb/4b7bec0c0322986bd1caefdbb09769bb/4b7bec0c0322986bd1caefdbb09769bb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345914.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/9bb/4b7bec0c0322986bd1caefdbb09769bb/4b7bec0c0322986bd1caefdbb09769bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345916.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d32/573801385c65cb523b77845b8d39ed32/573801385c65cb523b77845b8d39ed32.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345916.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d32/573801385c65cb523b77845b8d39ed32/573801385c65cb523b77845b8d39ed32.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345918.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/07e/4194580d57db5c3e2b6f68186086607e/4194580d57db5c3e2b6f68186086607e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345918.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/07e/4194580d57db5c3e2b6f68186086607e/4194580d57db5c3e2b6f68186086607e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345920.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/146/b43f57a19d8127db35bd62a31bea9146/b43f57a19d8127db35bd62a31bea9146.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345920.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/146/b43f57a19d8127db35bd62a31bea9146/b43f57a19d8127db35bd62a31bea9146.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml ... ok test DEBUG: /tmp/tmpu0uaaa_c/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.97547.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpu0uaaa_c/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345711.97547.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '75f9d33e533eda437abf5b75eb3ce3c4', '40f8827e-648b-4971-b5b9-bd721525ab85-', '1717345719.36520', '1717345719.36171', '0', '', '1717345719.36171', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('40f8827e-648b-4971-b5b9-bd721525ab85-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.39266.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '75f9d33e533eda437abf5b75eb3ce3c4', '40f8827e-648b-4971-b5b9-bd721525ab85-', '1717345719.36520', '1717345719.36171', '0', '', '1717345719.36171', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345719.37207', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345719.37207', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345719.37207', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345719.39309', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345719.39309', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345719.39266', 'epoch': '1717345719.39266', 'reported': 0, 'tombstones': -1}], '40f8827e-648b-4971-b5b9-bd721525ab85-') test DEBUG: /tmp/tmpx9uml1u9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.39266.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpx9uml1u9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.39266.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4ed89c0f391e8c85de106b23d0c17aa2', '08b35512-ebc0-41e9-a530-019b360c6eba-', '1717345719.52878', '1717345719.52392', '0', '', '1717345719.52392', 3, 0) test WARNING: /tmp/tmpy34v85oq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.56909.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmpy34v85oq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.56909.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4ed89c0f391e8c85de106b23d0c17aa2', '08b35512-ebc0-41e9-a530-019b360c6eba-', '1717345719.52878', '1717345719.68240', '0', '', '1717345719.52392', 3, 0) test WARNING: /tmp/tmpy34v85oq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.56909.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmpy34v85oq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.56909.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4ed89c0f391e8c85de106b23d0c17aa2', '49fc2090-58ed-4a4f-bc38-1f0869c79189-', '1717345719.74932', '1717345719.74133', '0', '', '1717345719.74133', 3, 0) test WARNING: /tmp/tmplvcr220k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.76426.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmplvcr220k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.76426.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4ed89c0f391e8c85de106b23d0c17aa2', '49fc2090-58ed-4a4f-bc38-1f0869c79189-', '1717345719.74932', '1717345719.83520', '0', '', '1717345719.74133', 3, 0) test WARNING: /tmp/tmplvcr220k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.76426.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmplvcr220k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.76426.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, 'de97933f24a29accb4bfffc7791d745d', 'fcfdf39b-492d-4525-8664-022dfd44969a-', '1717345719.89832', '1717345719.89315', '0', '', '1717345719.89315', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345719.97156', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345719.97156', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345719.97156', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345719.99134', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345719.99134', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345719.91149', 'epoch': '1717345719.91149', 'reported': 0, 'tombstones': -1}], 'fcfdf39b-492d-4525-8664-022dfd44969a-') test DEBUG: /tmp/tmptq8fk54p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.91149.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmptq8fk54p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.91149.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, 'de97933f24a29accb4bfffc7791d745d', 'fcfdf39b-492d-4525-8664-022dfd44969a-', '1717345719.89832', '1717345719.96009', '0', '', '1717345719.89315', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345719.97156', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345719.97156', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345719.97156', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345719.99134', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345719.99134', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345719.91149', 'epoch': '1717345719.91149', 'reported': 0, 'tombstones': -1}], 'fcfdf39b-492d-4525-8664-022dfd44969a-') test DEBUG: /tmp/tmptq8fk54p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.91149.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmptq8fk54p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345719.91149.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, 'eed34530d6463256ad505b4bd147902b', '7bd6abeb-f7aa-4ef5-bce2-df8ef6517d88-', '1717345720.10590', '1717345720.10231', '0', '', '1717345720.10231', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range ... ok Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 3.14 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 6.28 Search value matched 0 devices. The on-disk ring builder is unchanged. d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_unknown test.unit.cli.test_ringbuilder.TestCommands.test_unknown ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key ... ok test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Setting object count to 1058 (txn: txc60ed4b33ef241759f90b-00665c9dd5) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Setting object count to 1024 (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Setting object count to 1024 (txn: tx82a01695d659402cb3763-00665c9dd8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Setting object count to 1072 (txn: txdb6b7d7a6a3d42c69f5bc-00665c9ddb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx74e0737bccb441deae357-00665c9dde) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Setting object count to 1072 (txn: tx25954291ad544cbda6670-00665c9dcb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx45b606bffdca4ec2b80a3-00665c9dcd) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx45b606bffdca4ec2b80a3-00665c9dcd) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx45b606bffdca4ec2b80a3-00665c9dcd) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx45b606bffdca4ec2b80a3-00665c9dcd) test ERROR: Aborting listing from shards due to bad response: [404] (txn: tx45b606bffdca4ec2b80a3-00665c9dcd) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx19c8c9fbd84044f3aa225-00665c9dd0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: tx9469234bb9914b55bb586-00665c9dd3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test DEBUG: Setting object count to 3120 (txn: tx2a03cd86e5a0435ea313f-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx7a338887ec7243fb80cee-00665c9dd6) test DEBUG: Setting object count to 3120 (txn: tx7a338887ec7243fb80cee-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txc880bb053e8f46d5b638a-00665c9dd6) test DEBUG: Setting object count to 3120 (txn: txc880bb053e8f46d5b638a-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txf755417d700e40c5b8bdc-00665c9dd6) test DEBUG: Setting object count to 3120 (txn: txf755417d700e40c5b8bdc-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx7b68f76b29974bb382902-00665c9dd6) test DEBUG: Setting object count to 3120 (txn: tx7b68f76b29974bb382902-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: txb40f5cabf38945b3a5f1b-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx0c2ec30c102243bd897cc-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user ... ok proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0537f2dc71524520aeba6-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345720.17702', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345720.17702', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345720.17702', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345720.18496', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345720.18496', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345720.11981', 'epoch': '1717345720.11981', 'reported': 0, 'tombstones': -1}], '7bd6abeb-f7aa-4ef5-bce2-df8ef6517d88-') test DEBUG: /tmp/tmp6td4uy1b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345720.11981.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp6td4uy1b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345720.11981.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, 'eed34530d6463256ad505b4bd147902b', '7bd6abeb-f7aa-4ef5-bce2-df8ef6517d88-', '1717345720.10590', '1717345720.16737', '0', '', '1717345720.10231', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345720.17702', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345720.17702', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345720.17702', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345720.18496', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345720.18496', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345720.11981', 'epoch': '1717345720.11981', 'reported': 0, 'tombstones': -1}], '7bd6abeb-f7aa-4ef5-bce2-df8ef6517d88-') test DEBUG: /tmp/tmp6td4uy1b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345720.11981.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp6td4uy1b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345720.11981.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'b1035ad3502b59f50f7cd00cbc5776fa', '6b0af64f-27d6-464d-b252-7f05b7b22eba-', '1717345725.67705', '1717345725.67243', '0', '', '1717345725.67243', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345725.68571', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.68571', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345725.68571', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345725.69182', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.69182', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345725.69157', 'epoch': '1717345725.69157', 'reported': 0, 'tombstones': -1}], '6b0af64f-27d6-464d-b252-7f05b7b22eba-') test DEBUG: /tmp/tmpkf1d13lb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.69157.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpkf1d13lb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.69157.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'b1035ad3502b59f50f7cd00cbc5776fa', '6b0af64f-27d6-464d-b252-7f05b7b22eba-', '1717345725.67705', '1717345725.73553', '0', '', '1717345725.67243', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345725.68571', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.68571', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345725.68571', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345725.69182', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.69182', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345725.69157', 'epoch': '1717345725.69157', 'reported': 0, 'tombstones': -1}], '6b0af64f-27d6-464d-b252-7f05b7b22eba-') test DEBUG: /tmp/tmpkf1d13lb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.69157.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpkf1d13lb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.69157.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'b1035ad3502b59f50f7cd00cbc5776fa', 'f2936f42-92be-476f-bb7b-a6cd7ea99db1-', '1717345725.81824', '1717345725.81354', '0', '', '1717345725.81354', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345725.82709', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.82709', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345725.82709', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345725.83308', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.83308', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345725.83277', 'epoch': '1717345725.83277', 'reported': 0, 'tombstones': -1}], 'f2936f42-92be-476f-bb7b-a6cd7ea99db1-') test DEBUG: /tmp/tmp8f5d60ih/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.83277.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp8f5d60ih/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.83277.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'b1035ad3502b59f50f7cd00cbc5776fa', 'f2936f42-92be-476f-bb7b-a6cd7ea99db1-', '1717345725.81824', '1717345725.87689', '0', '', '1717345725.81354', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345725.82709', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.82709', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345725.82709', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345725.83308', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345725.83308', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345725.83277', 'epoch': '1717345725.83277', 'reported': 0, 'tombstones': -1}], 'f2936f42-92be-476f-bb7b-a6cd7ea99db1-') test DEBUG: /tmp/tmp8f5d60ih/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.83277.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp8f5d60ih/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717345725.83277.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'fc2f2cde1d5d06e995f427340ef3a556', '22cc476d-0491-4bc9-a305-c2c605b529f0-sda', '1717345725.97905', '1717345725.97407', '0', '', '1717345725.97407', 1, 0) test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 501 Not Implemented test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 220, in PUT self.req.get_acl_response(app, 'PUT', self.container) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server ERROR: Object returning 503 for [] (txn: tx97a6075e7a2c4de8bb219-00665c9dba) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.8:1008/sdi (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.7:1007/sdh (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.1:1001/sdb (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.13:1013/sdn (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx5caaa35ec6e34713b0452-00665c9dba) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 Trying to GET /a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txccf4736fa5494022a52ad-00665c9dba) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf2a2370d6ad84540ba981-00665c9dba) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Ignoring 412 from primary (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match ... ok test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Setting object count to 1072 (txn: tx74e0737bccb441deae357-00665c9dde) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Setting object count to 1072 (txn: txad5f474a6ae94b67b608f-00665c9de1) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: txcbd8049f46ad48cb91df3-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx8f8ef49e46834dd2b6e1a-00665c9dd6) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx8f8ef49e46834dd2b6e1a-00665c9dd6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8f8ef49e46834dd2b6e1a-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8f8ef49e46834dd2b6e1a-00665c9dd6) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx8f8ef49e46834dd2b6e1a-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx906afade315d4dcb97958-00665c9dd6) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx906afade315d4dcb97958-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx906afade315d4dcb97958-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx906afade315d4dcb97958-00665c9dd6) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx906afade315d4dcb97958-00665c9dd6) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx13e541b328d04c98a50f0-00665c9dd6) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx13e541b328d04c98a50f0-00665c9dd6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx13e541b328d04c98a50f0-00665c9dd6) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx13e541b328d04c98a50f0-00665c9dd6) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx13e541b328d04c98a50f0-00665c9dd6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx5b44b0755254498691a65-00665c9dd9) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx5b44b0755254498691a65-00665c9dd9) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: tx5b44b0755254498691a65-00665c9dd9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf93ddd28cd444096ae698-00665c9ddc) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('22cc476d-0491-4bc9-a305-c2c605b529f0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '6f6e25869e4afacad404601d9e12f3d8', '2655298d-07ce-47b6-98f3-76d6165458f0-sda', '1717345735.80911', '1717345735.80483', '0', '', '1717345735.80483', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345735.83973', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345735.83973', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345735.83973', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345735.84618', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345735.84618', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345735.84586', 'epoch': '1717345735.84586', 'reported': 0, 'tombstones': -1}], '2655298d-07ce-47b6-98f3-76d6165458f0-sda') test DEBUG: /tmp/tmp5jiw_pja/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp5jiw_pja/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '6f6e25869e4afacad404601d9e12f3d8', '2655298d-07ce-47b6-98f3-76d6165458f0-sda', '1717345735.80911', '1717345735.82649', '0', '', '1717345735.80483', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345735.83973', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345735.83973', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345735.83973', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345735.84618', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345735.84618', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345735.84586', 'epoch': '1717345735.84586', 'reported': 0, 'tombstones': -1}], '2655298d-07ce-47b6-98f3-76d6165458f0-sda') test DEBUG: /tmp/tmp5jiw_pja/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp5jiw_pja/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '6f6e25869e4afacad404601d9e12f3d8', '3b3dd50e-726b-4cb7-934f-5c2d51ad95f9-sda', '1717345735.95093', '1717345735.94719', '0', '', '1717345735.94719', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345735.97824', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345735.97824', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345735.97824', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345735.98363', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345735.98363', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345735.98335', 'epoch': '1717345735.98335', 'reported': 0, 'tombstones': -1}], '3b3dd50e-726b-4cb7-934f-5c2d51ad95f9-sda') test DEBUG: /tmp/tmpl4dl399y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpl4dl399y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '6f6e25869e4afacad404601d9e12f3d8', '3b3dd50e-726b-4cb7-934f-5c2d51ad95f9-sda', '1717345735.95093', '1717345735.96667', '0', '', '1717345735.94719', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717345735.97824', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717345735.97824', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345735.97824', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345735.98363', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345735.98363', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345735.98335', 'epoch': '1717345735.98335', 'reported': 0, 'tombstones': -1}], '3b3dd50e-726b-4cb7-934f-5c2d51ad95f9-sda') test DEBUG: /tmp/tmpl4dl399y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpl4dl399y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'd67cdd00832fd514e214be4e14c5b605', '1a4020c6-1f95-4ffa-a1ac-00b35ba1b14d-sda', '1717345736.08548', '1717345736.08185', '0', '', '1717345736.08185', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('1a4020c6-1f95-4ffa-a1ac-00b35ba1b14d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmpyxt2laq2/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'd67cdd00832fd514e214be4e14c5b605', 'cee5e669-6507-402d-a1eb-96bd9c06c111-sda', '1717345736.20355', '1717345736.19999', '0', '', '1717345736.19999', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('cee5e669-6507-402d-a1eb-96bd9c06c111-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'd67cdd00832fd514e214be4e14c5b605', 'c2051731-c361-42b5-bfc5-7d51dc6998f5-sda', '1717345736.34351', '1717345736.33961', '0', '', '1717345736.33961', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('c2051731-c361-42b5-bfc5-7d51dc6998f5-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmpsazvkqfx/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'd67cdd00832fd514e214be4e14c5b605', '686beadc-e823-4613-8fcd-c507f062ce43-sda', '1717345736.43469', '1717345736.43117', '0', '', '1717345736.43117', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('686beadc-e823-4613-8fcd-c507f062ce43-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b026ccb2-20aa-4829-b583-de79b9086a2e-sda', '1717345736.58519', '1717345736.58163', '0', '', '1717345736.58163', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp8u797ttb/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3cccd61e-2952-4fc4-a277-ef1f6e980acc-sdb', '1717345736.66529', '1717345737.00000', '0', '', '1717345737.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3cccd61e-2952-4fc4-a277-ef1f6e980acc-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. Unknown command: unknown swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate test.unit.cli.test_ringbuilder.TestCommands.test_validate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder test.unit.cli.test_ringbuilder.TestCommands.test_write_builder ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_ring ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value ... ok proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6badba587aaa46e4ac14e-00665c9db0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error ... ok test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txf93ddd28cd444096ae698-00665c9ddc) test DEBUG: Setting object count to 1065 (txn: txf93ddd28cd444096ae698-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx51210f426c9a42109c9dc-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx19d1aa74fac14659961a2-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/ascii/ascii From Object Server 10.0.0.2:1002/sdc (txn: txc8e69d5f9513460cb6658-00665c9ddc) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/툎/鐉 From Object Server 10.0.0.2:1002/sdc (txn: txa4081ffe6de44900a0ac7-00665c9ddc) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx386db2b97f2b482e9e01b-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx6ad3d0ac011f4ed8a319c-00665c9ddc) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx6ad3d0ac011f4ed8a319c-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txa5b7a9f73f6c4f4ca951a-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txa5b7a9f73f6c4f4ca951a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/ascii/ascii: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: txa5b7a9f73f6c4f4ca951a-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txf9d5a853800f407fae44f-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txf9d5a853800f407fae44f-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/툎/鐉: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: txf9d5a853800f407fae44f-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx25df06e83d254d6eb4d1a-00665c9ddc) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range ... ok REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3cccd61e-2952-4fc4-a277-ef1f6e980acc-sdb', '1717345736.66529', '1717345737.00000', '0', '', '1717345737.00000', 0, 0) test ERROR: Quarantining DB /tmp/tmp8u797ttb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3cccd61e-2952-4fc4-a277-ef1f6e980acc-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.66746 seconds (0.37489/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwv_1z7ql/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '85f1cbfe-d149-4a18-b69c-255a3fb18396-sda', '1717345742.55629', '1717345742', '0', '', '1717345742', 0, 0) test DEBUG: /tmp/tmpwv_1z7ql/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '85f1cbfe-d149-4a18-b69c-255a3fb18396-sda', '1717345742.55629', '1717345742', '0', '', '1717345742', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('85f1cbfe-d149-4a18-b69c-255a3fb18396-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04317 seconds (23.16266/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpsyjncorw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '658ae5a6d204b6324eebf2081185eb9b', '04b80099-d8bc-4212-b24c-e517e82650e7-sda', '1717345742.63641', '1717345742', '0', '', '1717345742', 1, 0) test DEBUG: /tmp/tmpsyjncorw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345744.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '04b80099-d8bc-4212-b24c-e517e82650e7-sda') test DEBUG: /tmp/tmpsyjncorw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0111678s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '658ae5a6d204b6324eebf2081185eb9b', '04b80099-d8bc-4212-b24c-e517e82650e7-sda', '1717345742.63641', '1717345742', '0', '', '1717345742', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04b80099-d8bc-4212-b24c-e517e82650e7-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06668 seconds (14.99806/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpn8032vhp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ad0bde68-827a-42b3-8b56-6f83fdb0e8c9-sda', '1717345742.74248', '1717345742', '0', '', '1717345742', 0, 0) test DEBUG: /tmp/tmpn8032vhp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ad0bde68-827a-42b3-8b56-6f83fdb0e8c9-sda', '1717345742.74248', '1717345742', '0', '', '1717345742', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ad0bde68-827a-42b3-8b56-6f83fdb0e8c9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05097 seconds (19.61858/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp05xmk9rf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '658ae5a6d204b6324eebf2081185eb9b', 'b7350374-ef0c-465b-871f-5b939be9b3a8-sda', '1717345742.82955', '1717345742', '0', '', '1717345742', 1, 0) test DEBUG: /tmp/tmp05xmk9rf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '658ae5a6d204b6324eebf2081185eb9b', 'b7350374-ef0c-465b-871f-5b939be9b3a8-sda', '1717345742.82955', '1717345742', '0', '', '1717345742', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b7350374-ef0c-465b-871f-5b939be9b3a8-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06404 seconds (15.61622/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp74xbzajz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ef667a71-da43-40f7-9c7c-10407b56bb70-sda', '1717345742.93303', '1717345742', '0', '', '1717345742', 0, 0) test DEBUG: /tmp/tmp74xbzajz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ef667a71-da43-40f7-9c7c-10407b56bb70-sda', '1717345742.93303', '1717345742', '0', '', '1717345742', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ef667a71-da43-40f7-9c7c-10407b56bb70-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04180 seconds (23.92174/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpufnvd635/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e7e8f5a3e89d809b955709b858fc4141', 'ad6fd1a7-f4b6-439e-8a37-164fc5d8d401-sda', '1717345743.01486', '1717345743', '0', '', '1717345743', 1, 1) test DEBUG: /tmp/tmpufnvd635/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345746.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'ad6fd1a7-f4b6-439e-8a37-164fc5d8d401-sda') test DEBUG: /tmp/tmpufnvd635/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0115056s) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete ... ok test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Setting object count to 1072 (txn: tx8ae07379342d4651ae310-00665c9de4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx886ccfd9d95d445bb6783-00665c9de7) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx886ccfd9d95d445bb6783-00665c9de7) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx886ccfd9d95d445bb6783-00665c9de7) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx886ccfd9d95d445bb6783-00665c9de7) test ERROR: Aborting listing from shards due to bad response: [404] (txn: tx886ccfd9d95d445bb6783-00665c9de7) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: txba6316256e57499cad7c8-00665c9dea) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx94f87a791af14f50b92f0-00665c9ded) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbdac0c06a6ba440f91028-00665c9dbf) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txa9a19130be3046ffb4083-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txa9a19130be3046ffb4083-00665c9ddc) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txa9a19130be3046ffb4083-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: txa9a19130be3046ffb4083-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: my-txn-id) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx5daa008d6b584de89e311-00665c9ddc) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx5daa008d6b584de89e311-00665c9ddc) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx5daa008d6b584de89e311-00665c9ddc) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx5daa008d6b584de89e311-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx2d130e68d5e143e2a89e2-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txc3c14525657a4b91b0871-00665c9ddc) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e7e8f5a3e89d809b955709b858fc4141', 'ad6fd1a7-f4b6-439e-8a37-164fc5d8d401-sda', '1717345743.01486', '1717345743', '0', '', '1717345743', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ad6fd1a7-f4b6-439e-8a37-164fc5d8d401-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 3.51504 seconds (0.28449/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpf5jdx4k6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd6c6804f-68aa-47a8-a636-8b32ce5b8e52-sda', '1717345746.56934', '1717345746', '0', '', '1717345746', 0, 0) test DEBUG: /tmp/tmpf5jdx4k6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd6c6804f-68aa-47a8-a636-8b32ce5b8e52-sda', '1717345746.56934', '1717345746', '0', '', '1717345746', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d6c6804f-68aa-47a8-a636-8b32ce5b8e52-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04207 seconds (23.77260/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpmayogu78/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'cd7b3a059c7c4c1ca62929375e956b71', '6fe42200-f889-4d15-8b11-6e53af6dba8b-sda', '1717345746.64931', '1717345746', '0', '', '1717345746', 1, 1) test DEBUG: /tmp/tmpmayogu78/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'cd7b3a059c7c4c1ca62929375e956b71', '6fe42200-f889-4d15-8b11-6e53af6dba8b-sda', '1717345746.64931', '1717345746', '0', '', '1717345746', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('6fe42200-f889-4d15-8b11-6e53af6dba8b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06106 seconds (16.37822/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkmuy0ml0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '6e12ed62-894d-4070-bcb6-09d2b3331ba4-sda', '1717345750.59305', '1717345752', '0', '', '1717345752', 0, 1) test DEBUG: /tmp/tmpkmuy0ml0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '6e12ed62-894d-4070-bcb6-09d2b3331ba4-sda', '1717345750.59305', '1717345752', '0', '', '1717345752', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('6e12ed62-894d-4070-bcb6-09d2b3331ba4-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03077 seconds (32.49700/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp25k72mey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '864652727089ce1d575ae3863e0140b5', '65de821d-a389-4744-a50d-7b849850bd10-sda', '1717345750.65354', '1717345752', '0', '', '1717345752', 1, 0) test DEBUG: /tmp/tmp25k72mey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345753.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '65de821d-a389-4744-a50d-7b849850bd10-sda') test DEBUG: /tmp/tmp25k72mey/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00736809s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '864652727089ce1d575ae3863e0140b5', '65de821d-a389-4744-a50d-7b849850bd10-sda', '1717345750.65354', '1717345752', '0', '', '1717345752', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('65de821d-a389-4744-a50d-7b849850bd10-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06607 seconds (15.13569/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpu9y0gfmp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '864652727089ce1d575ae3863e0140b5', '748524db-232c-4d66-a4ba-d23af827b2d3-sda', '1717345750.73964', '1717345750', '1717345751', '', '1717345751', 1, 0) test DEBUG: /tmp/tmpu9y0gfmp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345753.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '748524db-232c-4d66-a4ba-d23af827b2d3-sda') test DEBUG: /tmp/tmpu9y0gfmp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00854993s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '864652727089ce1d575ae3863e0140b5', '748524db-232c-4d66-a4ba-d23af827b2d3-sda', '1717345750.73964', '1717345750', '1717345751', '', '1717345751', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('748524db-232c-4d66-a4ba-d23af827b2d3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06070 seconds (16.47530/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptg_y91qp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '864652727089ce1d575ae3863e0140b5', 'ad8ac318-c991-44c3-a3aa-b0719c56ed94-sda', '1717345750.83426', '1717345750', '1717345751', '', '1717345751', 1, 1) test DEBUG: /tmp/tmptg_y91qp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 400 Bad Request test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error ... ok test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: tx94f87a791af14f50b92f0-00665c9ded) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test DEBUG: Setting object count to 3120 (txn: tx6cb7d4ded5ae4e5f88780-00665c9df0) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test DEBUG: Setting object count to 3120 (txn: tx8dd895ad6cf84a98b3a86-00665c9df0) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx86a56925eab74a45ab2e7-00665c9df1) test DEBUG: Setting object count to 3120 (txn: tx86a56925eab74a45ab2e7-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txbde960126c044002a967e-00665c9df1) test DEBUG: Setting object count to 3120 (txn: txbde960126c044002a967e-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test DEBUG: Setting object count to 3120 (txn: txaacd9f447f7c4bf5bdcea-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txd43785f637ff491c86b2c-00665c9df1) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: txd43785f637ff491c86b2c-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx43562a0bbc634e8192f23-00665c9df1) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx43562a0bbc634e8192f23-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx43562a0bbc634e8192f23-00665c9df1) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txc3c14525657a4b91b0871-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: txc3c14525657a4b91b0871-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx3d5492a066e8437f868e3-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx3d5492a066e8437f868e3-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx42bafd1d4695404dadbb9-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6c407a63d097475388887-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: tx6c407a63d097475388887-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx40aff0b1ce1d4848abe7b-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: tx40aff0b1ce1d4848abe7b-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txda0abbb405814c378a679-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txda0abbb405814c378a679-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe3d92f7c4a41426aa613b-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txe3d92f7c4a41426aa613b-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: txe3d92f7c4a41426aa613b-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txfa43af3a11fb44489bc83-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txfa43af3a11fb44489bc83-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx1bd0697b1e6544639b6a5-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txcb4716fb09ef451182839-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txcb4716fb09ef451182839-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx20ac66d8903c4e609e1de-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx20ac66d8903c4e609e1de-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: tx20ac66d8903c4e609e1de-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx19cc321d933c438ba00cd-00665c9ddc) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:28:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '864652727089ce1d575ae3863e0140b5', 'ad8ac318-c991-44c3-a3aa-b0719c56ed94-sda', '1717345750.83426', '1717345750', '1717345751', '', '1717345751', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ad8ac318-c991-44c3-a3aa-b0719c56ed94-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.83382 seconds (0.35288/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp24wx2402/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '557f9a1c-821f-4126-a3de-b6d538b8099d-sda', '1717345756.88094', '1717345758', '1717345757', '', '1717345758', 0, 1) test DEBUG: /tmp/tmp24wx2402/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '557f9a1c-821f-4126-a3de-b6d538b8099d-sda', '1717345756.88094', '1717345758', '1717345757', '', '1717345758', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('557f9a1c-821f-4126-a3de-b6d538b8099d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04235 seconds (23.61187/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpelb6i2v8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6a62786f63de2aba967e3adf3c72127a', 'cee0bff3-d48d-4764-b387-10c0dcf33f14-sda', '1717345756.96484', '1717345758', '1717345757', '', '1717345758', 1, 0) test DEBUG: /tmp/tmpelb6i2v8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345760.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'cee0bff3-d48d-4764-b387-10c0dcf33f14-sda') test DEBUG: /tmp/tmpelb6i2v8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00725532s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6a62786f63de2aba967e3adf3c72127a', 'cee0bff3-d48d-4764-b387-10c0dcf33f14-sda', '1717345756.96484', '1717345758', '1717345757', '', '1717345758', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('cee0bff3-d48d-4764-b387-10c0dcf33f14-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06190 seconds (16.15422/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpo9fu2r8q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e2d5c382-d898-4229-ab6e-b17b9b4db22b-sda', '1717345757.07110', '1717345759', '1717345758', '', '1717345759', 0, 0) test DEBUG: /tmp/tmpo9fu2r8q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e2d5c382-d898-4229-ab6e-b17b9b4db22b-sda', '1717345757.07110', '1717345759', '1717345758', '', '1717345759', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e2d5c382-d898-4229-ab6e-b17b9b4db22b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04480 seconds (22.32171/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpa406866w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'cac7fecf00e1b054c817eb77fdb1d7ad', '558955b2-75d2-4a8b-b31e-39837d6f6f46-sda', '1717345757.15734', '1717345759', '1717345758', '', '1717345759', 1, 0) test DEBUG: /tmp/tmpa406866w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'cac7fecf00e1b054c817eb77fdb1d7ad', '558955b2-75d2-4a8b-b31e-39837d6f6f46-sda', '1717345757.15734', '1717345759', '1717345758', '', '1717345759', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('558955b2-75d2-4a8b-b31e-39837d6f6f46-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05544 seconds (18.03735/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5aiivv4h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4c80d270-4257-4d3a-826d-d1ad02827853-sda', '1717345757.26118', '1717345760', '1717345759', '', '1717345760', 0, 1) test DEBUG: /tmp/tmp5aiivv4h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4c80d270-4257-4d3a-826d-d1ad02827853-sda', '1717345757.26118', '1717345760', '1717345759', '', '1717345760', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('4c80d270-4257-4d3a-826d-d1ad02827853-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03783 seconds (26.43290/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp77659cvd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9112e62bb189207f780238a301e8fcad', '542334f7-898f-4cfb-b8c9-78e6119a70c4-sda', '1717345757.34625', '1717345760', '1717345759', '', '1717345760', 1, 1) test DEBUG: /tmp/tmp77659cvd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345762.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '542334f7-898f-4cfb-b8c9-78e6119a70c4-sda') test DEBUG: /tmp/tmp77659cvd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00903106s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'Content-Md5': '0psj556+oeNaquEdDeMxig==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd29b23e79ebea1e35aaae11d0de3318a', 'Content-Length': '226', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission ... ok test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx43562a0bbc634e8192f23-00665c9df1) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx43562a0bbc634e8192f23-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx43562a0bbc634e8192f23-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx43562a0bbc634e8192f23-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx43562a0bbc634e8192f23-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx31856e8c13794e4d8b631-00665c9df1) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx31856e8c13794e4d8b631-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txfedac60745794540aeebd-00665c9df1) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txfedac60745794540aeebd-00665c9df1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfedac60745794540aeebd-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfedac60745794540aeebd-00665c9df1) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txfedac60745794540aeebd-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx715dc3ec290a4a9d92644-00665c9df1) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx715dc3ec290a4a9d92644-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx715dc3ec290a4a9d92644-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx715dc3ec290a4a9d92644-00665c9df1) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx715dc3ec290a4a9d92644-00665c9df1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txa4b723882d3f41acb7ea2-00665c9df1) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: txa4b723882d3f41acb7ea2-00665c9df1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txa4b723882d3f41acb7ea2-00665c9df1) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: txa4b723882d3f41acb7ea2-00665c9df1) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txa4b723882d3f41acb7ea2-00665c9df1) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txa250a027b1084f97a6ec0-00665c9df4) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txa250a027b1084f97a6ec0-00665c9df4) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: txa250a027b1084f97a6ec0-00665c9df4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa23860c2fb734513bfd75-00665c9df7) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmpkx0q5bvt/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpkx0q5bvt/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmpkx0q5bvt/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpkx0q5bvt/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0150 "-" 3543388 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0110 "-" 3543388 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0098 "-" 3543388 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0098 "-" 3543388 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txcb7a87b7597547fc8747c-00665c9dd7" "proxy-server 3543388" 0.0021 "-" 3543388 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:35519/sdb1/1/a/c" "txcb7a87b7597547fc8747c-00665c9dd7" "container-server 3543388" 0.0018 "-" 3543388 0 test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp ... ok proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx19cc321d933c438ba00cd-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx19cc321d933c438ba00cd-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: tx19cc321d933c438ba00cd-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx237f1924068b470eb8451-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx237f1924068b470eb8451-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx237f1924068b470eb8451-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: tx237f1924068b470eb8451-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx40a8a7c4afe0483786007-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx40a8a7c4afe0483786007-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx40a8a7c4afe0483786007-00665c9ddc) proxy-server ERROR: Object GET returning 503 for [] (txn: tx40a8a7c4afe0483786007-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx665ca868399340bfb9069-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: tx5433b3f46f7f46e1ab491-00665c9ddc) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345750.90052', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:11 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:50817/sda1/1/a/c" "txcb7a87b7597547fc8747c-00665c9dd7" "container-server 3543388" 0.0018 "-" 3543388 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txcb7a87b7597547fc8747c-00665c9dd7" "proxy-server 3543388" 7.0048 "-" 3543388 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txcb7a87b7597547fc8747c-00665c9dd7" "proxy-server 3543388" 6.9832 "-" 3543388 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/18 PUT /v1/a/c HTTP/1.0 201 - - t - - - txcb7a87b7597547fc8747c-00665c9dd7 - 7.0188 - - 1717345751.879011154 1717345758.897848368 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx78238344c9524063a6754-00665c9dde" "proxy-server 3543388" 0.0021 "-" 3543388 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35519/sdb1/1/a1/c1" "tx78238344c9524063a6754-00665c9dde" "container-server 3543388" 0.0017 "-" 3543388 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:50817/sda1/1/a1/c1" "tx78238344c9524063a6754-00665c9dde" "container-server 3543388" 0.0023 "-" 3543388 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx78238344c9524063a6754-00665c9dde" "proxy-server 3543388" 0.0340 "-" 3543388 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx78238344c9524063a6754-00665c9dde" "proxy-server 3543388" 0.0222 "-" 3543388 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/18 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx78238344c9524063a6754-00665c9dde - 0.0475 - - 1717345758.900100708 1717345758.947639465 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:18 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx002381f300284812bcb07-00665c9dde" "proxy-server 3543388" 0.0107 "-" 3543388 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:35519/sdb1/2/a/c1" "tx002381f300284812bcb07-00665c9dde" "container-server 3543388" 0.0125 "-" 3543388 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:50817/sda1/2/a/c1" "tx002381f300284812bcb07-00665c9dde" "container-server 3543388" 0.0022 "-" 3543388 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx002381f300284812bcb07-00665c9dde" "proxy-server 3543388" 0.0368 "-" 3543388 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx002381f300284812bcb07-00665c9dde" "proxy-server 3543388" 0.0531 "-" 3543388 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/19 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx002381f300284812bcb07-00665c9dde - 0.0749 - - 1717345758.949701071 1717345759.024626255 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6bf09a9202774dc3a703f-00665c9ddf" "proxy-server 3543388" 0.0084 "-" 3543388 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:35519/sdb1/1/a/c2" "tx6bf09a9202774dc3a703f-00665c9ddf" "container-server 3543388" 0.0027 "-" 3543388 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:50817/sda1/1/a/c2" "tx6bf09a9202774dc3a703f-00665c9ddf" "container-server 3543388" 0.0121 "-" 3543388 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx6bf09a9202774dc3a703f-00665c9ddf" "proxy-server 3543388" 0.0545 "-" 3543388 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:19 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx6bf09a9202774dc3a703f-00665c9ddf" "proxy-server 3543388" 0.0394 "-" 3543388 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/19 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx6bf09a9202774dc3a703f-00665c9ddf - 0.0778 - - 1717345759.027426958 1717345759.105252981 2 REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9112e62bb189207f780238a301e8fcad', '542334f7-898f-4cfb-b8c9-78e6119a70c4-sda', '1717345757.34625', '1717345760', '1717345759', '', '1717345760', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('542334f7-898f-4cfb-b8c9-78e6119a70c4-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.26415 seconds (0.44167/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpuxbj76gt/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '603d1d69-f959-4a15-bfff-8f1318de46f3-sda', '1717345759.65592', '1717345762', '1717345761', '', '1717345762', 0, 0) test DEBUG: /tmp/tmpuxbj76gt/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '603d1d69-f959-4a15-bfff-8f1318de46f3-sda', '1717345759.65592', '1717345762', '1717345761', '', '1717345762', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('603d1d69-f959-4a15-bfff-8f1318de46f3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07255 seconds (13.78307/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkhejmk2_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '15dc76d1e17c79f916c02248e491db19', 'a95dafef-6918-4208-8a4a-0bcf443f5fb4-sda', '1717345759.78360', '1717345762', '1717345761', '', '1717345762', 1, 1) test DEBUG: /tmp/tmpkhejmk2_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '15dc76d1e17c79f916c02248e491db19', 'a95dafef-6918-4208-8a4a-0bcf443f5fb4-sda', '1717345759.78360', '1717345762', '1717345761', '', '1717345762', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a95dafef-6918-4208-8a4a-0bcf443f5fb4-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05966 seconds (16.76230/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphqlsnir3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ad45d611-005e-4d81-81ed-90fcf6aceacd-sda', '1717345762.34340', '1717345765', '1717345764', '', '1717345765', 0, 0) test DEBUG: /tmp/tmphqlsnir3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ad45d611-005e-4d81-81ed-90fcf6aceacd-sda', '1717345762.34340', '1717345765', '1717345764', '', '1717345765', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ad45d611-005e-4d81-81ed-90fcf6aceacd-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06272 seconds (15.94487/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6geryv_j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e89fa8d7035a3e7baeeec3cb8085855f', '04a5f366-20c7-4a0e-9547-b897efebd783-sda', '1717345762.45160', '1717345765', '1717345764', '', '1717345765', 1, 0) test DEBUG: /tmp/tmp6geryv_j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345766.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '04a5f366-20c7-4a0e-9547-b897efebd783-sda') test DEBUG: /tmp/tmp6geryv_j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00803638s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e89fa8d7035a3e7baeeec3cb8085855f', '04a5f366-20c7-4a0e-9547-b897efebd783-sda', '1717345762.45160', '1717345765', '1717345764', '', '1717345765', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04a5f366-20c7-4a0e-9547-b897efebd783-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06602 seconds (15.14788/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpuejuxtjn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c13b0286-67cb-4dce-b2f1-4bf5e436cf4d-sda', '1717345762.56396', '1717345765', '1717345764', '', '1717345765', 0, 1) test DEBUG: /tmp/tmpuejuxtjn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c13b0286-67cb-4dce-b2f1-4bf5e436cf4d-sda', '1717345762.56396', '1717345765', '1717345764', '', '1717345765', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c13b0286-67cb-4dce-b2f1-4bf5e436cf4d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04732 seconds (21.13080/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1s7lgahk/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e89fa8d7035a3e7baeeec3cb8085855f', '4280940a-39d3-4813-a789-d0fb66fe581b-sda', '1717345762.65585', '1717345765', '1717345764', '', '1717345765', 1, 1) test DEBUG: /tmp/tmp1s7lgahk/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e89fa8d7035a3e7baeeec3cb8085855f', '4280940a-39d3-4813-a789-d0fb66fe581b-sda', '1717345762.65585', '1717345765', '1717345764', '', '1717345765', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('4280940a-39d3-4813-a789-d0fb66fe581b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05947 seconds (16.81634/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:10 -0000', 'Content-Md5': 'i4CS9WiTvEfCFGqAHP8j0A==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8b8092f56893bc47c2146a801cff23d0', 'Content-Length': '1030946'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission ... ok test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txa23860c2fb734513bfd75-00665c9df7) test DEBUG: Setting object count to 1065 (txn: txa23860c2fb734513bfd75-00665c9df7) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpqhcflnh_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpqhcflnh_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpqhcflnh_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpqhcflnh_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0151 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0147 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0105 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0135 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx031989c69cb9411f9999a-00665c9df7" "proxy-server 3543398" 0.0026 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:38193/sda1/1/a/c" "tx031989c69cb9411f9999a-00665c9df7" "container-server 3543398" 0.0021 "-" 3543398 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:43321/sdb1/1/a/c" "tx031989c69cb9411f9999a-00665c9df7" "container-server 3543398" 0.0020 "-" 3543398 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx031989c69cb9411f9999a-00665c9df7" "proxy-server 3543398" 0.0229 "-" 3543398 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx031989c69cb9411f9999a-00665c9df7" "proxy-server 3543398" 0.0554 "-" 3543398 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/43 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx031989c69cb9411f9999a-00665c9df7 - 0.0722 - - 1717345783.861080647 1717345783.933286667 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txb38629bba60d4de7a6524-00665c9df7" "proxy-server 3543398" 0.0019 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:43321/sdb1/1/a1/c1" "txb38629bba60d4de7a6524-00665c9df7" "container-server 3543398" 0.0014 "-" 3543398 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:38193/sda1/1/a1/c1" "txb38629bba60d4de7a6524-00665c9df7" "container-server 3543398" 0.0014 "-" 3543398 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txb38629bba60d4de7a6524-00665c9df7" "proxy-server 3543398" 0.0295 "-" 3543398 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txb38629bba60d4de7a6524-00665c9df7" "proxy-server 3543398" 0.0186 "-" 3543398 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/43 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txb38629bba60d4de7a6524-00665c9df7 - 0.0422 - - 1717345783.936036110 1717345783.978261948 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:43 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb7abae78347b430e8e9a1-00665c9df7" "proxy-server 3543398" 0.0064 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:43321/sdb1/2/a/c1" "txb7abae78347b430e8e9a1-00665c9df7" "container-server 3543398" 0.0014 "-" 3543398 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:38193/sda1/2/a/c1" "txb7abae78347b430e8e9a1-00665c9df7" "container-server 3543398" 0.0084 "-" 3543398 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb7abae78347b430e8e9a1-00665c9df7" "proxy-server 3543398" 0.0239 "-" 3543398 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb7abae78347b430e8e9a1-00665c9df7" "proxy-server 3543398" 0.0365 "-" 3543398 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/44 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txb7abae78347b430e8e9a1-00665c9df7 - 0.0515 - - 1717345783.980142117 1717345784.031601667 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx78fa54d4bcf040b6a0169-00665c9df8" "proxy-server 3543398" 0.0089 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:43321/sdb1/1/a/c2" "tx78fa54d4bcf040b6a0169-00665c9df8" "container-server 3543398" 0.0065 "-" 3543398 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:38193/sda1/1/a/c2" "tx78fa54d4bcf040b6a0169-00665c9df8" "container-server 3543398" 0.0013 "-" 3543398 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx78fa54d4bcf040b6a0169-00665c9df8" "proxy-server 3543398" 0.0213 "-" 3543398 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx78fa54d4bcf040b6a0169-00665c9df8" "proxy-server 3543398" 0.0334 "-" 3543398 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/44 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx78fa54d4bcf040b6a0169-00665c9df8 - 0.0511 - - 1717345784.033444643 1717345784.084583521 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings ... ok swift: Warning: Cannot ratelimit without a memcached client swift: Warning: Cannot ratelimit without a memcached client swift: Warning: Cannot ratelimit without a memcached client swift: Warning: Cannot ratelimit without a memcached client swift: Warning: Cannot ratelimit without a memcached client swift: Option account_whitelist is deprecated. Use an internal client to POST a `X-Account-Sysmeta-Global-Write-Ratelimit: WHITELIST` header to the specific accounts instead. swift: Option account_blacklist is deprecated. Use an internal client to POST a `X-Account-Sysmeta-Global-Write-Ratelimit: BLACKLIST` header to the specific accounts instead. swift: Ratelimit sleep log: 0.5 for a/c/None swift: Returning 498 for PUT to a/c. Ratelimit (Max Sleep) Max Sleep Time Exceeded: 1.00 swift: Returning 498 for PUT to a/c. Ratelimit (Max Sleep) Max Sleep Time Exceeded: 1.00 swift: Ratelimit sleep log: 0.1 for a/c/None swift: Ratelimit sleep log: 0.5 for a/c/o swift: Returning 498 for PUT to a/c/o. Ratelimit (Max Sleep) Max Sleep Time Exceeded: 1.00 swift: Returning 498 for PUT to a/c/o. Ratelimit (Max Sleep) Max Sleep Time Exceeded: 1.00 swift: Ratelimit sleep log: 0.1 for a/c/o test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6e3eca78ceab4c2192ae4-00665c9dc3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx02e987ad54b544a4b86ad-00665c9dc7) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f0769e4403d4eb7a3919-00665c9dcb) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9326536713864f1d973be-00665c9dd1) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmppimcuguo/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '77b8faa1-1983-4ce3-9882-f87cf02542be-sda', '1717345767.09829', '1717345771', '1717345770', '', '1717345771', 0, 1) test DEBUG: /tmp/tmppimcuguo/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '77b8faa1-1983-4ce3-9882-f87cf02542be-sda', '1717345767.09829', '1717345771', '1717345770', '', '1717345771', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('77b8faa1-1983-4ce3-9882-f87cf02542be-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03553 seconds (28.14640/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpx2d_c3jn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3051f74915dfa04eb11adc89bf51b9e8', 'eee213d3-2ff1-43d0-9ba0-0ed63b772aa4-sda', '1717345767.18468', '1717345771', '1717345770', '', '1717345771', 1, 0) test DEBUG: /tmp/tmpx2d_c3jn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345772.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'eee213d3-2ff1-43d0-9ba0-0ed63b772aa4-sda') test DEBUG: /tmp/tmpx2d_c3jn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.011395s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3051f74915dfa04eb11adc89bf51b9e8', 'eee213d3-2ff1-43d0-9ba0-0ed63b772aa4-sda', '1717345767.18468', '1717345771', '1717345770', '', '1717345771', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('eee213d3-2ff1-43d0-9ba0-0ed63b772aa4-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07626 seconds (13.11378/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpbajfv5uy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e70abfb4-1c01-4751-8ff4-e54b635192c1-sda', '1717345767.30795', '1717345771', '1717345770', '', '1717345771', 0, 1) test DEBUG: /tmp/tmpbajfv5uy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e70abfb4-1c01-4751-8ff4-e54b635192c1-sda', '1717345767.30795', '1717345771', '1717345770', '', '1717345771', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e70abfb4-1c01-4751-8ff4-e54b635192c1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04638 seconds (21.55984/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmplogsagnw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3051f74915dfa04eb11adc89bf51b9e8', 'c7b498b8-3f89-408c-9e54-bab5195b7601-sda', '1717345767.40488', '1717345771', '1717345770', '', '1717345771', 1, 0) test DEBUG: /tmp/tmplogsagnw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3051f74915dfa04eb11adc89bf51b9e8', 'c7b498b8-3f89-408c-9e54-bab5195b7601-sda', '1717345767.40488', '1717345771', '1717345770', '', '1717345771', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c7b498b8-3f89-408c-9e54-bab5195b7601-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08763 seconds (11.41172/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpvhltpr6q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3b458354-248a-422b-8f9f-eb0803451eec-sda', '1717345767.56799', '1717345772', '1717345771', '', '1717345772', 0, 1) test DEBUG: /tmp/tmpvhltpr6q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3b458354-248a-422b-8f9f-eb0803451eec-sda', '1717345767.56799', '1717345772', '1717345771', '', '1717345772', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3b458354-248a-422b-8f9f-eb0803451eec-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03947 seconds (25.33551/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6r11liaf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6d4a3e96ab37f0978f2618feccaa58a0', 'ac2052a2-8c1a-4514-8aaa-c4e480fbf7f9-sda', '1717345767.66445', '1717345772', '1717345771', '', '1717345772', 1, 0) test DEBUG: /tmp/tmp6r11liaf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345773.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'ac2052a2-8c1a-4514-8aaa-c4e480fbf7f9-sda') test DEBUG: /tmp/tmp6r11liaf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00819492s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6d4a3e96ab37f0978f2618feccaa58a0', 'ac2052a2-8c1a-4514-8aaa-c4e480fbf7f9-sda', '1717345767.66445', '1717345772', '1717345771', '', '1717345772', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ac2052a2-8c1a-4514-8aaa-c4e480fbf7f9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05845 seconds (17.10862/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps ... ok swift: Ratelimit sleep log: 0.5 for a/c/None swift: Returning 498 for GET to a/c. Ratelimit (Max Sleep) Max Sleep Time Exceeded: 1.00 swift: Returning 498 for GET to a/c. Ratelimit (Max Sleep) Max Sleep Time Exceeded: 1.00 swift: Ratelimit sleep log: 0.1 for a/c/None test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission ... ok swift: Option account_whitelist is deprecated. Use an internal client to POST a `X-Account-Sysmeta-Global-Write-Ratelimit: WHITELIST` header to the specific accounts instead. swift: Option account_blacklist is deprecated. Use an internal client to POST a `X-Account-Sysmeta-Global-Write-Ratelimit: BLACKLIST` header to the specific accounts instead. swift: Returning 497 because of blacklisting: b swift: Option account_whitelist is deprecated. Use an internal client to POST a `X-Account-Sysmeta-Global-Write-Ratelimit: WHITELIST` header to the specific accounts instead. swift: Option account_blacklist is deprecated. Use an internal client to POST a `X-Account-Sysmeta-Global-Write-Ratelimit: BLACKLIST` header to the specific accounts instead. cont1 DEBUG: Could not load '/tmp/tmpwee5ghrs/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpwee5ghrs/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpwee5ghrs/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpwee5ghrs/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0124 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0105 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0113 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0113 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa01cece927ad455287474-00665c9df8" "proxy-server 3543398" 0.0022 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:60395/sdb1/1/a/c" "txa01cece927ad455287474-00665c9df8" "container-server 3543398" 0.0015 "-" 3543398 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:35165/sda1/1/a/c" "txa01cece927ad455287474-00665c9df8" "container-server 3543398" 0.0013 "-" 3543398 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa01cece927ad455287474-00665c9df8" "proxy-server 3543398" 0.0338 "-" 3543398 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa01cece927ad455287474-00665c9df8" "proxy-server 3543398" 0.0202 "-" 3543398 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/44 PUT /v1/a/c HTTP/1.0 201 - - t - - - txa01cece927ad455287474-00665c9df8 - 0.0480 - - 1717345784.213121176 1717345784.261074543 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx59e38d0497bc4224bf54e-00665c9df8" "proxy-server 3543398" 0.0017 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:60395/sdb1/1/a1/c1" "tx59e38d0497bc4224bf54e-00665c9df8" "container-server 3543398" 0.0015 "-" 3543398 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35165/sda1/1/a1/c1" "tx59e38d0497bc4224bf54e-00665c9df8" "container-server 3543398" 0.0014 "-" 3543398 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx59e38d0497bc4224bf54e-00665c9df8" "proxy-server 3543398" 0.0288 "-" 3543398 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx59e38d0497bc4224bf54e-00665c9df8" "proxy-server 3543398" 0.0180 "-" 3543398 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/44 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx59e38d0497bc4224bf54e-00665c9df8 - 0.0400 - - 1717345784.262984991 1717345784.302973509 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf044f0d7d7ab44dcb8c2b-00665c9df8" "proxy-server 3543398" 0.0087 "-" 3543398 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:35165/sda1/2/a/c1" "txf044f0d7d7ab44dcb8c2b-00665c9df8" "container-server 3543398" 0.0064 "-" 3543398 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:60395/sdb1/2/a/c1" "txf044f0d7d7ab44dcb8c2b-00665c9df8" "container-server 3543398" 0.0014 "-" 3543398 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txf044f0d7d7ab44dcb8c2b-00665c9df8" "proxy-server 3543398" 0.0243 "-" 3543398 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txf044f0d7d7ab44dcb8c2b-00665c9df8" "proxy-server 3543398" 0.0366 "-" 3543398 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/44 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txf044f0d7d7ab44dcb8c2b-00665c9df8 - 0.0546 - - 1717345784.304919720 1717345784.359515905 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd2208ea088ab400396b54-00665c9df8" "proxy-server 3543398" 0.0086 "-" 3543398 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:60395/sdb1/1/a/c2" "txd2208ea088ab400396b54-00665c9df8" "container-server 3543398" 0.0052 "-" 3543398 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:35165/sda1/1/a/c2" "txd2208ea088ab400396b54-00665c9df8" "container-server 3543398" 0.0014 "-" 3543398 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd2208ea088ab400396b54-00665c9df8" "proxy-server 3543398" 0.0347 "-" 3543398 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:29:44 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd2208ea088ab400396b54-00665c9df8" "proxy-server 3543398" 0.0230 "-" 3543398 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/29/44 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txd2208ea088ab400396b54-00665c9df8 - 0.0538 - - 1717345784.361406326 1717345784.415241718 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': True, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 4}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.2, 'concurrent_ec_extra_requests': 0}, app) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: tx5433b3f46f7f46e1ab491-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txcdf7c8d0e0724d37aac5a-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx4eb44f552e7a4e8fb5f17-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: tx1176c2dc2a6f4b718761f-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx4054fd39fff44eea8ee37-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txdf4ed57da1bd44d7a4461-00665c9de3) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: txdf4ed57da1bd44d7a4461-00665c9de3) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txdf4ed57da1bd44d7a4461-00665c9de3) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txdf4ed57da1bd44d7a4461-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: tx037d57066ab044fb8ee94-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: tx7c57930c4c9d4b2fbe2c4-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: txb4ee5791dd924de3aa486-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: txaa831bf211f9484590ea0-00665c9de3) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: txaa831bf211f9484590ea0-00665c9de3) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.0:1000/sda (txn: txaa831bf211f9484590ea0-00665c9de3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txb6737958581d4aa3b7bfb-00665c9de3) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txb6737958581d4aa3b7bfb-00665c9de3) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx6bc8fd4063844cdb9966f-00665c9de3) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx1c94e532e77648ba89804-00665c9de3) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx1c94e532e77648ba89804-00665c9de3) proxy-server ERROR: Object HEAD returning 503 for [] (txn: tx1c94e532e77648ba89804-00665c9de3) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx63ab06c57af04009b126c-00665c9de3) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx63ab06c57af04009b126c-00665c9de3) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx63ab06c57af04009b126c-00665c9de3) proxy-server ERROR: Object HEAD returning 503 for [507] (txn: tx63ab06c57af04009b126c-00665c9de3) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx575f7e276a404c0f815c2-00665c9de3) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx575f7e276a404c0f815c2-00665c9de3) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx575f7e276a404c0f815c2-00665c9de3) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'Content-Md5': '/JzPJjlXCqpRBmlyhGXNDg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'fc9ccf2639570aaa510669728465cd0e', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'Content-Md5': 'PggRsy85Tu07EsbATYB/cA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '3e0811b32f394eed3b12c6c04d807f70', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'Content-Md5': 'zDBBHq17IpuHzskOm+GHEQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'cc30411ead7b229b87cec90e9be18711', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'Content-Md5': 'hTnOh9sxGnzPiScnD+t0Mg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8539ce87db311a7ccf8927270feb7432', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345753.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345753.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345753.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345753.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'Content-Md5': 'mbEtd8ufKpkpCIfMjGD7Tw==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '99b12d77cb9f2a99290887cc8c60fb4f', 'Content-Length': '90'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys ... ok test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5txtcv60/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0aa82287-1f1c-4d95-b03b-16dfced48f8a-sda', '1717345769.67010', '1717345774', '1717345773', '', '1717345774', 0, 1) test DEBUG: /tmp/tmp5txtcv60/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0aa82287-1f1c-4d95-b03b-16dfced48f8a-sda', '1717345769.67010', '1717345774', '1717345773', '', '1717345774', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('0aa82287-1f1c-4d95-b03b-16dfced48f8a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05114 seconds (19.55547/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmprhhtefgq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '94fe8c4938142004e0cda5137e2b89d2', '350f800a-793f-4380-b9be-c3cc97920702-sda', '1717345769.77939', '1717345774', '1717345773', '', '1717345774', 1, 1) test DEBUG: /tmp/tmprhhtefgq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '94fe8c4938142004e0cda5137e2b89d2', '350f800a-793f-4380-b9be-c3cc97920702-sda', '1717345769.77939', '1717345774', '1717345773', '', '1717345774', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('350f800a-793f-4380-b9be-c3cc97920702-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05325 seconds (18.77760/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '67f5aee1-5eef-4ab9-8e1f-6bc4c3ec82f5-sda', '1717345772.57273', '1717345772.00000', '0', '', '1717345772.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '67f5aee1-5eef-4ab9-8e1f-6bc4c3ec82f5-sda', '1717345772.57273', '1717345773.00000', '0', '', '1717345772.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'a4d697227078942d59fcea118ab0875f', '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda', '1717345772.62606', '1717345772.62098', '0', '', '1717345772.62098', 11, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o_0', 'created_at': '1717345772.64669', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o_1', 'created_at': '1717345772.64767', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o_2', 'created_at': '1717345772.64794', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o_3', 'created_at': '1717345772.64820', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o_4', 'created_at': '1717345772.64845', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o_5', 'created_at': '1717345772.64870', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o_6', 'created_at': '1717345772.64895', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o_7', 'created_at': '1717345772.64920', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o_8', 'created_at': '1717345772.64945', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o_9', 'created_at': '1717345772.64970', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing_0', 'created_at': '1717345772.65000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 12, 'name': '/a/c/o_missing_1', 'created_at': '1717345772.69863', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 13, 'name': '/a/c/o_missing_2', 'created_at': '1717345772.71388', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 14, 'name': '/a/c/o_missing_3', 'created_at': '1717345772.72902', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 15, 'name': '/a/c/o_missing_4', 'created_at': '1717345772.74234', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 16, 'name': '/a/c/o_missing_5', 'created_at': '1717345772.75727', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 17, 'name': '/a/c/o_missing_6', 'created_at': '1717345772.77205', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 18, 'name': '/a/c/o_missing_7', 'created_at': '1717345772.78513', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 19, 'name': '/a/c/o_missing_8', 'created_at': '1717345772.79845', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 20, 'name': '/a/c/o_missing_9', 'created_at': '1717345772.81075', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (21, '33a8c8ed17bc43e2cca08f00d59a2377', '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda', '1717345772.62606', '1717345772.62098', '0', '', '1717345772.62098', 21, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': '/a/c/o_missing_10', 'created_at': '1717345772.82553', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '5340382e-0e6e-49c0-942a-e1f5254e72f5-sda') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345922.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f00/6a870efeed12ff79acaaccc49768bf00/6a870efeed12ff79acaaccc49768bf00.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345922.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f00/6a870efeed12ff79acaaccc49768bf00/6a870efeed12ff79acaaccc49768bf00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345924.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/588/c78b63404a36b167536f4ed251bbb588/c78b63404a36b167536f4ed251bbb588.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345924.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/588/c78b63404a36b167536f4ed251bbb588/c78b63404a36b167536f4ed251bbb588.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345926.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d20/8f1337ce1e0b2515153390fabda43d20/8f1337ce1e0b2515153390fabda43d20.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345926.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/d20/8f1337ce1e0b2515153390fabda43d20/8f1337ce1e0b2515153390fabda43d20.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345928.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f0e/e53acf826d2a01b622ad9dd9af6c9f0e/e53acf826d2a01b622ad9dd9af6c9f0e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345928.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f0e/e53acf826d2a01b622ad9dd9af6c9f0e/e53acf826d2a01b622ad9dd9af6c9f0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345930.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f13/23b989ab586a56f915d48aa3be383f13/23b989ab586a56f915d48aa3be383f13.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345930.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/f13/23b989ab586a56f915d48aa3be383f13/23b989ab586a56f915d48aa3be383f13.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345932.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/bf0/75e83382751077aaea735371fe300bf0/75e83382751077aaea735371fe300bf0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345932.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/bf0/75e83382751077aaea735371fe300bf0/75e83382751077aaea735371fe300bf0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345934.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/84a/5d688662c8238642a8e81f5a54aa984a/5d688662c8238642a8e81f5a54aa984a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345934.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/84a/5d688662c8238642a8e81f5a54aa984a/5d688662c8238642a8e81f5a54aa984a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345936.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/848/e6a9f60a24341f002da8658223e1c848/e6a9f60a24341f002da8658223e1c848.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345936.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/848/e6a9f60a24341f002da8658223e1c848/e6a9f60a24341f002da8658223e1c848.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345938.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e1f/308e1a665e614c904467fb34ec729e1f/308e1a665e614c904467fb34ec729e1f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345938.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/e1f/308e1a665e614c904467fb34ec729e1f/308e1a665e614c904467fb34ec729e1f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345940.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/b34/c049a4d2744918c296c65634ff3ddb34/c049a4d2744918c296c65634ff3ddb34.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345940.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/b34/c049a4d2744918c296c65634ff3ddb34/c049a4d2744918c296c65634ff3ddb34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345942.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dd9/519658248d5be788a2a906d89da84dd9/519658248d5be788a2a906d89da84dd9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345942.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/dd9/519658248d5be788a2a906d89da84dd9/519658248d5be788a2a906d89da84dd9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345944.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/cad/91f619a42b30d9bdbe312eef683a2cad/91f619a42b30d9bdbe312eef683a2cad.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345944.00000, db: /tmp/tmpgb9tzaln/sda/containers/0/cad/91f619a42b30d9bdbe312eef683a2cad/91f619a42b30d9bdbe312eef683a2cad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345716.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/7fb/6f44efff34033fc1a4544b5d858a07fb/6f44efff34033fc1a4544b5d858a07fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345721.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ec7/42d50d280fbd92825b081058e6f33ec7/42d50d280fbd92825b081058e6f33ec7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345726.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/623/a01df0bd8810ab70578bd46b4767a623/a01df0bd8810ab70578bd46b4767a623.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345731.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/765/8a6104dbf865614d7e82caf3f4185765/8a6104dbf865614d7e82caf3f4185765.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345736.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ecf/f54f2025ea7d4dd2fc3e41c7acbabecf/f54f2025ea7d4dd2fc3e41c7acbabecf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345741.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b85/7a5ce94efd2fef057e589d829bc0ab85/7a5ce94efd2fef057e589d829bc0ab85.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345746.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/288/6421fbddb250ac23384da03a1e4f2288/6421fbddb250ac23384da03a1e4f2288.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345751.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/bb6/ef5c7bd258e1f2ce32fcee97b246ebb6/ef5c7bd258e1f2ce32fcee97b246ebb6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345756.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ad1/48d6dd4605e47f98344cabefadd56ad1/48d6dd4605e47f98344cabefadd56ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345761.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d59/32422c87d1fb66064bce430ccb6e3d59/32422c87d1fb66064bce430ccb6e3d59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345766.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/1da/3db2cdf033579bdc225b531834a7c1da/3db2cdf033579bdc225b531834a7c1da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345771.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f13/13dcd0d823693a4a26ba602b28f31f13/13dcd0d823693a4a26ba602b28f31f13.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345776.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/c19/034fb88049f3a8d5079c0d41ee1a9c19/034fb88049f3a8d5079c0d41ee1a9c19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345781.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/820/e0394c420df70a09693db04f8c5e0820/e0394c420df70a09693db04f8c5e0820.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345786.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/259/0ba1254d13681b3dbdfd13e7b6af2259/0ba1254d13681b3dbdfd13e7b6af2259.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345791.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a04/4e8dd23d553a52a0e9b40fb0b2444a04/4e8dd23d553a52a0e9b40fb0b2444a04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345796.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b1a/20cf6d43b472d6058d95a9561081cb1a/20cf6d43b472d6058d95a9561081cb1a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345801.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/20d/3e8b352ea7f04005b10420a14754820d/3e8b352ea7f04005b10420a14754820d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345806.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/40c/966a5c926b0b10b14c7d4be945cc640c/966a5c926b0b10b14c7d4be945cc640c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345811.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b5d/a0ca57477be8e33821cf5fd06a32cb5d/a0ca57477be8e33821cf5fd06a32cb5d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345816.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/9d2/2a3795cecce7781774e152c8f81189d2/2a3795cecce7781774e152c8f81189d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345821.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b65/e1c0e81ab4d058bfb30ca522df863b65/e1c0e81ab4d058bfb30ca522df863b65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345826.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6c2/09ba571dadaea6baf83522cbaea666c2/09ba571dadaea6baf83522cbaea666c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345831.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/648/5d054da4c811982b1ec46474cc885648/5d054da4c811982b1ec46474cc885648.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345836.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/95b/9da272ac05615067086f41d755a3795b/9da272ac05615067086f41d755a3795b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345841.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/df5/9b5d5d50ed914c8dc3913b84e6045df5/9b5d5d50ed914c8dc3913b84e6045df5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345846.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/c37/e7dba346d34358f008f0ed774e651c37/e7dba346d34358f008f0ed774e651c37.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345851.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345856.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/46b/d9d6a0063ccf348d96d880b13688446b/d9d6a0063ccf348d96d880b13688446b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345861.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/eff/4676e9a95da312dc7ad25d3acea38eff/4676e9a95da312dc7ad25d3acea38eff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345866.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b4e/2db922433fe5a20d94ef41ec93212b4e/2db922433fe5a20d94ef41ec93212b4e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345871.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/577/ff75c4e36d977172a3d89f3beab16577/ff75c4e36d977172a3d89f3beab16577.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345876.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/c96/65114baa6159aa86d4bdf50d4e864c96/65114baa6159aa86d4bdf50d4e864c96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345881.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/cf9/7e546adec99e0fa281dfd403f52d1cf9/7e546adec99e0fa281dfd403f52d1cf9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345886.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2ed/277e533052e8d17bfea1e9edf9a672ed/277e533052e8d17bfea1e9edf9a672ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345891.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/20d/4eb4de89a5b06c3f96b6b1e27ba4320d/4eb4de89a5b06c3f96b6b1e27ba4320d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345896.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a09/6f3079015f6e806747b747aac873ea09/6f3079015f6e806747b747aac873ea09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345901.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/50f/2dfda13b8b2c063211a229b170b7950f/2dfda13b8b2c063211a229b170b7950f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345906.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/4d3/76b441cf2b0607691c2cb984068864d3/76b441cf2b0607691c2cb984068864d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345921.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/977/28186d1f70ef76a28bc8ace056955977/28186d1f70ef76a28bc8ace056955977.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345926.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/1ab/bb86b1d8f3e41d16afaa2b1b5ba911ab/bb86b1d8f3e41d16afaa2b1b5ba911ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345936.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6ce/9369170e1c48e6afd0c5b47cb7f726ce/9369170e1c48e6afd0c5b47cb7f726ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345941.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ad7/9aebbbd44c6ca8b6a9e71960af3cead7/9aebbbd44c6ca8b6a9e71960af3cead7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345946.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345951.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6f3/5145c6253305303db4393a56882626f3/5145c6253305303db4393a56882626f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345956.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e14/10e5c1e149b25cf3b6522e6269a4be14/10e5c1e149b25cf3b6522e6269a4be14.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345966.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/0ed/16a1ae8285942162f69d710c5a57d0ed/16a1ae8285942162f69d710c5a57d0ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345971.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345976.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ef6/379172803f41a0e92f620996a6740ef6/379172803f41a0e92f620996a6740ef6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345981.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/4a1/f6a147dfdf96222d3d20fd37815534a1/f6a147dfdf96222d3d20fd37815534a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345986.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a86/cee9b0a96eb2f4124cf61e01ea590a86/cee9b0a96eb2f4124cf61e01ea590a86.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345996.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f97/a9a9c61c644e627ef53f7b67b03d1f97/a9a9c61c644e627ef53f7b67b03d1f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346001.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/837/ba631526b18a59321d0855f186ee7837/ba631526b18a59321d0855f186ee7837.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346006.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346011.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/9b1/cecec856fd6c654aee8d170536e519b1/cecec856fd6c654aee8d170536e519b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346016.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/300/877a2ac6f2b81e47fd796b1879668300/877a2ac6f2b81e47fd796b1879668300.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346026.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d65/61bbd97b4675786fb5dc5d4a248b5d65/61bbd97b4675786fb5dc5d4a248b5d65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346036.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/271/8f38eee399580620c5ed9c4937aa7271/8f38eee399580620c5ed9c4937aa7271.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346041.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/685/4ffcdd81d57287b719521b14ca851685/4ffcdd81d57287b719521b14ca851685.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346046.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/bc9/464bccfc4534158b53a5055e3ed2bbc9/464bccfc4534158b53a5055e3ed2bbc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346051.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346056.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a20/570f6f426b196077b75cbfc70e0f6a20/570f6f426b196077b75cbfc70e0f6a20.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346061.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2e1/b79738364dc0d9b4ab38db5a3fdc52e1/b79738364dc0d9b4ab38db5a3fdc52e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346066.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a06/360bb3da57fc722726b9bd5af568ca06/360bb3da57fc722726b9bd5af568ca06.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346071.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/32d/2293f80a7b9db3f5f9ab86f42c9cb32d/2293f80a7b9db3f5f9ab86f42c9cb32d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346076.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/864/0ae0168a1f44b86adcf87700f40dc864/0ae0168a1f44b86adcf87700f40dc864.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346081.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/39f/b47b6b9fa58d2c6f4946f170f534739f/b47b6b9fa58d2c6f4946f170f534739f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346086.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/39e/a4116ac5ad831759a84e45854ccab39e/a4116ac5ad831759a84e45854ccab39e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346091.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d8a/ae0bf7520b1d386abbb62849a3aebd8a/ae0bf7520b1d386abbb62849a3aebd8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346096.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/c4f/0e678d8ebb818d8ee1d8eff46543cc4f/0e678d8ebb818d8ee1d8eff46543cc4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346101.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/7d7/6f45006d2ad4502123d61c60ae22e7d7/6f45006d2ad4502123d61c60ae22e7d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346106.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/50f/b1bd653a5e34d2410fddc16d8237750f/b1bd653a5e34d2410fddc16d8237750f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346111.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/254/848a0bad6b01f855062fb20a421a5254/848a0bad6b01f855062fb20a421a5254.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346116.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/022/67a2abaa52fe2ab7e9af4717a712d022/67a2abaa52fe2ab7e9af4717a712d022.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346121.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/949/f1aa5ab4515192f952abacf628b7e949/f1aa5ab4515192f952abacf628b7e949.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346126.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/769/2187a1fe3961d0458f4abbd003ca2769/2187a1fe3961d0458f4abbd003ca2769.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346131.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/c8f/8b5fa0e2c718d6f7e374bea4bee66c8f/8b5fa0e2c718d6f7e374bea4bee66c8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346136.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/eba/17cbf096b44647e8d26f4bb1d9ab1eba/17cbf096b44647e8d26f4bb1d9ab1eba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346141.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/26e/164f8b200251a0cb7a9f845e07eaa26e/164f8b200251a0cb7a9f845e07eaa26e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346146.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/8c1/1a78e058c420d84bb0d1c1f79c9f68c1/1a78e058c420d84bb0d1c1f79c9f68c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346151.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/010/c2cb8f1945cc7ca7ac309d1e63ad4010/c2cb8f1945cc7ca7ac309d1e63ad4010.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346156.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/439/6e25c51246ddc11ae6e072cea14de439/6e25c51246ddc11ae6e072cea14de439.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346161.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/066/733bef89dd06f42d88d6df810f847066/733bef89dd06f42d88d6df810f847066.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346166.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/bd1/600e394b20bdcdedbdc68b4f08907bd1/600e394b20bdcdedbdc68b4f08907bd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346171.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/eee/81c953943458438b4df515817d309eee/81c953943458438b4df515817d309eee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346176.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/7a2/e60faae50760bd6abab24907c5c237a2/e60faae50760bd6abab24907c5c237a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346181.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/dec/1d666f53a66f82f744dfa8d1d16b1dec/1d666f53a66f82f744dfa8d1d16b1dec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346186.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e2a/84e3cd4133b87fb2996bba0177f99e2a/84e3cd4133b87fb2996bba0177f99e2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346191.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ae0/8b9df3d8cd8d9b4c5c356d4eb850bae0/8b9df3d8cd8d9b4c5c356d4eb850bae0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346196.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f8f/3eea9db9546e213a2d266d3a606a1f8f/3eea9db9546e213a2d266d3a606a1f8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346201.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/7c8/baf991f05365ad2b2f29973b61a9d7c8/baf991f05365ad2b2f29973b61a9d7c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346206.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d2a/106d56997456c3dd3f975a987d81fd2a/106d56997456c3dd3f975a987d81fd2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346211.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/177/d3c13864b9d488fdc3af082c89586177/d3c13864b9d488fdc3af082c89586177.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346216.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/daf/1469ce45e5fa3d377e81809564b64daf/1469ce45e5fa3d377e81809564b64daf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346221.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/999/7a210c6ecea682ab33778684afeea999/7a210c6ecea682ab33778684afeea999.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346226.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6f0/93d43dc8e3588b15d457097319f806f0/93d43dc8e3588b15d457097319f806f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346231.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/13a/fefc29cd793b61103d1fb7f7a4ae613a/fefc29cd793b61103d1fb7f7a4ae613a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346236.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ab6/6b3931d784b1cb140ba50b5e340d0ab6/6b3931d784b1cb140ba50b5e340d0ab6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346241.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/193/0d4bff9281f3f7fffd02c64bb155f193/0d4bff9281f3f7fffd02c64bb155f193.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346246.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/cbf/7f1d65cd683541cc459900d81d656cbf/7f1d65cd683541cc459900d81d656cbf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346251.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/266/5e734352455c242c623aaa618c70c266/5e734352455c242c623aaa618c70c266.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346256.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/acb/f8c4c42ac7bf5f427a4bbf0b096daacb/f8c4c42ac7bf5f427a4bbf0b096daacb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346261.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ba5/2b20295f553c342cd5ac12abf91fbba5/2b20295f553c342cd5ac12abf91fbba5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346266.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/83e/c2fafcbae74c4e6d71c8ab55948fd83e/c2fafcbae74c4e6d71c8ab55948fd83e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346271.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2a8/943e567eda81a60d82b9b99d180c12a8/943e567eda81a60d82b9b99d180c12a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346276.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/98f/c3a409e3b14ca414a02bafa1904a598f/c3a409e3b14ca414a02bafa1904a598f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346281.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6b0/f58b4a3cffd5edcfa82da0d4f0ace6b0/f58b4a3cffd5edcfa82da0d4f0ace6b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346286.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f0e/26c0facd6eabc8bd101888f90b4d9f0e/26c0facd6eabc8bd101888f90b4d9f0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346291.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f19/095fd57f03a777a5e2018ff0701b4f19/095fd57f03a777a5e2018ff0701b4f19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346296.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/17e/99bc018de757fbda6a984455da76f17e/99bc018de757fbda6a984455da76f17e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346301.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/5ff/3795d54ca8e2273e978e4d83c9a095ff/3795d54ca8e2273e978e4d83c9a095ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346306.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/056/79e6ece54fe99aee2cdc814f77acb056/79e6ece54fe99aee2cdc814f77acb056.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346311.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/866/b070e8a3a4d7f61003bbc208bc7ed866/b070e8a3a4d7f61003bbc208bc7ed866.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346316.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/88d/e99dcb84141f8df8d4223abd116b888d/e99dcb84141f8df8d4223abd116b888d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346321.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/5b2/27843332fdb7e9f8e4aac754949bb5b2/27843332fdb7e9f8e4aac754949bb5b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346326.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2ba/76247e7ed045249e780718660bd0e2ba/76247e7ed045249e780718660bd0e2ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346331.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e9a/d9e9a91d5aec77b73132a2144b514e9a/d9e9a91d5aec77b73132a2144b514e9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346336.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d4b/8702181f37aa66e5bc776238ac6a3d4b/8702181f37aa66e5bc776238ac6a3d4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346341.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e54/a5a62d08dd0e265e789d06837fe36e54/a5a62d08dd0e265e789d06837fe36e54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346346.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/10d/7cca4a284896061c8a16f239e20a410d/7cca4a284896061c8a16f239e20a410d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346351.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/91e/9ba5bd785360d5350cc71b610bca091e/9ba5bd785360d5350cc71b610bca091e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346356.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ade/7baa23f0efaf73a432f642d7c0696ade/7baa23f0efaf73a432f642d7c0696ade.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346361.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/011/f5ad9bf745f27c1bac04091114fb1011/f5ad9bf745f27c1bac04091114fb1011.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346366.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/5c7/3b4bd8cb2c5d304b8deef63e73f475c7/3b4bd8cb2c5d304b8deef63e73f475c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346371.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/629/3f8bb749ba6301a8a5f2f35d3bbb9629/3f8bb749ba6301a8a5f2f35d3bbb9629.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346376.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/721/f7070b1b1cca9df8b26ef9f6524d2721/f7070b1b1cca9df8b26ef9f6524d2721.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346381.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/3ee/85f36d4aad0eac8eb1aab014097243ee/85f36d4aad0eac8eb1aab014097243ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346386.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/289/530cb418b56977d647009b51c91cb289/530cb418b56977d647009b51c91cb289.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346391.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b4d/7b14a00faf9cde99577d3167d12eab4d/7b14a00faf9cde99577d3167d12eab4d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346396.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/96d/6effb7b23ad166635c07f647d686396d/6effb7b23ad166635c07f647d686396d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346401.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b9a/2f50a25a1e883d406d1c7e44367b3b9a/2f50a25a1e883d406d1c7e44367b3b9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346406.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2ea/798534ad35629b1723ccea6dc11312ea/798534ad35629b1723ccea6dc11312ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346411.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/fbf/101fbd9819bc2bda9c9af00eb0130fbf/101fbd9819bc2bda9c9af00eb0130fbf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346416.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ff5/74c79cf142a627ee98c33085128c0ff5/74c79cf142a627ee98c33085128c0ff5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346421.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/676/f679b1e47c6c49c091986203842d8676/f679b1e47c6c49c091986203842d8676.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346426.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/591/a2e4ea80ed2509efe5726075df842591/a2e4ea80ed2509efe5726075df842591.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346431.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/bc1/8dd55857f364342095749dbb389a2bc1/8dd55857f364342095749dbb389a2bc1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346436.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/52b/1bf5da7ee093ceef5f493e41c35f352b/1bf5da7ee093ceef5f493e41c35f352b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346441.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6cd/6d7073fb1d5de6f8bec017c3e305a6cd/6d7073fb1d5de6f8bec017c3e305a6cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346446.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/cfb/a0d4f96fc99b83d32c0249daa6f4acfb/a0d4f96fc99b83d32c0249daa6f4acfb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346451.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/448/0d828fffcb8b313167e38b6f60e64448/0d828fffcb8b313167e38b6f60e64448.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346456.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/131/7d6af65ac62525f6cdc3b2541602e131/7d6af65ac62525f6cdc3b2541602e131.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346461.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/9a6/c7cf11af8a3de4cac2dd9d55133429a6/c7cf11af8a3de4cac2dd9d55133429a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346466.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a41/76f0eca3e420c8a84d847df48bbc1a41/76f0eca3e420c8a84d847df48bbc1a41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346471.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/edc/e11e843a43cd9ff9f4a4bbade50dbedc/e11e843a43cd9ff9f4a4bbade50dbedc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346476.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f2f/302d4ea3075f2499909ce06e18dfff2f/302d4ea3075f2499909ce06e18dfff2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346481.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/355/1e22ae6e66ed08000b9063a9cad4b355/1e22ae6e66ed08000b9063a9cad4b355.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346486.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/105/314ee83598860021ac0a5a3e39877105/314ee83598860021ac0a5a3e39877105.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346491.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f94/97294f9b7a4b69fe6f868dccb4fb0f94/97294f9b7a4b69fe6f868dccb4fb0f94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346496.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a72/5a1bf6362b6e6d43e5a5a9fd94d1fa72/5a1bf6362b6e6d43e5a5a9fd94d1fa72.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346501.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/5dd/2e9506c0a1e235f352f2efddb3cee5dd/2e9506c0a1e235f352f2efddb3cee5dd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346506.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/29b/e8e844ebfe6b8ccab71a0d55ea3a029b/e8e844ebfe6b8ccab71a0d55ea3a029b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346511.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/726/1e812704aa89da89eccfffab69611726/1e812704aa89da89eccfffab69611726.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346516.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a92/29191eeb0cea623acd6820c443c51a92/29191eeb0cea623acd6820c443c51a92.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346521.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b7f/c573e675c138c14a87a25821c8ca6b7f/c573e675c138c14a87a25821c8ca6b7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346526.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/31a/ecd08b661fdbaa9dfec2dca9d9bbd31a/ecd08b661fdbaa9dfec2dca9d9bbd31a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346531.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/34d/ac125feff52d8dc44fd7fbd44150234d/ac125feff52d8dc44fd7fbd44150234d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346536.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/cd2/a6f46592df74f3a24407aa631ee67cd2/a6f46592df74f3a24407aa631ee67cd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346541.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e9b/c3a5a8a8b1934bc498650eece49a3e9b/c3a5a8a8b1934bc498650eece49a3e9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346546.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/241/8a37827570811899e620daa4b6aa9241/8a37827570811899e620daa4b6aa9241.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346551.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a9d/d4c944732105424487fdab82a2506a9d/d4c944732105424487fdab82a2506a9d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346556.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/5cc/d9f158953001b566e5974c1595c345cc/d9f158953001b566e5974c1595c345cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346561.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2af/c40476b9a47e959824d06db6ebf722af/c40476b9a47e959824d06db6ebf722af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346566.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/142/13f1b33dfab5965618b448fb376dc142/13f1b33dfab5965618b448fb376dc142.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346571.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d5c/736df77992dc003c9ccd4a0e4e455d5c/736df77992dc003c9ccd4a0e4e455d5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346576.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/325/b9708a8d511b417203152e7c28c41325/b9708a8d511b417203152e7c28c41325.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346581.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b80/8c7f75b7d1d64cc7c962f4e05de97b80/8c7f75b7d1d64cc7c962f4e05de97b80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346586.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/bb0/038d52c5904164fe0fab6ead7481bbb0/038d52c5904164fe0fab6ead7481bbb0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346591.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/31d/a6488416f14f0b4f3f534eac59fca31d/a6488416f14f0b4f3f534eac59fca31d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346596.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/303/de1cb7998125f662f74a090dab604303/de1cb7998125f662f74a090dab604303.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346601.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/fac/99e71e06245fc8890b97bf832b7a9fac/99e71e06245fc8890b97bf832b7a9fac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346606.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/8f7/8fa9df1270271479053702311909c8f7/8fa9df1270271479053702311909c8f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346611.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/4ac/8d01052bb477112e6fb074359b61a4ac/8d01052bb477112e6fb074359b61a4ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346616.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/46b/e5c6b74d51636b9417eab6a6a88a646b/e5c6b74d51636b9417eab6a6a88a646b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346621.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/8bc/e23ad349d36b035ab6ed1dd70546d8bc/e23ad349d36b035ab6ed1dd70546d8bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346626.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/8cf/ff62bfba7e0786d19ee983efcb2df8cf/ff62bfba7e0786d19ee983efcb2df8cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346631.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/615/b9cf1891f80fd28c90a56852f89e5615/b9cf1891f80fd28c90a56852f89e5615.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346636.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b8c/f8bdc9f66caaf7fc3e25c618b7793b8c/f8bdc9f66caaf7fc3e25c618b7793b8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346641.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/dd1/44d88e4eea3229aec89142fd830ccdd1/44d88e4eea3229aec89142fd830ccdd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346646.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/904/7955488331aea11be73468a18e9f6904/7955488331aea11be73468a18e9f6904.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346651.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/514/cd7af719ed7cba988d0bbf59210c3514/cd7af719ed7cba988d0bbf59210c3514.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346656.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b04/7c044ed2d36411f0868de9775c62eb04/7c044ed2d36411f0868de9775c62eb04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346661.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/231/362e7c45086d94f2a315b2e2299bf231/362e7c45086d94f2a315b2e2299bf231.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346666.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/95d/2fbdf021314bd9f8b3ed2f3694a1a95d/2fbdf021314bd9f8b3ed2f3694a1a95d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346671.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b8e/e61d0f410427a933043f4d112cc93b8e/e61d0f410427a933043f4d112cc93b8e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346676.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/071/872e53f4db7638ecdf0cd7ffa854f071/872e53f4db7638ecdf0cd7ffa854f071.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346681.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e04/7db9a1503848303b57c518650613ae04/7db9a1503848303b57c518650613ae04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346686.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/37e/d1b910760f853214c46be9b9aa94037e/d1b910760f853214c46be9b9aa94037e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346691.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/3ea/5e57199fdadceb10d331a41234a0f3ea/5e57199fdadceb10d331a41234a0f3ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346696.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/025/2c54d84db1c0e6ea3be8b78911122025/2c54d84db1c0e6ea3be8b78911122025.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346701.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e6b/2c9fce9754e6d73f5cb7d742752b4e6b/2c9fce9754e6d73f5cb7d742752b4e6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346706.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/eec/803807172fd3f19955a09e51f364beec/803807172fd3f19955a09e51f364beec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346711.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/64c/3046a12150c35279cc2c9bdf3a5d064c/3046a12150c35279cc2c9bdf3a5d064c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346716.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/eea/93a20978a7983578d593e40859598eea/93a20978a7983578d593e40859598eea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346721.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/5dc/747446fd02aa4cec1bc1e36b718c65dc/747446fd02aa4cec1bc1e36b718c65dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346726.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/46c/edfd11a7d28adef550308aa8251ed46c/edfd11a7d28adef550308aa8251ed46c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346731.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/723/1f033709510da7627c15238ec0801723/1f033709510da7627c15238ec0801723.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346736.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/079/56239aa0c54253f2d21b9ed79f404079/56239aa0c54253f2d21b9ed79f404079.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346741.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/71f/126e19e95ddd69b322c5dfdf90bf971f/126e19e95ddd69b322c5dfdf90bf971f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346746.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/30e/bfb3644d6557790c535aeae9df98d30e/bfb3644d6557790c535aeae9df98d30e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346751.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e69/e434bece05f15918a5aa4d03179b6e69/e434bece05f15918a5aa4d03179b6e69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346756.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/3ee/5427740bef95380fbd87aaeae3ad83ee/5427740bef95380fbd87aaeae3ad83ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346761.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/743/33d13dd588be629b72118fb8764cd743/33d13dd588be629b72118fb8764cd743.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346766.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ff7/7e952901cb777ec8d29139a0db289ff7/7e952901cb777ec8d29139a0db289ff7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346771.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/4de/edf05b2ab09cc31fe4617c1dc38bb4de/edf05b2ab09cc31fe4617c1dc38bb4de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346776.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/622/4dfa320e9e006ab5713b90854e61e622/4dfa320e9e006ab5713b90854e61e622.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346781.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/976/1990dbb3d59a3a54fa11f4a79c516976/1990dbb3d59a3a54fa11f4a79c516976.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346786.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/190/97c9aa8a62524b2118487575201f1190/97c9aa8a62524b2118487575201f1190.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346791.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e3d/13b31e0956e611fbc313ddd7acc91e3d/13b31e0956e611fbc313ddd7acc91e3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346796.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b30/9c8943c478bdce126e12bd2fee393b30/9c8943c478bdce126e12bd2fee393b30.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346801.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/dbb/cd2d2a8586be2201cae605b1640eddbb/cd2d2a8586be2201cae605b1640eddbb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346806.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a5d/9bd4f10e59cb8140fc57859916f74a5d/9bd4f10e59cb8140fc57859916f74a5d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346811.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/617/6f890c0ab7f802a88fe6003dcffa1617/6f890c0ab7f802a88fe6003dcffa1617.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346816.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a10/40fa86d0045ee59d70cec7340d18ba10/40fa86d0045ee59d70cec7340d18ba10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346821.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/b8b/3b5d2ad9778a5cc7f1cdbe8f59a53b8b/3b5d2ad9778a5cc7f1cdbe8f59a53b8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346826.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ac0/86c853a0713e47df6fa77a647654bac0/86c853a0713e47df6fa77a647654bac0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346831.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/9c8/7419c1bf933232292236e0564b8489c8/7419c1bf933232292236e0564b8489c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346836.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/974/6fa61649b52d3f4b56f316a2fab3e974/6fa61649b52d3f4b56f316a2fab3e974.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346841.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a97/58d2f37dfd345d80dfe7b9a62d607a97/58d2f37dfd345d80dfe7b9a62d607a97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346846.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/ecb/e6970b2475fefc76f6d5670edecfcecb/e6970b2475fefc76f6d5670edecfcecb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346851.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/513/58bc77c7934ca0d5085dbc5bba00c513/58bc77c7934ca0d5085dbc5bba00c513.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346856.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2c3/a2266e630199e59bc7b7cf6d9daf72c3/a2266e630199e59bc7b7cf6d9daf72c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346861.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/4dd/2bba245c7bdd9b572e5b7877bd5294dd/2bba245c7bdd9b572e5b7877bd5294dd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346866.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e64/df03e33f67710de8a3bae16897fdee64/df03e33f67710de8a3bae16897fdee64.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346871.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/6b4/caaafe2dfc854133464d1636dc9be6b4/caaafe2dfc854133464d1636dc9be6b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346876.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/366/6273f53143421e771870ba48a8e81366/6273f53143421e771870ba48a8e81366.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346881.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d8f/8c0b26fcddc1ef58f1ab42ed9f01fd8f/8c0b26fcddc1ef58f1ab42ed9f01fd8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346886.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/c74/79ba06c5ea84704772a71d32820e9c74/79ba06c5ea84704772a71d32820e9c74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346891.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/a31/4d71949833f952096a340dbdff2fca31/4d71949833f952096a340dbdff2fca31.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346896.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/55c/a8274e82a0afa34439f2ecd617f7155c/a8274e82a0afa34439f2ecd617f7155c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346901.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d0a/f56a32e948ec06c77b8b16598f51cd0a/f56a32e948ec06c77b8b16598f51cd0a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346906.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/7a1/26fa2b060b6fa3745589fae5b0cd07a1/26fa2b060b6fa3745589fae5b0cd07a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346911.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/170/645b0b43ab440d7c4aec8bf9671be170/645b0b43ab440d7c4aec8bf9671be170.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346916.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/fdc/7880b86380ce578e8e29c9af2f162fdc/7880b86380ce578e8e29c9af2f162fdc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346921.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/001/478082087940ca479b44e9660771f001/478082087940ca479b44e9660771f001.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346926.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/dda/f2ecafd34bc649f7eddc79b39c7c2dda/f2ecafd34bc649f7eddc79b39c7c2dda.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346931.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/d53/3a17e9fb38e31eee4c085ca317b33d53/3a17e9fb38e31eee4c085ca317b33d53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346936.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/4e5/bd16878c78fb5cc601330322a849b4e5/bd16878c78fb5cc601330322a849b4e5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346941.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/1e5/70389dbee5af9a9189f05450c73bb1e5/70389dbee5af9a9189f05450c73bb1e5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346946.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/443/6613e584dd31d740a06d4639c4f34443/6613e584dd31d740a06d4639c4f34443.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346951.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/cd4/2d031029ad4a42b1c015164faa0b4cd4/2d031029ad4a42b1c015164faa0b4cd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346956.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/f8b/ccad5d9e508eeb053e18b2b18c329f8b/ccad5d9e508eeb053e18b2b18c329f8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346961.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/e24/ff30141d60b49b3776f420eb3d82fe24/ff30141d60b49b3776f420eb3d82fe24.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container test.unit.container.test_sharder.TestSharder.test_audit_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container test.unit.container.test_sharder.TestSharder.test_audit_shard_container ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3c17c56bdc904aa3bc372-00665c9de8) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3c17c56bdc904aa3bc372-00665c9de8) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3c17c56bdc904aa3bc372-00665c9de8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txecdb6ef86773456689940-00665c9de8) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txecdb6ef86773456689940-00665c9de8) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txecdb6ef86773456689940-00665c9de8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6a8242bbc6854903a1207-00665c9de8) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6a8242bbc6854903a1207-00665c9de8) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6a8242bbc6854903a1207-00665c9de8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345760.40315', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'Content-Md5': '9Z+cE+YYsVYXLtR89GLUSA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'f59f9c13e618b156172ed47cf462d448', 'Content-Length': '88'} test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1968a007f43e4f92b22af-00665c9dd4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 3, Lengths: [458, 458, 458, 458, 458, 458, 458, 458, 490, 458]: Invalid fragment payload in ECPyECLibDriver.decode (txn: txa0c251f150004a94841f5-00665c9dd8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd78f65318e2a4408b5a2d-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd78f65318e2a4408b5a2d-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd78f65318e2a4408b5a2d-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 412 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from primary (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx477e8cc4a3854e0c95306-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpftlprb0n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3396a8f7-5d0e-4760-a5e0-57cff81e4b1a-sda', '1717345777.64172', '1717345778', '0', '', '1717345778', 0, 1) test DEBUG: /tmp/tmpftlprb0n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3396a8f7-5d0e-4760-a5e0-57cff81e4b1a-sda', '1717345777.64172', '1717345778', '0', '', '1717345778', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3396a8f7-5d0e-4760-a5e0-57cff81e4b1a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04188 seconds (23.87843/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp__5k70q6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3ac032868e199fa9b28db90fb5f07124', 'fa5027c1-119c-4189-ab0e-817156ae0576-sda', '1717345777.71754', '1717345778', '0', '', '1717345778', 1, 0) test DEBUG: /tmp/tmp__5k70q6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345779.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'fa5027c1-119c-4189-ab0e-817156ae0576-sda') test DEBUG: /tmp/tmp__5k70q6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00912714s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3ac032868e199fa9b28db90fb5f07124', 'fa5027c1-119c-4189-ab0e-817156ae0576-sda', '1717345777.71754', '1717345778', '0', '', '1717345778', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fa5027c1-119c-4189-ab0e-817156ae0576-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp__5k70q6/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08700 seconds (11.49466/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp__5k70q6/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8818122497d0576b4f8fef465f3f223e', 'f5b016ec-baf0-4813-b86a-59c58213c2b8-sda', '1717345777.79554', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('f5b016ec-baf0-4813-b86a-59c58213c2b8-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8818122497d0576b4f8fef465f3f223e', 'f5b016ec-baf0-4813-b86a-59c58213c2b8-sda', '1717345777.79554', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('f5b016ec-baf0-4813-b86a-59c58213c2b8-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2d6u2x1t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '86d47cee-e2c3-4c61-9e1e-56dfb7abe908-sda', '1717345777.87810', '1717345778', '0', '', '1717345778', 0, 1) test DEBUG: /tmp/tmp2d6u2x1t/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '86d47cee-e2c3-4c61-9e1e-56dfb7abe908-sda', '1717345777.87810', '1717345778', '0', '', '1717345778', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('86d47cee-e2c3-4c61-9e1e-56dfb7abe908-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04604 seconds (21.72174/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpdi72le4v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3ac032868e199fa9b28db90fb5f07124', 'e2bcca1b-d059-430c-beee-600df9acdfe6-sda', '1717345777.96152', '1717345778', '0', '', '1717345778', 1, 0) test DEBUG: /tmp/tmpdi72le4v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3ac032868e199fa9b28db90fb5f07124', 'e2bcca1b-d059-430c-beee-600df9acdfe6-sda', '1717345777.96152', '1717345778', '0', '', '1717345778', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e2bcca1b-d059-430c-beee-600df9acdfe6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpdi72le4v/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07923 seconds (12.62180/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpdi72le4v/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8818122497d0576b4f8fef465f3f223e', '943ce525-0783-4ce2-aa94-7eec4724ede0-sda', '1717345778.03554', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('943ce525-0783-4ce2-aa94-7eec4724ede0-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8818122497d0576b4f8fef465f3f223e', '943ce525-0783-4ce2-aa94-7eec4724ede0-sda', '1717345778.03554', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('943ce525-0783-4ce2-aa94-7eec4724ede0-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpp911u5ku/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '221af8c0-a50a-4bab-b5d8-60822b991bb5-sda', '1717345778.12684', '1717345779', '1717345780', '', '1717345780', 0, 1) test DEBUG: /tmp/tmpp911u5ku/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346966.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/1a2/3116306f3d6aaec7e9e18e95594091a2/3116306f3d6aaec7e9e18e95594091a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346971.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2b3/596917ac152ac0099bfac8931fc972b3/596917ac152ac0099bfac8931fc972b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346976.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/941/433b6a03c3f87422a42d865eccfb2941/433b6a03c3f87422a42d865eccfb2941.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346981.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/82d/e2c30d069d78b415daa7a32ec9bbe82d/e2c30d069d78b415daa7a32ec9bbe82d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346986.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/81c/c35c9fdf4c3e19da77e50895abd1b81c/c35c9fdf4c3e19da77e50895abd1b81c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346991.00000-0, db: /tmp/tmpwp6h6s8o/sda/containers/0/2a0/02f065575e743e9a39d2af431e5d92a0/02f065575e743e9a39d2af431e5d92a0.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'found': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'created': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'sharding': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpu_91h31r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: own_shard_range reset to None should be 1717345764.00000, path: a/c, db: /tmp/tmpiqr5c7z8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345764.00000.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': a-c a-f, a-f c-f, path: a/c, db: /tmp/tmpo33l2auc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345765.00000-0, db: /tmp/tmp4_2gchbx/sda/containers/0/36e/a1a6f74da88400a38f6908e346dba36e/a1a6f74da88400a38f6908e346dba36e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345765.00000-0, db: /tmp/tmp4_2gchbx/sda/containers/0/36e/a1a6f74da88400a38f6908e346dba36e/a1a6f74da88400a38f6908e346dba36e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345768.00000-0, db: /tmp/tmp4_2gchbx/sda/containers/0/066/8536367c0f1ce460123c39b0b94f7066/8536367c0f1ce460123c39b0b94f7066.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345768.00000-0, db: /tmp/tmp4_2gchbx/sda/containers/0/066/8536367c0f1ce460123c39b0b94f7066/8536367c0f1ce460123c39b0b94f7066.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717345765.00000, (0, 0) as of 1717345765.00000, active as of 1717345765.00000> to ShardRange<'k' to 't' as of 1717345766.00000, (0, 0) as of 1717345766.00000, active as of 1717345766.00000>, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345770.00000, (0, 0) as of 1717345770.00000, shrinking as of 1717345767.00000> to ShardRange<'k' to 't' as of 1717345770.00000, (0, 0) as of 1717345770.00000, sharding as of 1717345771.00000>, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmp28x4wvms/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: txd41b7482443c4c0eb0499-00665c9ded) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Expect: 100-continue on /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: Connection refused: Please insert ten dollars (txn: tx45dcfc5570fd4056bad0c-00665c9ded) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.0:1000/sda (txn: tx2905f9541ff94cb98f86a-00665c9ded) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.1:1001/sdb (txn: tx522eb36884424096a03b1-00665c9ded) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.2:1002/sdc (txn: tx7eae1018304944acb7e47-00665c9ded) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Expect: 100-continue on /a/c/o.jpg: Timeout (Nones) (txn: tx02dbb655c77448598d394-00665c9ded) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Expect: 100-continue on /a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txbdd85e3e4c8b409fa6551-00665c9ded) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.2:1002/sdc (txn: tx6b95e98ad0fc4a8ba4e83-00665c9ded) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx6b21e9b3a4974e8c98642-00665c9ded) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/a/c/o.jpg: Timeout (Nones) (txn: tx71361b1497bb4588b0eba-00665c9ded) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to get final status of PUT to /v1/a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception (txn: tx542ba93b1b8e4c6bb1d8c-00665c9ded) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: txb640df1832104aaa9a9b7-00665c9ded) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345768.17860', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:29:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': 'E/Ado7cVxhkhUwZZGHtBvA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '13f01da3b715c61921530659187b41bc', 'Content-Length': '289'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'Content-Md5': 'PAq+63qG/0mqLKSN+Zv4ZQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '3c0abeeb7a86ff49aa2ca48df99bf865', 'Content-Length': '230'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:full_control FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-96251b3c6e16893b3c165daf35a6a40a-1717345769.00000-2, db: /tmp/tmp6x0_ttdf/sda/containers/0/aca/aeb6eb24f136d724442500a145a16aca/aeb6eb24f136d724442500a145a16aca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345774.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/552/50ff16038e84d8e7af296770eca04552/50ff16038e84d8e7af296770eca04552.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345774.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/552/50ff16038e84d8e7af296770eca04552/50ff16038e84d8e7af296770eca04552.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345783.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/6c7/0a173fca40daa0d9dfa86799bc5d06c7/0a173fca40daa0d9dfa86799bc5d06c7.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345783.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/6c7/0a173fca40daa0d9dfa86799bc5d06c7/0a173fca40daa0d9dfa86799bc5d06c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345792.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/568/d42d879510fea123ff9b399a6f675568/d42d879510fea123ff9b399a6f675568.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345792.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/568/d42d879510fea123ff9b399a6f675568/d42d879510fea123ff9b399a6f675568.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345801.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/fc6/05c006b24ef0a3b75db3e4a07a64bfc6/05c006b24ef0a3b75db3e4a07a64bfc6.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345801.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/fc6/05c006b24ef0a3b75db3e4a07a64bfc6/05c006b24ef0a3b75db3e4a07a64bfc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345810.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/c70/50752006bd4bcd45b409c279493bbc70/50752006bd4bcd45b409c279493bbc70.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345810.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/c70/50752006bd4bcd45b409c279493bbc70/50752006bd4bcd45b409c279493bbc70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345819.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/1cc/963addce298c9c7d6e73d45ee874c1cc/963addce298c9c7d6e73d45ee874c1cc.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345819.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/1cc/963addce298c9c7d6e73d45ee874c1cc/963addce298c9c7d6e73d45ee874c1cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345828.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/9ba/0616e66b874161abefef5387e0b549ba/0616e66b874161abefef5387e0b549ba.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345828.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/9ba/0616e66b874161abefef5387e0b549ba/0616e66b874161abefef5387e0b549ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345837.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/b06/61f0eec8286e276322f1af1b857e0b06/61f0eec8286e276322f1af1b857e0b06.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345837.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/b06/61f0eec8286e276322f1af1b857e0b06/61f0eec8286e276322f1af1b857e0b06.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345846.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/ac4/4fc14e115a8e801cb150b9293e7ceac4/4fc14e115a8e801cb150b9293e7ceac4.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345846.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/ac4/4fc14e115a8e801cb150b9293e7ceac4/4fc14e115a8e801cb150b9293e7ceac4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345846.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/ac4/4fc14e115a8e801cb150b9293e7ceac4/4fc14e115a8e801cb150b9293e7ceac4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345855.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/394/a2309c4ebe43950a45d0b91da2c0d394/a2309c4ebe43950a45d0b91da2c0d394.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345855.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/394/a2309c4ebe43950a45d0b91da2c0d394/a2309c4ebe43950a45d0b91da2c0d394.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345855.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/394/a2309c4ebe43950a45d0b91da2c0d394/a2309c4ebe43950a45d0b91da2c0d394.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345864.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/0bf/c433101ba0cc4ea69b035f75931a80bf/c433101ba0cc4ea69b035f75931a80bf.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345864.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/0bf/c433101ba0cc4ea69b035f75931a80bf/c433101ba0cc4ea69b035f75931a80bf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345864.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/0bf/c433101ba0cc4ea69b035f75931a80bf/c433101ba0cc4ea69b035f75931a80bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345873.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/6e9/17eb33e1adb9ad2e9e2418c96941b6e9/17eb33e1adb9ad2e9e2418c96941b6e9.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345873.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/6e9/17eb33e1adb9ad2e9e2418c96941b6e9/17eb33e1adb9ad2e9e2418c96941b6e9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345873.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/6e9/17eb33e1adb9ad2e9e2418c96941b6e9/17eb33e1adb9ad2e9e2418c96941b6e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345882.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/e71/38dbade5661e24c395296961d8227e71/38dbade5661e24c395296961d8227e71.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345882.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/e71/38dbade5661e24c395296961d8227e71/38dbade5661e24c395296961d8227e71.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345882.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/e71/38dbade5661e24c395296961d8227e71/38dbade5661e24c395296961d8227e71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345891.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/a26/1f715828bf49901dc02b4b77f8b43a26/1f715828bf49901dc02b4b77f8b43a26.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded ... ok proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server ERROR: Object returning 503 for [] (txn: txed3854f38c80433cade2a-00665c9ddc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '221af8c0-a50a-4bab-b5d8-60822b991bb5-sda', '1717345778.12684', '1717345779', '1717345780', '', '1717345780', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('221af8c0-a50a-4bab-b5d8-60822b991bb5-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.95724 seconds (0.51092/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwmhdn5tw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1dc42651010d50ae79b5f9ef15b24e79', '67d20d2a-b9fc-4500-8231-20ded9e85fab-sda', '1717345780.12511', '1717345781', '1717345782', '', '1717345782', 1, 0) test DEBUG: /tmp/tmpwmhdn5tw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345783.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '67d20d2a-b9fc-4500-8231-20ded9e85fab-sda') test DEBUG: /tmp/tmpwmhdn5tw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0094316s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1dc42651010d50ae79b5f9ef15b24e79', '67d20d2a-b9fc-4500-8231-20ded9e85fab-sda', '1717345780.12511', '1717345781', '1717345782', '', '1717345782', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('67d20d2a-b9fc-4500-8231-20ded9e85fab-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpwmhdn5tw/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08739 seconds (11.44261/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpwmhdn5tw/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '06bef5f865d29e3b6e2187ddc02c02d3', 'efded0ff-628f-48c4-bd97-86f913376371-sda', '1717345780.20988', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('efded0ff-628f-48c4-bd97-86f913376371-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '06bef5f865d29e3b6e2187ddc02c02d3', 'efded0ff-628f-48c4-bd97-86f913376371-sda', '1717345780.20988', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('efded0ff-628f-48c4-bd97-86f913376371-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpumqr4pxe/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '57963c4e-0ae0-436e-a48e-9dd21b5a3668-sda', '1717345780.27711', '1717345781', '1717345782', '', '1717345782', 0, 1) test DEBUG: /tmp/tmpumqr4pxe/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '57963c4e-0ae0-436e-a48e-9dd21b5a3668-sda', '1717345780.27711', '1717345781', '1717345782', '', '1717345782', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('57963c4e-0ae0-436e-a48e-9dd21b5a3668-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04676 seconds (21.38724/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5dksl24p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1dc42651010d50ae79b5f9ef15b24e79', 'a0404c5d-d3a8-48c8-806e-855f886da31d-sda', '1717345780.36097', '1717345781', '1717345782', '', '1717345782', 1, 1) test DEBUG: /tmp/tmp5dksl24p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1dc42651010d50ae79b5f9ef15b24e79', 'a0404c5d-d3a8-48c8-806e-855f886da31d-sda', '1717345780.36097', '1717345781', '1717345782', '', '1717345782', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a0404c5d-d3a8-48c8-806e-855f886da31d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp5dksl24p/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07641 seconds (13.08747/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp5dksl24p/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '722ba50726a80be3c779f9c233ecb3d6', 'fb803617-8975-4ff6-8801-2b26f8542ff0-sda', '1717345780.43298', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('fb803617-8975-4ff6-8801-2b26f8542ff0-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '722ba50726a80be3c779f9c233ecb3d6', 'fb803617-8975-4ff6-8801-2b26f8542ff0-sda', '1717345780.43298', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('fb803617-8975-4ff6-8801-2b26f8542ff0-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp9yynytp7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a8cab922-fd7f-46be-aa85-94ae01f110ac-sda', '1717345782.97981', '1717345782', '1717345783', '', '1717345783', 0, 1) test DEBUG: /tmp/tmp9yynytp7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a8cab922-fd7f-46be-aa85-94ae01f110ac-sda', '1717345782.97981', '1717345782', '1717345783', '', '1717345783', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a8cab922-fd7f-46be-aa85-94ae01f110ac-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03950 seconds (25.31746/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: txa69b46ed1d48406992dab-00665c9df3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txf852b7132dcf45fdae958-00665c9df3) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txf852b7132dcf45fdae958-00665c9df3) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txf852b7132dcf45fdae958-00665c9df3) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: txf852b7132dcf45fdae958-00665c9df3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1020, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1662, in read raise Exception('exception message') Exception: exception message (txn: tx9b98a23924e24ea8b79f0-00665c9df3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception: boom (txn: txb6f8c0daf81944b5b6f58-00665c9df3) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Timeout (Nones) (txn: txac5873a3df1046e98d771-00665c9df3) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txc7201e7a03a344179d087-00665c9df3) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx32c64d44c3b543f387a55-00665c9df3) proxy-server ERROR: ERROR 500 b'' Trying to PUT /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89 From Object Server 10.0.0.2:1002/sdc (txn: tx32c64d44c3b543f387a55-00665c9df3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx1f5e0354c99e4b04af45c-00665c9df3) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx1a120d54e66f41119aea4-00665c9df3) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.1:1001/sdb (txn: tx1a120d54e66f41119aea4-00665c9df3) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:28 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345768.19942', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345771.40642', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error Problem occurred while reading builder file: /tmp/tmpxf8rp8ck/tmpgq1isqr0. Generic error occurred Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sdb_"" with 1.0 weight got id 4 Device d5r1z1-127.0.0.1:6200R127.0.0.1:6200/sdc_"" with 1.0 weight got id 5 Reassigned 192 (300.00%) partitions. Balance is now 109.38. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 109.38 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 214.58. Dispersion is now 0.52 ------------------------------------------------------------------------------- NOTE: Dispersion of 0.520833 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Note: using /tmp/tmpbzqxj7wu/tmp82j474_o.builder instead of /tmp/tmpbzqxj7wu/tmp82j474_o.ring.gz as builder file Reassigned 1 (1.56%) partitions. Balance is now 100.00. Dispersion is now 0.52 ------------------------------------------------------------------------------- NOTE: Dispersion of 0.520833 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 24 (37.50%) partitions. Balance is now 1.04. Dispersion is now 0.00 Note: using /tmp/tmpvojirifn/tmp7jnmrsuv.builder instead of /tmp/tmpvojirifn/tmp7jnmrsuv.ring.gz as builder file Note: using /tmp/tmp1l2akcid/tmpp3c40b3v.builder instead of /tmp/tmp1l2akcid/tmpp3c40b3v.ring.gz as builder file Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 DEBUG: New builder; performing initial balance New builder; performing initial balance DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 768 new parts and 0 removed parts from replica-count change 768 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 69 (Last start was 0) Gather start is 69 (Last start was 0) DEBUG: Gathered 768 parts Gathered 768 parts DEBUG: Placed 223/0 onto dev r1z1-10.1.1.1/sdd Placed 223/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 223/1 onto dev r1z1-10.1.1.1/sda Placed 223/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 223/2 onto dev r1z1-10.1.1.1/sdc Placed 223/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 36/0 onto dev r1z1-10.1.1.1/sdb Placed 36/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 36/1 onto dev r1z1-10.1.1.1/sdd Placed 36/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 36/2 onto dev r1z1-10.1.1.1/sda Placed 36/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 143/0 onto dev r1z1-10.1.1.1/sdc Placed 143/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 143/1 onto dev r1z1-10.1.1.1/sdb Placed 143/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 143/2 onto dev r1z1-10.1.1.1/sdd Placed 143/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 73/0 onto dev r1z1-10.1.1.1/sda Placed 73/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 73/1 onto dev r1z1-10.1.1.1/sdc Placed 73/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 73/2 onto dev r1z1-10.1.1.1/sdb Placed 73/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 220/0 onto dev r1z1-10.1.1.1/sdd Placed 220/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 220/1 onto dev r1z1-10.1.1.1/sda Placed 220/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 220/2 onto dev r1z1-10.1.1.1/sdc Placed 220/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 151/0 onto dev r1z1-10.1.1.1/sdb Placed 151/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 151/1 onto dev r1z1-10.1.1.1/sdd Placed 151/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 151/2 onto dev r1z1-10.1.1.1/sda Placed 151/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 128/0 onto dev r1z1-10.1.1.1/sdc Placed 128/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 128/1 onto dev r1z1-10.1.1.1/sdb Placed 128/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 128/2 onto dev r1z1-10.1.1.1/sdd Placed 128/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 192/0 onto dev r1z1-10.1.1.1/sda Placed 192/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 192/1 onto dev r1z1-10.1.1.1/sdc Placed 192/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 192/2 onto dev r1z1-10.1.1.1/sdb Placed 192/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 182/0 onto dev r1z1-10.1.1.1/sdd Placed 182/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 182/1 onto dev r1z1-10.1.1.1/sda Placed 182/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 182/2 onto dev r1z1-10.1.1.1/sdc Placed 182/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 112/0 onto dev r1z1-10.1.1.1/sdb Placed 112/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 112/1 onto dev r1z1-10.1.1.1/sdd Placed 112/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 112/2 onto dev r1z1-10.1.1.1/sda Placed 112/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 28/0 onto dev r1z1-10.1.1.1/sdc Placed 28/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 28/1 onto dev r1z1-10.1.1.1/sdb Placed 28/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 28/2 onto dev r1z1-10.1.1.1/sdd Placed 28/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 243/0 onto dev r1z1-10.1.1.1/sda Placed 243/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 243/1 onto dev r1z1-10.1.1.1/sdc Placed 243/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 243/2 onto dev r1z1-10.1.1.1/sdb Placed 243/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 21/0 onto dev r1z1-10.1.1.1/sdd Placed 21/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 21/1 onto dev r1z1-10.1.1.1/sda Placed 21/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 21/2 onto dev r1z1-10.1.1.1/sdc Placed 21/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 7/0 onto dev r1z1-10.1.1.1/sdb Placed 7/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 7/1 onto dev r1z1-10.1.1.1/sdd Placed 7/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 7/2 onto dev r1z1-10.1.1.1/sda Placed 7/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 41/0 onto dev r1z1-10.1.1.1/sdc Placed 41/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 41/1 onto dev r1z1-10.1.1.1/sdb Placed 41/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 41/2 onto dev r1z1-10.1.1.1/sdd Placed 41/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 48/0 onto dev r1z1-10.1.1.1/sda Placed 48/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 48/1 onto dev r1z1-10.1.1.1/sdc Placed 48/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 48/2 onto dev r1z1-10.1.1.1/sdb Placed 48/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 254/0 onto dev r1z1-10.1.1.1/sdd Placed 254/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 254/1 onto dev r1z1-10.1.1.1/sda Placed 254/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 254/2 onto dev r1z1-10.1.1.1/sdc Placed 254/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 38/0 onto dev r1z1-10.1.1.1/sdb Placed 38/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 38/1 onto dev r1z1-10.1.1.1/sdd Placed 38/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 38/2 onto dev r1z1-10.1.1.1/sda Placed 38/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 77/0 onto dev r1z1-10.1.1.1/sdc Placed 77/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 77/1 onto dev r1z1-10.1.1.1/sdb Placed 77/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 77/2 onto dev r1z1-10.1.1.1/sdd Placed 77/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 153/0 onto dev r1z1-10.1.1.1/sda Placed 153/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 153/1 onto dev r1z1-10.1.1.1/sdc Placed 153/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 153/2 onto dev r1z1-10.1.1.1/sdb Placed 153/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 173/0 onto dev r1z1-10.1.1.1/sdd Placed 173/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 173/1 onto dev r1z1-10.1.1.1/sda Placed 173/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 173/2 onto dev r1z1-10.1.1.1/sdc Placed 173/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 250/0 onto dev r1z1-10.1.1.1/sdb Placed 250/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 250/1 onto dev r1z1-10.1.1.1/sdd Placed 250/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 250/2 onto dev r1z1-10.1.1.1/sda Placed 250/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 89/0 onto dev r1z1-10.1.1.1/sdc Placed 89/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 89/1 onto dev r1z1-10.1.1.1/sdb Placed 89/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 89/2 onto dev r1z1-10.1.1.1/sdd Placed 89/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 84/0 onto dev r1z1-10.1.1.1/sda Placed 84/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 84/1 onto dev r1z1-10.1.1.1/sdc Placed 84/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 84/2 onto dev r1z1-10.1.1.1/sdb Placed 84/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 145/0 onto dev r1z1-10.1.1.1/sdd Placed 145/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 145/1 onto dev r1z1-10.1.1.1/sda Placed 145/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 145/2 onto dev r1z1-10.1.1.1/sdc Placed 145/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 172/0 onto dev r1z1-10.1.1.1/sdb Placed 172/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 172/1 onto dev r1z1-10.1.1.1/sdd Placed 172/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 172/2 onto dev r1z1-10.1.1.1/sda Placed 172/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 207/0 onto dev r1z1-10.1.1.1/sdc Placed 207/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 207/1 onto dev r1z1-10.1.1.1/sdb Placed 207/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 207/2 onto dev r1z1-10.1.1.1/sdd Placed 207/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 165/0 onto dev r1z1-10.1.1.1/sda Placed 165/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 165/1 onto dev r1z1-10.1.1.1/sdc Placed 165/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 165/2 onto dev r1z1-10.1.1.1/sdb Placed 165/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 160/0 onto dev r1z1-10.1.1.1/sdd Placed 160/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 160/1 onto dev r1z1-10.1.1.1/sda Placed 160/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 160/2 onto dev r1z1-10.1.1.1/sdc Placed 160/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 113/0 onto dev r1z1-10.1.1.1/sdb Placed 113/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 113/1 onto dev r1z1-10.1.1.1/sdd Placed 113/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 113/2 onto dev r1z1-10.1.1.1/sda Placed 113/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 104/0 onto dev r1z1-10.1.1.1/sdc Placed 104/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 104/1 onto dev r1z1-10.1.1.1/sdb Placed 104/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 104/2 onto dev r1z1-10.1.1.1/sdd Placed 104/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 215/0 onto dev r1z1-10.1.1.1/sda Placed 215/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 215/1 onto dev r1z1-10.1.1.1/sdc Placed 215/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 215/2 onto dev r1z1-10.1.1.1/sdb Placed 215/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 189/0 onto dev r1z1-10.1.1.1/sdd Placed 189/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 189/1 onto dev r1z1-10.1.1.1/sda Placed 189/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 189/2 onto dev r1z1-10.1.1.1/sdc Placed 189/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 205/0 onto dev r1z1-10.1.1.1/sdb Placed 205/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 205/1 onto dev r1z1-10.1.1.1/sdd Placed 205/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 205/2 onto dev r1z1-10.1.1.1/sda Placed 205/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 60/0 onto dev r1z1-10.1.1.1/sdc Placed 60/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 60/1 onto dev r1z1-10.1.1.1/sdb Placed 60/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 60/2 onto dev r1z1-10.1.1.1/sdd Placed 60/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 242/0 onto dev r1z1-10.1.1.1/sda Placed 242/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 242/1 onto dev r1z1-10.1.1.1/sdc Placed 242/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 242/2 onto dev r1z1-10.1.1.1/sdb Placed 242/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 212/0 onto dev r1z1-10.1.1.1/sdd Placed 212/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 212/1 onto dev r1z1-10.1.1.1/sda Placed 212/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 212/2 onto dev r1z1-10.1.1.1/sdc Placed 212/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 147/0 onto dev r1z1-10.1.1.1/sdb Placed 147/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 147/1 onto dev r1z1-10.1.1.1/sdd Placed 147/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 147/2 onto dev r1z1-10.1.1.1/sda Placed 147/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 59/0 onto dev r1z1-10.1.1.1/sdc Placed 59/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 59/1 onto dev r1z1-10.1.1.1/sdb Placed 59/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 59/2 onto dev r1z1-10.1.1.1/sdd Placed 59/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 54/0 onto dev r1z1-10.1.1.1/sda Placed 54/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 54/1 onto dev r1z1-10.1.1.1/sdc Placed 54/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 54/2 onto dev r1z1-10.1.1.1/sdb Placed 54/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 229/0 onto dev r1z1-10.1.1.1/sdd Placed 229/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 229/1 onto dev r1z1-10.1.1.1/sda Placed 229/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 229/2 onto dev r1z1-10.1.1.1/sdc Placed 229/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 150/0 onto dev r1z1-10.1.1.1/sdb Placed 150/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 150/1 onto dev r1z1-10.1.1.1/sdd Placed 150/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 150/2 onto dev r1z1-10.1.1.1/sda Placed 150/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 57/0 onto dev r1z1-10.1.1.1/sdc Placed 57/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 57/1 onto dev r1z1-10.1.1.1/sdb Placed 57/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 57/2 onto dev r1z1-10.1.1.1/sdd Placed 57/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 253/0 onto dev r1z1-10.1.1.1/sda Placed 253/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 253/1 onto dev r1z1-10.1.1.1/sdc Placed 253/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 253/2 onto dev r1z1-10.1.1.1/sdb Placed 253/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 198/0 onto dev r1z1-10.1.1.1/sdd Placed 198/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 198/1 onto dev r1z1-10.1.1.1/sda Placed 198/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 198/2 onto dev r1z1-10.1.1.1/sdc Placed 198/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 140/0 onto dev r1z1-10.1.1.1/sdb Placed 140/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 140/1 onto dev r1z1-10.1.1.1/sdd Placed 140/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 140/2 onto dev r1z1-10.1.1.1/sda Placed 140/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 208/0 onto dev r1z1-10.1.1.1/sdc Placed 208/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 208/1 onto dev r1z1-10.1.1.1/sdb Placed 208/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 208/2 onto dev r1z1-10.1.1.1/sdd Placed 208/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 193/0 onto dev r1z1-10.1.1.1/sda Placed 193/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 193/1 onto dev r1z1-10.1.1.1/sdc Placed 193/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 193/2 onto dev r1z1-10.1.1.1/sdb Placed 193/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 211/0 onto dev r1z1-10.1.1.1/sdd Placed 211/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 211/1 onto dev r1z1-10.1.1.1/sda Placed 211/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 211/2 onto dev r1z1-10.1.1.1/sdc Placed 211/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 12/0 onto dev r1z1-10.1.1.1/sdb Placed 12/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 12/1 onto dev r1z1-10.1.1.1/sdd Placed 12/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 12/2 onto dev r1z1-10.1.1.1/sda Placed 12/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 158/0 onto dev r1z1-10.1.1.1/sdc Placed 158/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 158/1 onto dev r1z1-10.1.1.1/sdb Placed 158/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 158/2 onto dev r1z1-10.1.1.1/sdd Placed 158/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 114/0 onto dev r1z1-10.1.1.1/sda Placed 114/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 114/1 onto dev r1z1-10.1.1.1/sdc Placed 114/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 114/2 onto dev r1z1-10.1.1.1/sdb Placed 114/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 81/0 onto dev r1z1-10.1.1.1/sdd Placed 81/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 81/1 onto dev r1z1-10.1.1.1/sda Placed 81/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 81/2 onto dev r1z1-10.1.1.1/sdc Placed 81/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 159/0 onto dev r1z1-10.1.1.1/sdb Placed 159/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 159/1 onto dev r1z1-10.1.1.1/sdd Placed 159/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 159/2 onto dev r1z1-10.1.1.1/sda Placed 159/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 255/0 onto dev r1z1-10.1.1.1/sdc Placed 255/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 255/1 onto dev r1z1-10.1.1.1/sdb Placed 255/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 255/2 onto dev r1z1-10.1.1.1/sdd Placed 255/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 124/0 onto dev r1z1-10.1.1.1/sda Placed 124/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 124/1 onto dev r1z1-10.1.1.1/sdc Placed 124/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 124/2 onto dev r1z1-10.1.1.1/sdb Placed 124/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 217/0 onto dev r1z1-10.1.1.1/sdd Placed 217/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 217/1 onto dev r1z1-10.1.1.1/sda Placed 217/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 217/2 onto dev r1z1-10.1.1.1/sdc Placed 217/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 45/0 onto dev r1z1-10.1.1.1/sdb Placed 45/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 45/1 onto dev r1z1-10.1.1.1/sdd Placed 45/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 45/2 onto dev r1z1-10.1.1.1/sda Placed 45/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 154/0 onto dev r1z1-10.1.1.1/sdc Placed 154/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 154/1 onto dev r1z1-10.1.1.1/sdb Placed 154/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 154/2 onto dev r1z1-10.1.1.1/sdd Placed 154/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 16/0 onto dev r1z1-10.1.1.1/sda Placed 16/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 16/1 onto dev r1z1-10.1.1.1/sdc Placed 16/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 16/2 onto dev r1z1-10.1.1.1/sdb Placed 16/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 237/0 onto dev r1z1-10.1.1.1/sdd Placed 237/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 237/1 onto dev r1z1-10.1.1.1/sda Placed 237/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 237/2 onto dev r1z1-10.1.1.1/sdc Placed 237/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 123/0 onto dev r1z1-10.1.1.1/sdb Placed 123/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 123/1 onto dev r1z1-10.1.1.1/sdd Placed 123/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 123/2 onto dev r1z1-10.1.1.1/sda Placed 123/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 19/0 onto dev r1z1-10.1.1.1/sdc Placed 19/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 19/1 onto dev r1z1-10.1.1.1/sdb Placed 19/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 19/2 onto dev r1z1-10.1.1.1/sdd Placed 19/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 127/0 onto dev r1z1-10.1.1.1/sda Placed 127/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 127/1 onto dev r1z1-10.1.1.1/sdc Placed 127/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 127/2 onto dev r1z1-10.1.1.1/sdb Placed 127/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 222/0 onto dev r1z1-10.1.1.1/sdd Placed 222/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 222/1 onto dev r1z1-10.1.1.1/sda Placed 222/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 222/2 onto dev r1z1-10.1.1.1/sdc Placed 222/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 2/0 onto dev r1z1-10.1.1.1/sdb Placed 2/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 2/1 onto dev r1z1-10.1.1.1/sdd Placed 2/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 2/2 onto dev r1z1-10.1.1.1/sda Placed 2/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 58/0 onto dev r1z1-10.1.1.1/sdc Placed 58/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 58/1 onto dev r1z1-10.1.1.1/sdb Placed 58/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 58/2 onto dev r1z1-10.1.1.1/sdd Placed 58/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 196/0 onto dev r1z1-10.1.1.1/sda Placed 196/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 196/1 onto dev r1z1-10.1.1.1/sdc Placed 196/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 196/2 onto dev r1z1-10.1.1.1/sdb Placed 196/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 30/0 onto dev r1z1-10.1.1.1/sdd Placed 30/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 30/1 onto dev r1z1-10.1.1.1/sda Placed 30/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 30/2 onto dev r1z1-10.1.1.1/sdc Placed 30/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 202/0 onto dev r1z1-10.1.1.1/sdb Placed 202/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 202/1 onto dev r1z1-10.1.1.1/sdd Placed 202/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 202/2 onto dev r1z1-10.1.1.1/sda Placed 202/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 148/0 onto dev r1z1-10.1.1.1/sdc Placed 148/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 148/1 onto dev r1z1-10.1.1.1/sdb Placed 148/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 148/2 onto dev r1z1-10.1.1.1/sdd Placed 148/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 216/0 onto dev r1z1-10.1.1.1/sda Placed 216/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 216/1 onto dev r1z1-10.1.1.1/sdc Placed 216/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 216/2 onto dev r1z1-10.1.1.1/sdb Placed 216/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 234/0 onto dev r1z1-10.1.1.1/sdd Placed 234/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 234/1 onto dev r1z1-10.1.1.1/sda Placed 234/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 234/2 onto dev r1z1-10.1.1.1/sdc Placed 234/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 108/0 onto dev r1z1-10.1.1.1/sdb Placed 108/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 108/1 onto dev r1z1-10.1.1.1/sdd Placed 108/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 108/2 onto dev r1z1-10.1.1.1/sda Placed 108/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 43/0 onto dev r1z1-10.1.1.1/sdc Placed 43/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 43/1 onto dev r1z1-10.1.1.1/sdb Placed 43/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 43/2 onto dev r1z1-10.1.1.1/sdd Placed 43/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 138/0 onto dev r1z1-10.1.1.1/sda Placed 138/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 138/1 onto dev r1z1-10.1.1.1/sdc Placed 138/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 138/2 onto dev r1z1-10.1.1.1/sdb Placed 138/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 168/0 onto dev r1z1-10.1.1.1/sdd Placed 168/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 168/1 onto dev r1z1-10.1.1.1/sda Placed 168/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 168/2 onto dev r1z1-10.1.1.1/sdc Placed 168/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 130/0 onto dev r1z1-10.1.1.1/sdb Placed 130/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 130/1 onto dev r1z1-10.1.1.1/sdd Placed 130/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 130/2 onto dev r1z1-10.1.1.1/sda Placed 130/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 179/0 onto dev r1z1-10.1.1.1/sdc Placed 179/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 179/1 onto dev r1z1-10.1.1.1/sdb Placed 179/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 179/2 onto dev r1z1-10.1.1.1/sdd Placed 179/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 33/0 onto dev r1z1-10.1.1.1/sda Placed 33/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 33/1 onto dev r1z1-10.1.1.1/sdc Placed 33/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 33/2 onto dev r1z1-10.1.1.1/sdb Placed 33/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 191/0 onto dev r1z1-10.1.1.1/sdd Placed 191/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 191/1 onto dev r1z1-10.1.1.1/sda Placed 191/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 191/2 onto dev r1z1-10.1.1.1/sdc Placed 191/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 241/0 onto dev r1z1-10.1.1.1/sdb Placed 241/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 241/1 onto dev r1z1-10.1.1.1/sdd Placed 241/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 241/2 onto dev r1z1-10.1.1.1/sda Placed 241/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 50/0 onto dev r1z1-10.1.1.1/sdc Placed 50/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 50/1 onto dev r1z1-10.1.1.1/sdb Placed 50/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 50/2 onto dev r1z1-10.1.1.1/sdd Placed 50/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 194/0 onto dev r1z1-10.1.1.1/sda Placed 194/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 194/1 onto dev r1z1-10.1.1.1/sdc Placed 194/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 194/2 onto dev r1z1-10.1.1.1/sdb Placed 194/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 175/0 onto dev r1z1-10.1.1.1/sdd Placed 175/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 175/1 onto dev r1z1-10.1.1.1/sda Placed 175/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 175/2 onto dev r1z1-10.1.1.1/sdc Placed 175/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 209/0 onto dev r1z1-10.1.1.1/sdb Placed 209/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 209/1 onto dev r1z1-10.1.1.1/sdd Placed 209/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 209/2 onto dev r1z1-10.1.1.1/sda Placed 209/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 8/0 onto dev r1z1-10.1.1.1/sdc Placed 8/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 8/1 onto dev r1z1-10.1.1.1/sdb Placed 8/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 8/2 onto dev r1z1-10.1.1.1/sdd Placed 8/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 228/0 onto dev r1z1-10.1.1.1/sda Placed 228/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 228/1 onto dev r1z1-10.1.1.1/sdc Placed 228/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 228/2 onto dev r1z1-10.1.1.1/sdb Placed 228/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 236/0 onto dev r1z1-10.1.1.1/sdd Placed 236/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 236/1 onto dev r1z1-10.1.1.1/sda Placed 236/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 236/2 onto dev r1z1-10.1.1.1/sdc Placed 236/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 101/0 onto dev r1z1-10.1.1.1/sdb Placed 101/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 101/1 onto dev r1z1-10.1.1.1/sdd Placed 101/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 101/2 onto dev r1z1-10.1.1.1/sda Placed 101/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 163/0 onto dev r1z1-10.1.1.1/sdc Placed 163/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 163/1 onto dev r1z1-10.1.1.1/sdb Placed 163/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 163/2 onto dev r1z1-10.1.1.1/sdd Placed 163/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 72/0 onto dev r1z1-10.1.1.1/sda Placed 72/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 72/1 onto dev r1z1-10.1.1.1/sdc Placed 72/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 72/2 onto dev r1z1-10.1.1.1/sdb Placed 72/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 20/0 onto dev r1z1-10.1.1.1/sdd Placed 20/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 20/1 onto dev r1z1-10.1.1.1/sda Placed 20/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 20/2 onto dev r1z1-10.1.1.1/sdc Placed 20/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 188/0 onto dev r1z1-10.1.1.1/sdb Placed 188/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 188/1 onto dev r1z1-10.1.1.1/sdd Placed 188/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 188/2 onto dev r1z1-10.1.1.1/sda Placed 188/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 83/0 onto dev r1z1-10.1.1.1/sdc Placed 83/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 83/1 onto dev r1z1-10.1.1.1/sdb Placed 83/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 83/2 onto dev r1z1-10.1.1.1/sdd Placed 83/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 247/0 onto dev r1z1-10.1.1.1/sda Placed 247/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 247/1 onto dev r1z1-10.1.1.1/sdc Placed 247/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 247/2 onto dev r1z1-10.1.1.1/sdb Placed 247/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 186/0 onto dev r1z1-10.1.1.1/sdd Placed 186/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 186/1 onto dev r1z1-10.1.1.1/sda Placed 186/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 186/2 onto dev r1z1-10.1.1.1/sdc Placed 186/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 177/0 onto dev r1z1-10.1.1.1/sdb Placed 177/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 177/1 onto dev r1z1-10.1.1.1/sdd Placed 177/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 177/2 onto dev r1z1-10.1.1.1/sda Placed 177/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 170/0 onto dev r1z1-10.1.1.1/sdc Placed 170/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 170/1 onto dev r1z1-10.1.1.1/sdb Placed 170/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 170/2 onto dev r1z1-10.1.1.1/sdd Placed 170/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 115/0 onto dev r1z1-10.1.1.1/sda Placed 115/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 115/1 onto dev r1z1-10.1.1.1/sdc Placed 115/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 115/2 onto dev r1z1-10.1.1.1/sdb Placed 115/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 116/0 onto dev r1z1-10.1.1.1/sdd Placed 116/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 116/1 onto dev r1z1-10.1.1.1/sda Placed 116/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 116/2 onto dev r1z1-10.1.1.1/sdc Placed 116/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 239/0 onto dev r1z1-10.1.1.1/sdb Placed 239/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 239/1 onto dev r1z1-10.1.1.1/sdd Placed 239/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 239/2 onto dev r1z1-10.1.1.1/sda Placed 239/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 157/0 onto dev r1z1-10.1.1.1/sdc Placed 157/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 157/1 onto dev r1z1-10.1.1.1/sdb Placed 157/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 157/2 onto dev r1z1-10.1.1.1/sdd Placed 157/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 10/0 onto dev r1z1-10.1.1.1/sda Placed 10/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 10/1 onto dev r1z1-10.1.1.1/sdc Placed 10/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 10/2 onto dev r1z1-10.1.1.1/sdb Placed 10/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 185/0 onto dev r1z1-10.1.1.1/sdd Placed 185/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 185/1 onto dev r1z1-10.1.1.1/sda Placed 185/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 185/2 onto dev r1z1-10.1.1.1/sdc Placed 185/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 80/0 onto dev r1z1-10.1.1.1/sdb Placed 80/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 80/1 onto dev r1z1-10.1.1.1/sdd Placed 80/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 80/2 onto dev r1z1-10.1.1.1/sda Placed 80/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 139/0 onto dev r1z1-10.1.1.1/sdc Placed 139/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 139/1 onto dev r1z1-10.1.1.1/sdb Placed 139/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 139/2 onto dev r1z1-10.1.1.1/sdd Placed 139/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 102/0 onto dev r1z1-10.1.1.1/sda Placed 102/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 102/1 onto dev r1z1-10.1.1.1/sdc Placed 102/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 102/2 onto dev r1z1-10.1.1.1/sdb Placed 102/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 197/0 onto dev r1z1-10.1.1.1/sdd Placed 197/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 197/1 onto dev r1z1-10.1.1.1/sda Placed 197/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 197/2 onto dev r1z1-10.1.1.1/sdc Placed 197/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 40/0 onto dev r1z1-10.1.1.1/sdb Placed 40/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 40/1 onto dev r1z1-10.1.1.1/sdd Placed 40/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 40/2 onto dev r1z1-10.1.1.1/sda Placed 40/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 109/0 onto dev r1z1-10.1.1.1/sdc Placed 109/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 109/1 onto dev r1z1-10.1.1.1/sdb Placed 109/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 109/2 onto dev r1z1-10.1.1.1/sdd Placed 109/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 13/0 onto dev r1z1-10.1.1.1/sda Placed 13/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 13/1 onto dev r1z1-10.1.1.1/sdc Placed 13/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 13/2 onto dev r1z1-10.1.1.1/sdb Placed 13/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 187/0 onto dev r1z1-10.1.1.1/sdd Placed 187/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 187/1 onto dev r1z1-10.1.1.1/sda Placed 187/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 187/2 onto dev r1z1-10.1.1.1/sdc Placed 187/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 225/0 onto dev r1z1-10.1.1.1/sdb Placed 225/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 225/1 onto dev r1z1-10.1.1.1/sdd Placed 225/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 225/2 onto dev r1z1-10.1.1.1/sda Placed 225/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 22/0 onto dev r1z1-10.1.1.1/sdc Placed 22/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 22/1 onto dev r1z1-10.1.1.1/sdb Placed 22/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 22/2 onto dev r1z1-10.1.1.1/sdd Placed 22/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 135/0 onto dev r1z1-10.1.1.1/sda Placed 135/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 135/1 onto dev r1z1-10.1.1.1/sdc Placed 135/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 135/2 onto dev r1z1-10.1.1.1/sdb Placed 135/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 195/0 onto dev r1z1-10.1.1.1/sdd Placed 195/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 195/1 onto dev r1z1-10.1.1.1/sda Placed 195/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 195/2 onto dev r1z1-10.1.1.1/sdc Placed 195/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 6/0 onto dev r1z1-10.1.1.1/sdb Placed 6/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 6/1 onto dev r1z1-10.1.1.1/sdd Placed 6/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 6/2 onto dev r1z1-10.1.1.1/sda Placed 6/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 94/0 onto dev r1z1-10.1.1.1/sdc Placed 94/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 94/1 onto dev r1z1-10.1.1.1/sdb Placed 94/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 94/2 onto dev r1z1-10.1.1.1/sdd Placed 94/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 245/0 onto dev r1z1-10.1.1.1/sda Placed 245/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 245/1 onto dev r1z1-10.1.1.1/sdc Placed 245/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 245/2 onto dev r1z1-10.1.1.1/sdb Placed 245/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 122/0 onto dev r1z1-10.1.1.1/sdd Placed 122/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 122/1 onto dev r1z1-10.1.1.1/sda Placed 122/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 122/2 onto dev r1z1-10.1.1.1/sdc Placed 122/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 26/0 onto dev r1z1-10.1.1.1/sdb Placed 26/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 26/1 onto dev r1z1-10.1.1.1/sdd Placed 26/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 26/2 onto dev r1z1-10.1.1.1/sda Placed 26/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 204/0 onto dev r1z1-10.1.1.1/sdc Placed 204/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 204/1 onto dev r1z1-10.1.1.1/sdb Placed 204/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 204/2 onto dev r1z1-10.1.1.1/sdd Placed 204/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 183/0 onto dev r1z1-10.1.1.1/sda Placed 183/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 183/1 onto dev r1z1-10.1.1.1/sdc Placed 183/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 183/2 onto dev r1z1-10.1.1.1/sdb Placed 183/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 70/0 onto dev r1z1-10.1.1.1/sdd Placed 70/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 70/1 onto dev r1z1-10.1.1.1/sda Placed 70/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 70/2 onto dev r1z1-10.1.1.1/sdc Placed 70/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 90/0 onto dev r1z1-10.1.1.1/sdb Placed 90/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 90/1 onto dev r1z1-10.1.1.1/sdd Placed 90/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 90/2 onto dev r1z1-10.1.1.1/sda Placed 90/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 238/0 onto dev r1z1-10.1.1.1/sdc Placed 238/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 238/1 onto dev r1z1-10.1.1.1/sdb Placed 238/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 238/2 onto dev r1z1-10.1.1.1/sdd Placed 238/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 240/0 onto dev r1z1-10.1.1.1/sda Placed 240/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 240/1 onto dev r1z1-10.1.1.1/sdc Placed 240/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 240/2 onto dev r1z1-10.1.1.1/sdb Placed 240/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 224/0 onto dev r1z1-10.1.1.1/sdd Placed 224/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 224/1 onto dev r1z1-10.1.1.1/sda Placed 224/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 224/2 onto dev r1z1-10.1.1.1/sdc Placed 224/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 106/0 onto dev r1z1-10.1.1.1/sdb Placed 106/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 106/1 onto dev r1z1-10.1.1.1/sdd Placed 106/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 106/2 onto dev r1z1-10.1.1.1/sda Placed 106/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 125/0 onto dev r1z1-10.1.1.1/sdc Placed 125/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 125/1 onto dev r1z1-10.1.1.1/sdb Placed 125/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 125/2 onto dev r1z1-10.1.1.1/sdd Placed 125/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 206/0 onto dev r1z1-10.1.1.1/sda Placed 206/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 206/1 onto dev r1z1-10.1.1.1/sdc Placed 206/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 206/2 onto dev r1z1-10.1.1.1/sdb Placed 206/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 203/0 onto dev r1z1-10.1.1.1/sdd Placed 203/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 203/1 onto dev r1z1-10.1.1.1/sda Placed 203/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 203/2 onto dev r1z1-10.1.1.1/sdc Placed 203/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 67/0 onto dev r1z1-10.1.1.1/sdb Placed 67/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 67/1 onto dev r1z1-10.1.1.1/sdd Placed 67/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 67/2 onto dev r1z1-10.1.1.1/sda Placed 67/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 218/0 onto dev r1z1-10.1.1.1/sdc Placed 218/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 218/1 onto dev r1z1-10.1.1.1/sdb Placed 218/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 218/2 onto dev r1z1-10.1.1.1/sdd Placed 218/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 120/0 onto dev r1z1-10.1.1.1/sda Placed 120/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 120/1 onto dev r1z1-10.1.1.1/sdc Placed 120/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 120/2 onto dev r1z1-10.1.1.1/sdb Placed 120/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 190/0 onto dev r1z1-10.1.1.1/sdd Placed 190/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 190/1 onto dev r1z1-10.1.1.1/sda Placed 190/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 190/2 onto dev r1z1-10.1.1.1/sdc Placed 190/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 98/0 onto dev r1z1-10.1.1.1/sdb Placed 98/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 98/1 onto dev r1z1-10.1.1.1/sdd Placed 98/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 98/2 onto dev r1z1-10.1.1.1/sda Placed 98/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 4/0 onto dev r1z1-10.1.1.1/sdc Placed 4/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 4/1 onto dev r1z1-10.1.1.1/sdb Placed 4/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 4/2 onto dev r1z1-10.1.1.1/sdd Placed 4/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 144/0 onto dev r1z1-10.1.1.1/sda Placed 144/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 144/1 onto dev r1z1-10.1.1.1/sdc Placed 144/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 144/2 onto dev r1z1-10.1.1.1/sdb Placed 144/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 146/0 onto dev r1z1-10.1.1.1/sdd Placed 146/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 146/1 onto dev r1z1-10.1.1.1/sda Placed 146/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 146/2 onto dev r1z1-10.1.1.1/sdc Placed 146/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 233/0 onto dev r1z1-10.1.1.1/sdb Placed 233/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 233/1 onto dev r1z1-10.1.1.1/sdd Placed 233/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 233/2 onto dev r1z1-10.1.1.1/sda Placed 233/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 95/0 onto dev r1z1-10.1.1.1/sdc Placed 95/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 95/1 onto dev r1z1-10.1.1.1/sdb Placed 95/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 95/2 onto dev r1z1-10.1.1.1/sdd Placed 95/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 25/0 onto dev r1z1-10.1.1.1/sda Placed 25/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 25/1 onto dev r1z1-10.1.1.1/sdc Placed 25/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 25/2 onto dev r1z1-10.1.1.1/sdb Placed 25/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 39/0 onto dev r1z1-10.1.1.1/sdd Placed 39/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 39/1 onto dev r1z1-10.1.1.1/sda Placed 39/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 39/2 onto dev r1z1-10.1.1.1/sdc Placed 39/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 181/0 onto dev r1z1-10.1.1.1/sdb Placed 181/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 181/1 onto dev r1z1-10.1.1.1/sdd Placed 181/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 181/2 onto dev r1z1-10.1.1.1/sda Placed 181/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 119/0 onto dev r1z1-10.1.1.1/sdc Placed 119/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 119/1 onto dev r1z1-10.1.1.1/sdb Placed 119/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 119/2 onto dev r1z1-10.1.1.1/sdd Placed 119/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 131/0 onto dev r1z1-10.1.1.1/sda Placed 131/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 131/1 onto dev r1z1-10.1.1.1/sdc Placed 131/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 131/2 onto dev r1z1-10.1.1.1/sdb Placed 131/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 44/0 onto dev r1z1-10.1.1.1/sdd Placed 44/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 44/1 onto dev r1z1-10.1.1.1/sda Placed 44/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 44/2 onto dev r1z1-10.1.1.1/sdc Placed 44/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 251/0 onto dev r1z1-10.1.1.1/sdb Placed 251/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 251/1 onto dev r1z1-10.1.1.1/sdd Placed 251/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 251/2 onto dev r1z1-10.1.1.1/sda Placed 251/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 164/0 onto dev r1z1-10.1.1.1/sdc Placed 164/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 164/1 onto dev r1z1-10.1.1.1/sdb Placed 164/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 164/2 onto dev r1z1-10.1.1.1/sdd Placed 164/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 111/0 onto dev r1z1-10.1.1.1/sda Placed 111/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 111/1 onto dev r1z1-10.1.1.1/sdc Placed 111/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 111/2 onto dev r1z1-10.1.1.1/sdb Placed 111/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 201/0 onto dev r1z1-10.1.1.1/sdd Placed 201/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 201/1 onto dev r1z1-10.1.1.1/sda Placed 201/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 201/2 onto dev r1z1-10.1.1.1/sdc Placed 201/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 213/0 onto dev r1z1-10.1.1.1/sdb Placed 213/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 213/1 onto dev r1z1-10.1.1.1/sdd Placed 213/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 213/2 onto dev r1z1-10.1.1.1/sda Placed 213/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 249/0 onto dev r1z1-10.1.1.1/sdc Placed 249/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 249/1 onto dev r1z1-10.1.1.1/sdb Placed 249/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 249/2 onto dev r1z1-10.1.1.1/sdd Placed 249/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 232/0 onto dev r1z1-10.1.1.1/sda Placed 232/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 232/1 onto dev r1z1-10.1.1.1/sdc Placed 232/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 232/2 onto dev r1z1-10.1.1.1/sdb Placed 232/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 210/0 onto dev r1z1-10.1.1.1/sdd Placed 210/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 210/1 onto dev r1z1-10.1.1.1/sda Placed 210/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 210/2 onto dev r1z1-10.1.1.1/sdc Placed 210/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 230/0 onto dev r1z1-10.1.1.1/sdb Placed 230/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 230/1 onto dev r1z1-10.1.1.1/sdd Placed 230/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 230/2 onto dev r1z1-10.1.1.1/sda Placed 230/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 18/0 onto dev r1z1-10.1.1.1/sdc Placed 18/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 18/1 onto dev r1z1-10.1.1.1/sdb Placed 18/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 18/2 onto dev r1z1-10.1.1.1/sdd Placed 18/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 184/0 onto dev r1z1-10.1.1.1/sda Placed 184/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 184/1 onto dev r1z1-10.1.1.1/sdc Placed 184/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 184/2 onto dev r1z1-10.1.1.1/sdb Placed 184/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 64/0 onto dev r1z1-10.1.1.1/sdd Placed 64/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 64/1 onto dev r1z1-10.1.1.1/sda Placed 64/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 64/2 onto dev r1z1-10.1.1.1/sdc Placed 64/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 178/0 onto dev r1z1-10.1.1.1/sdb Placed 178/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 178/1 onto dev r1z1-10.1.1.1/sdd Placed 178/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 178/2 onto dev r1z1-10.1.1.1/sda Placed 178/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 126/0 onto dev r1z1-10.1.1.1/sdc Placed 126/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 126/1 onto dev r1z1-10.1.1.1/sdb Placed 126/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 126/2 onto dev r1z1-10.1.1.1/sdd Placed 126/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 47/0 onto dev r1z1-10.1.1.1/sda Placed 47/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 47/1 onto dev r1z1-10.1.1.1/sdc Placed 47/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 47/2 onto dev r1z1-10.1.1.1/sdb Placed 47/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 1/0 onto dev r1z1-10.1.1.1/sdd Placed 1/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 1/1 onto dev r1z1-10.1.1.1/sda Placed 1/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 1/2 onto dev r1z1-10.1.1.1/sdc Placed 1/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 117/0 onto dev r1z1-10.1.1.1/sdb Placed 117/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 117/1 onto dev r1z1-10.1.1.1/sdd Placed 117/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 117/2 onto dev r1z1-10.1.1.1/sda Placed 117/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 121/0 onto dev r1z1-10.1.1.1/sdc Placed 121/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 121/1 onto dev r1z1-10.1.1.1/sdb Placed 121/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 121/2 onto dev r1z1-10.1.1.1/sdd Placed 121/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 11/0 onto dev r1z1-10.1.1.1/sda Placed 11/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 11/1 onto dev r1z1-10.1.1.1/sdc Placed 11/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 11/2 onto dev r1z1-10.1.1.1/sdb Placed 11/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 85/0 onto dev r1z1-10.1.1.1/sdd Placed 85/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 85/1 onto dev r1z1-10.1.1.1/sda Placed 85/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 85/2 onto dev r1z1-10.1.1.1/sdc Placed 85/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 29/0 onto dev r1z1-10.1.1.1/sdb Placed 29/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 29/1 onto dev r1z1-10.1.1.1/sdd Placed 29/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 29/2 onto dev r1z1-10.1.1.1/sda Placed 29/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 53/0 onto dev r1z1-10.1.1.1/sdc Placed 53/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 53/1 onto dev r1z1-10.1.1.1/sdb Placed 53/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 53/2 onto dev r1z1-10.1.1.1/sdd Placed 53/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 24/0 onto dev r1z1-10.1.1.1/sda Placed 24/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 24/1 onto dev r1z1-10.1.1.1/sdc Placed 24/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 24/2 onto dev r1z1-10.1.1.1/sdb Placed 24/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 162/0 onto dev r1z1-10.1.1.1/sdd Placed 162/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 162/1 onto dev r1z1-10.1.1.1/sda Placed 162/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 162/2 onto dev r1z1-10.1.1.1/sdc Placed 162/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 35/0 onto dev r1z1-10.1.1.1/sdb Placed 35/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 35/1 onto dev r1z1-10.1.1.1/sdd Placed 35/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 35/2 onto dev r1z1-10.1.1.1/sda Placed 35/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 155/0 onto dev r1z1-10.1.1.1/sdc Placed 155/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 155/1 onto dev r1z1-10.1.1.1/sdb Placed 155/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 155/2 onto dev r1z1-10.1.1.1/sdd Placed 155/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 100/0 onto dev r1z1-10.1.1.1/sda Placed 100/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 100/1 onto dev r1z1-10.1.1.1/sdc Placed 100/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 100/2 onto dev r1z1-10.1.1.1/sdb Placed 100/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 69/0 onto dev r1z1-10.1.1.1/sdd Placed 69/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 69/1 onto dev r1z1-10.1.1.1/sda Placed 69/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 69/2 onto dev r1z1-10.1.1.1/sdc Placed 69/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 176/0 onto dev r1z1-10.1.1.1/sdb Placed 176/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 176/1 onto dev r1z1-10.1.1.1/sdd Placed 176/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 176/2 onto dev r1z1-10.1.1.1/sda Placed 176/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 252/0 onto dev r1z1-10.1.1.1/sdc Placed 252/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 252/1 onto dev r1z1-10.1.1.1/sdb Placed 252/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 252/2 onto dev r1z1-10.1.1.1/sdd Placed 252/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 142/0 onto dev r1z1-10.1.1.1/sda Placed 142/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 142/1 onto dev r1z1-10.1.1.1/sdc Placed 142/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 142/2 onto dev r1z1-10.1.1.1/sdb Placed 142/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 134/0 onto dev r1z1-10.1.1.1/sdd Placed 134/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 134/1 onto dev r1z1-10.1.1.1/sda Placed 134/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 134/2 onto dev r1z1-10.1.1.1/sdc Placed 134/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 32/0 onto dev r1z1-10.1.1.1/sdb Placed 32/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 32/1 onto dev r1z1-10.1.1.1/sdd Placed 32/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 32/2 onto dev r1z1-10.1.1.1/sda Placed 32/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 214/0 onto dev r1z1-10.1.1.1/sdc Placed 214/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 214/1 onto dev r1z1-10.1.1.1/sdb Placed 214/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 214/2 onto dev r1z1-10.1.1.1/sdd Placed 214/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 87/0 onto dev r1z1-10.1.1.1/sda Placed 87/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 87/1 onto dev r1z1-10.1.1.1/sdc Placed 87/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 87/2 onto dev r1z1-10.1.1.1/sdb Placed 87/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 63/0 onto dev r1z1-10.1.1.1/sdd Placed 63/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 63/1 onto dev r1z1-10.1.1.1/sda Placed 63/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 63/2 onto dev r1z1-10.1.1.1/sdc Placed 63/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 103/0 onto dev r1z1-10.1.1.1/sdb Placed 103/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 103/1 onto dev r1z1-10.1.1.1/sdd Placed 103/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 103/2 onto dev r1z1-10.1.1.1/sda Placed 103/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 79/0 onto dev r1z1-10.1.1.1/sdc Placed 79/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 79/1 onto dev r1z1-10.1.1.1/sdb Placed 79/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 79/2 onto dev r1z1-10.1.1.1/sdd Placed 79/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 231/0 onto dev r1z1-10.1.1.1/sda Placed 231/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 231/1 onto dev r1z1-10.1.1.1/sdc Placed 231/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 231/2 onto dev r1z1-10.1.1.1/sdb Placed 231/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 62/0 onto dev r1z1-10.1.1.1/sdd Placed 62/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 62/1 onto dev r1z1-10.1.1.1/sda Placed 62/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 62/2 onto dev r1z1-10.1.1.1/sdc Placed 62/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 17/0 onto dev r1z1-10.1.1.1/sdb Placed 17/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 17/1 onto dev r1z1-10.1.1.1/sdd Placed 17/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 17/2 onto dev r1z1-10.1.1.1/sda Placed 17/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 136/0 onto dev r1z1-10.1.1.1/sdc Placed 136/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 136/1 onto dev r1z1-10.1.1.1/sdb Placed 136/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 136/2 onto dev r1z1-10.1.1.1/sdd Placed 136/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 96/0 onto dev r1z1-10.1.1.1/sda Placed 96/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 96/1 onto dev r1z1-10.1.1.1/sdc Placed 96/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 96/2 onto dev r1z1-10.1.1.1/sdb Placed 96/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 34/0 onto dev r1z1-10.1.1.1/sdd Placed 34/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 34/1 onto dev r1z1-10.1.1.1/sda Placed 34/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 34/2 onto dev r1z1-10.1.1.1/sdc Placed 34/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 107/0 onto dev r1z1-10.1.1.1/sdb Placed 107/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 107/1 onto dev r1z1-10.1.1.1/sdd Placed 107/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 107/2 onto dev r1z1-10.1.1.1/sda Placed 107/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 49/0 onto dev r1z1-10.1.1.1/sdc Placed 49/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 49/1 onto dev r1z1-10.1.1.1/sdb Placed 49/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 49/2 onto dev r1z1-10.1.1.1/sdd Placed 49/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 221/0 onto dev r1z1-10.1.1.1/sda Placed 221/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 221/1 onto dev r1z1-10.1.1.1/sdc Placed 221/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 221/2 onto dev r1z1-10.1.1.1/sdb Placed 221/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 152/0 onto dev r1z1-10.1.1.1/sdd Placed 152/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 152/1 onto dev r1z1-10.1.1.1/sda Placed 152/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 152/2 onto dev r1z1-10.1.1.1/sdc Placed 152/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 23/0 onto dev r1z1-10.1.1.1/sdb Placed 23/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 23/1 onto dev r1z1-10.1.1.1/sdd Placed 23/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 23/2 onto dev r1z1-10.1.1.1/sda Placed 23/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 93/0 onto dev r1z1-10.1.1.1/sdc Placed 93/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 93/1 onto dev r1z1-10.1.1.1/sdb Placed 93/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 93/2 onto dev r1z1-10.1.1.1/sdd Placed 93/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 92/0 onto dev r1z1-10.1.1.1/sda Placed 92/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 92/1 onto dev r1z1-10.1.1.1/sdc Placed 92/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 92/2 onto dev r1z1-10.1.1.1/sdb Placed 92/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 65/0 onto dev r1z1-10.1.1.1/sdd Placed 65/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 65/1 onto dev r1z1-10.1.1.1/sda Placed 65/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 65/2 onto dev r1z1-10.1.1.1/sdc Placed 65/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 42/0 onto dev r1z1-10.1.1.1/sdb Placed 42/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 42/1 onto dev r1z1-10.1.1.1/sdd Placed 42/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 42/2 onto dev r1z1-10.1.1.1/sda Placed 42/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 78/0 onto dev r1z1-10.1.1.1/sdc Placed 78/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 78/1 onto dev r1z1-10.1.1.1/sdb Placed 78/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 78/2 onto dev r1z1-10.1.1.1/sdd Placed 78/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 31/0 onto dev r1z1-10.1.1.1/sda Placed 31/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 31/1 onto dev r1z1-10.1.1.1/sdc Placed 31/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 31/2 onto dev r1z1-10.1.1.1/sdb Placed 31/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 74/0 onto dev r1z1-10.1.1.1/sdd Placed 74/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 74/1 onto dev r1z1-10.1.1.1/sda Placed 74/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 74/2 onto dev r1z1-10.1.1.1/sdc Placed 74/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 132/0 onto dev r1z1-10.1.1.1/sdb Placed 132/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 132/1 onto dev r1z1-10.1.1.1/sdd Placed 132/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 132/2 onto dev r1z1-10.1.1.1/sda Placed 132/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 156/0 onto dev r1z1-10.1.1.1/sdc Placed 156/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 156/1 onto dev r1z1-10.1.1.1/sdb Placed 156/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 156/2 onto dev r1z1-10.1.1.1/sdd Placed 156/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 118/0 onto dev r1z1-10.1.1.1/sda Placed 118/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 118/1 onto dev r1z1-10.1.1.1/sdc Placed 118/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 118/2 onto dev r1z1-10.1.1.1/sdb Placed 118/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 0/0 onto dev r1z1-10.1.1.1/sdd Placed 0/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 0/1 onto dev r1z1-10.1.1.1/sda Placed 0/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 0/2 onto dev r1z1-10.1.1.1/sdc Placed 0/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 82/0 onto dev r1z1-10.1.1.1/sdb Placed 82/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 82/1 onto dev r1z1-10.1.1.1/sdd Placed 82/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 82/2 onto dev r1z1-10.1.1.1/sda Placed 82/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 75/0 onto dev r1z1-10.1.1.1/sdc Placed 75/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 75/1 onto dev r1z1-10.1.1.1/sdb Placed 75/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 75/2 onto dev r1z1-10.1.1.1/sdd Placed 75/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 15/0 onto dev r1z1-10.1.1.1/sda Placed 15/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 15/1 onto dev r1z1-10.1.1.1/sdc Placed 15/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 15/2 onto dev r1z1-10.1.1.1/sdb Placed 15/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 51/0 onto dev r1z1-10.1.1.1/sdd Placed 51/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 51/1 onto dev r1z1-10.1.1.1/sda Placed 51/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 51/2 onto dev r1z1-10.1.1.1/sdc Placed 51/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 105/0 onto dev r1z1-10.1.1.1/sdb Placed 105/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 105/1 onto dev r1z1-10.1.1.1/sdd Placed 105/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 105/2 onto dev r1z1-10.1.1.1/sda Placed 105/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 71/0 onto dev r1z1-10.1.1.1/sdc Placed 71/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 71/1 onto dev r1z1-10.1.1.1/sdb Placed 71/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 71/2 onto dev r1z1-10.1.1.1/sdd Placed 71/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 110/0 onto dev r1z1-10.1.1.1/sda Placed 110/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 110/1 onto dev r1z1-10.1.1.1/sdc Placed 110/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 110/2 onto dev r1z1-10.1.1.1/sdb Placed 110/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 99/0 onto dev r1z1-10.1.1.1/sdd Placed 99/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 99/1 onto dev r1z1-10.1.1.1/sda Placed 99/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 99/2 onto dev r1z1-10.1.1.1/sdc Placed 99/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 14/0 onto dev r1z1-10.1.1.1/sdb Placed 14/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 14/1 onto dev r1z1-10.1.1.1/sdd Placed 14/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 14/2 onto dev r1z1-10.1.1.1/sda Placed 14/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 137/0 onto dev r1z1-10.1.1.1/sdc Placed 137/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 137/1 onto dev r1z1-10.1.1.1/sdb Placed 137/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 137/2 onto dev r1z1-10.1.1.1/sdd Placed 137/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 227/0 onto dev r1z1-10.1.1.1/sda Placed 227/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 227/1 onto dev r1z1-10.1.1.1/sdc Placed 227/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 227/2 onto dev r1z1-10.1.1.1/sdb Placed 227/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 246/0 onto dev r1z1-10.1.1.1/sdd Placed 246/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 246/1 onto dev r1z1-10.1.1.1/sda Placed 246/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 246/2 onto dev r1z1-10.1.1.1/sdc Placed 246/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 199/0 onto dev r1z1-10.1.1.1/sdb Placed 199/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 199/1 onto dev r1z1-10.1.1.1/sdd Placed 199/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 199/2 onto dev r1z1-10.1.1.1/sda Placed 199/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 9/0 onto dev r1z1-10.1.1.1/sdc Placed 9/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 9/1 onto dev r1z1-10.1.1.1/sdb Placed 9/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 9/2 onto dev r1z1-10.1.1.1/sdd Placed 9/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 3/0 onto dev r1z1-10.1.1.1/sda Placed 3/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 3/1 onto dev r1z1-10.1.1.1/sdc Placed 3/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 3/2 onto dev r1z1-10.1.1.1/sdb Placed 3/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 169/0 onto dev r1z1-10.1.1.1/sdd Placed 169/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 169/1 onto dev r1z1-10.1.1.1/sda Placed 169/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 169/2 onto dev r1z1-10.1.1.1/sdc Placed 169/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 171/0 onto dev r1z1-10.1.1.1/sdb Placed 171/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 171/1 onto dev r1z1-10.1.1.1/sdd Placed 171/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 171/2 onto dev r1z1-10.1.1.1/sda Placed 171/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 129/0 onto dev r1z1-10.1.1.1/sdc Placed 129/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 129/1 onto dev r1z1-10.1.1.1/sdb Placed 129/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 129/2 onto dev r1z1-10.1.1.1/sdd Placed 129/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 5/0 onto dev r1z1-10.1.1.1/sda Placed 5/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 5/1 onto dev r1z1-10.1.1.1/sdc Placed 5/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 5/2 onto dev r1z1-10.1.1.1/sdb Placed 5/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 235/0 onto dev r1z1-10.1.1.1/sdd Placed 235/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 235/1 onto dev r1z1-10.1.1.1/sda Placed 235/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 235/2 onto dev r1z1-10.1.1.1/sdc Placed 235/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 244/0 onto dev r1z1-10.1.1.1/sdb Placed 244/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 244/1 onto dev r1z1-10.1.1.1/sdd Placed 244/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 244/2 onto dev r1z1-10.1.1.1/sda Placed 244/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 76/0 onto dev r1z1-10.1.1.1/sdc Placed 76/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 76/1 onto dev r1z1-10.1.1.1/sdb Placed 76/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 76/2 onto dev r1z1-10.1.1.1/sdd Placed 76/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 226/0 onto dev r1z1-10.1.1.1/sda Placed 226/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 226/1 onto dev r1z1-10.1.1.1/sdc Placed 226/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 226/2 onto dev r1z1-10.1.1.1/sdb Placed 226/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 68/0 onto dev r1z1-10.1.1.1/sdd Placed 68/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 68/1 onto dev r1z1-10.1.1.1/sda Placed 68/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 68/2 onto dev r1z1-10.1.1.1/sdc Placed 68/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 174/0 onto dev r1z1-10.1.1.1/sdb Placed 174/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 174/1 onto dev r1z1-10.1.1.1/sdd Placed 174/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 174/2 onto dev r1z1-10.1.1.1/sda Placed 174/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 91/0 onto dev r1z1-10.1.1.1/sdc Placed 91/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 91/1 onto dev r1z1-10.1.1.1/sdb Placed 91/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 91/2 onto dev r1z1-10.1.1.1/sdd Placed 91/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 167/0 onto dev r1z1-10.1.1.1/sda Placed 167/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 167/1 onto dev r1z1-10.1.1.1/sdc Placed 167/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 167/2 onto dev r1z1-10.1.1.1/sdb Placed 167/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 248/0 onto dev r1z1-10.1.1.1/sdd Placed 248/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 248/1 onto dev r1z1-10.1.1.1/sda Placed 248/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 248/2 onto dev r1z1-10.1.1.1/sdc Placed 248/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 46/0 onto dev r1z1-10.1.1.1/sdb Placed 46/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 46/1 onto dev r1z1-10.1.1.1/sdd Placed 46/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 46/2 onto dev r1z1-10.1.1.1/sda Placed 46/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 219/0 onto dev r1z1-10.1.1.1/sdc Placed 219/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 219/1 onto dev r1z1-10.1.1.1/sdb Placed 219/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 219/2 onto dev r1z1-10.1.1.1/sdd Placed 219/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 166/0 onto dev r1z1-10.1.1.1/sda Placed 166/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 166/1 onto dev r1z1-10.1.1.1/sdc Placed 166/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 166/2 onto dev r1z1-10.1.1.1/sdb Placed 166/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 55/0 onto dev r1z1-10.1.1.1/sdd Placed 55/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 55/1 onto dev r1z1-10.1.1.1/sda Placed 55/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 55/2 onto dev r1z1-10.1.1.1/sdc Placed 55/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 161/0 onto dev r1z1-10.1.1.1/sdb Placed 161/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 161/1 onto dev r1z1-10.1.1.1/sdd Placed 161/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 161/2 onto dev r1z1-10.1.1.1/sda Placed 161/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 97/0 onto dev r1z1-10.1.1.1/sdc Placed 97/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 97/1 onto dev r1z1-10.1.1.1/sdb Placed 97/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 97/2 onto dev r1z1-10.1.1.1/sdd Placed 97/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 61/0 onto dev r1z1-10.1.1.1/sda Placed 61/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 61/1 onto dev r1z1-10.1.1.1/sdc Placed 61/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 61/2 onto dev r1z1-10.1.1.1/sdb Placed 61/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 52/0 onto dev r1z1-10.1.1.1/sdd Placed 52/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 52/1 onto dev r1z1-10.1.1.1/sda Placed 52/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 52/2 onto dev r1z1-10.1.1.1/sdc Placed 52/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 180/0 onto dev r1z1-10.1.1.1/sdb Placed 180/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 180/1 onto dev r1z1-10.1.1.1/sdd Placed 180/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 180/2 onto dev r1z1-10.1.1.1/sda Placed 180/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 149/0 onto dev r1z1-10.1.1.1/sdc Placed 149/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 149/1 onto dev r1z1-10.1.1.1/sdb Placed 149/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 149/2 onto dev r1z1-10.1.1.1/sdd Placed 149/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 133/0 onto dev r1z1-10.1.1.1/sda Placed 133/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 133/1 onto dev r1z1-10.1.1.1/sdc Placed 133/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 133/2 onto dev r1z1-10.1.1.1/sdb Placed 133/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 141/0 onto dev r1z1-10.1.1.1/sdd Placed 141/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 141/1 onto dev r1z1-10.1.1.1/sda Placed 141/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 141/2 onto dev r1z1-10.1.1.1/sdc Placed 141/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 88/0 onto dev r1z1-10.1.1.1/sdb Placed 88/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 88/1 onto dev r1z1-10.1.1.1/sdd Placed 88/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 88/2 onto dev r1z1-10.1.1.1/sda Placed 88/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 37/0 onto dev r1z1-10.1.1.1/sdc Placed 37/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 37/1 onto dev r1z1-10.1.1.1/sdb Placed 37/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 37/2 onto dev r1z1-10.1.1.1/sdd Placed 37/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 66/0 onto dev r1z1-10.1.1.1/sda Placed 66/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 66/1 onto dev r1z1-10.1.1.1/sdc Placed 66/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 66/2 onto dev r1z1-10.1.1.1/sdb Placed 66/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 56/0 onto dev r1z1-10.1.1.1/sdd Placed 56/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 56/1 onto dev r1z1-10.1.1.1/sda Placed 56/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 56/2 onto dev r1z1-10.1.1.1/sdc Placed 56/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 200/0 onto dev r1z1-10.1.1.1/sdb Placed 200/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 200/1 onto dev r1z1-10.1.1.1/sdd Placed 200/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 200/2 onto dev r1z1-10.1.1.1/sda Placed 200/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 86/0 onto dev r1z1-10.1.1.1/sdc Placed 86/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 86/1 onto dev r1z1-10.1.1.1/sdb Placed 86/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 86/2 onto dev r1z1-10.1.1.1/sdd Placed 86/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 27/0 onto dev r1z1-10.1.1.1/sda Placed 27/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 27/1 onto dev r1z1-10.1.1.1/sdc Placed 27/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 27/2 onto dev r1z1-10.1.1.1/sdb Placed 27/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Assigned 768 parts Assigned 768 parts DEBUG: Finished rebalance plan after 1 attempts Finished rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 151 (Last start was 69) DEBUG: Gather start is 151 (Last start was 69) Gather start is 151 (Last start was 69) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 151 (Last start was 69) DEBUG: Gather start is 151 (Last start was 69) DEBUG: Gather start is 151 (Last start was 69) Gather start is 151 (Last start was 69) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults ... ok test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting ... ok test.unit.common.test_db.TestDatabaseBroker.test_commit_puts test.unit.common.test_db.TestDatabaseBroker.test_commit_puts ... ok test.unit.common.test_db.TestDatabaseBroker.test_delete_db test.unit.common.test_db.TestDatabaseBroker.test_delete_db ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate ... ok test.unit.common.test_db.TestDatabaseBroker.test_initialize test.unit.common.test_db.TestDatabaseBroker.test_initialize ... ok test.unit.common.test_db.TestDatabaseBroker.test_lock test.unit.common.test_db.TestDatabaseBroker.test_lock ... ok test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init ... ok test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata test.unit.common.test_db.TestDatabaseBroker.test_metadata ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_newid test.unit.common.test_db.TestDatabaseBroker.test_newid ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpz5en90v2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f2a612c3-6cd4-40bc-ac2d-e59990c823da-sda', '1717345783.05694', '1717345783', '1717345784', '', '1717345784', 0, 0) test DEBUG: /tmp/tmpz5en90v2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f2a612c3-6cd4-40bc-ac2d-e59990c823da-sda', '1717345783.05694', '1717345783', '1717345784', '', '1717345784', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f2a612c3-6cd4-40bc-ac2d-e59990c823da-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.54576 seconds (0.39281/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpz854t8eu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd3e68ac0-b7f8-4688-86e9-9bddc454cc10-sda', '1717345788.78846', '1717345790', '0', '', '1717345790', 0, 0) test DEBUG: /tmp/tmpz854t8eu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd3e68ac0-b7f8-4688-86e9-9bddc454cc10-sda', '1717345788.78846', '1717345790', '0', '', '1717345790', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d3e68ac0-b7f8-4688-86e9-9bddc454cc10-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04772 seconds (20.95660/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6vk1bb5n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '194572a838fae095d05c544a0e92c4ef', '5c2f2ded-685b-4f20-971d-20af031e5d6c-sda', '1717345788.86808', '1717345790', '0', '', '1717345790', 1, 1) test DEBUG: /tmp/tmp6vk1bb5n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '194572a838fae095d05c544a0e92c4ef', '5c2f2ded-685b-4f20-971d-20af031e5d6c-sda', '1717345788.86808', '1717345790', '0', '', '1717345790', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5c2f2ded-685b-4f20-971d-20af031e5d6c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp6vk1bb5n/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10305 seconds (9.70368/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp6vk1bb5n/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '61fd7e21fa6cfe9e780e67dcec25260b', '9f8b04f2-13e2-4184-907a-d4c73d99dc74-sda', '1717345788.96571', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9f8b04f2-13e2-4184-907a-d4c73d99dc74-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '61fd7e21fa6cfe9e780e67dcec25260b', '9f8b04f2-13e2-4184-907a-d4c73d99dc74-sda', '1717345788.96571', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9f8b04f2-13e2-4184-907a-d4c73d99dc74-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd306c320-428b-4248-b768-e24907b7593e-sda', '1717345789.01682', '1717345789.01345', '0', '', '1717345789.01345', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpai8p9wep/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '494d3100-1d97-4387-a1d6-7026c5abf107-sda', '1717345789.03304', '1717345789.02978', '0', '', '1717345789.02978', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('494d3100-1d97-4387-a1d6-7026c5abf107-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '494d3100-1d97-4387-a1d6-7026c5abf107-sda', '1717345789.03304', '1717345789.02978', '0', '', '1717345789.02978', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('494d3100-1d97-4387-a1d6-7026c5abf107-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.02621 seconds (38.15695/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '71db5295c6904513e553eec39da7d97f', '33dea391-154e-4622-a65a-8b4bd301b8cc-sda', '1717345789.08016', '1717345789.07685', '0', '', '1717345789.07685', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('33dea391-154e-4622-a65a-8b4bd301b8cc-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '089017d7b89c70cd1d501a7f63e819e9', '870373c3-339a-4269-9abc-e5b73e2042b6-sda', '1717345789.14105', '1717345789.13779', '0', '', '1717345789.13779', 11, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [412] (txn: tx2eebc3f793ff4be480d15-00665c9dfb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending enough data (txn: tx020961832b0641b4ba6b8-00665c9dfb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1795, in _send_chunk self.conn.send(to_send) File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 926, in send self.give_send(self, data) File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1426, in capture_send raise exc Exception: Exception while sending data on connection (txn: tx06149499c6c84f25b9b25-00665c9dfb) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: ChunkWriteTimeout (Nones) (txn: tx34e4f535a0344fcd9afe1-00665c9dfb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: txa85f7ba0f8ef4a3f8a52e-00665c9dfb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test DEBUG: tenant mismatch: BLAN_a != tenant_id test DEBUG: allow user with role(s) admin,swiftoperator as account admin test DEBUG: allow user with role(s) admin,swiftoperator as account admin test DEBUG: allowing proxy 127.0.0.1 for container-sync test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: Names allowed in acls. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL: allowme authorizing test DEBUG: Names allowed in acls. test DEBUG: user *:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:* allowed in ACL authorizing. test DEBUG: User tenant_name:user_name is not allowed to delete its own account test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Using identity: {'user': ('user_id', 'user_name'), 'tenant': ('test', 'tenant_name'), 'roles': ['reselleradmin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: User test has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. Quarantined /tmp/tmpq5le2qsl/dev/dbs/par/pre/db to /tmp/tmpq5le2qsl/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmpq5le2qsl/dev/dbs/par/pre/db to /tmp/tmpq5le2qsl/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmpq5le2qsl/dev/dbs/par/pre/db to /tmp/tmpq5le2qsl/dev/quarantined/tests/db due to corrupted database Quarantined /tmp/tmpq5le2qsl/dev/dbs/par/pre/db to /tmp/tmpq5le2qsl/dev/quarantined/tests/db due to disk error while accessing database test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors ... ok test.unit.common.test_db.TestDatabaseBroker.test_put_record test.unit.common.test_db.TestDatabaseBroker.test_put_record ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok Quarantined /tmp/tmpir6oasay/dev/dbs/par/pre/db to /tmp/tmpir6oasay/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info ... ok test.unit.common.test_db.TestDatabaseBroker.test_skip_commits test.unit.common.test_db.TestDatabaseBroker.test_skip_commits ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type ... ok Quarantined /tmp/tmpxyb23tk3/dev/dbs/par/pre/db to /tmp/tmpxyb23tk3/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.common.test_swob.TestAccept.test_accept_invalid test.unit.common.test_swob.TestAccept.test_accept_invalid ... ok test.unit.common.test_swob.TestAccept.test_accept_json test.unit.common.test_swob.TestAccept.test_accept_json ... ok test.unit.common.test_swob.TestAccept.test_accept_plain test.unit.common.test_swob.TestAccept.test_accept_plain ... ok test.unit.common.test_swob.TestAccept.test_accept_xml test.unit.common.test_swob.TestAccept.test_accept_xml ... ok test.unit.common.test_swob.TestAccept.test_repr test.unit.common.test_swob.TestAccept.test_repr ... ok test.unit.common.test_swob.TestMatch.test_match test.unit.common.test_swob.TestMatch.test_match ... ok test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_noquote test.unit.common.test_swob.TestMatch.test_match_noquote ... ok test.unit.common.test_swob.TestMatch.test_match_star test.unit.common.test_swob.TestMatch.test_match_star ... ok test.unit.common.test_swob.TestStatusMap.test_status_map test.unit.common.test_swob.TestStatusMap.test_status_map ... ok test.unit.common.test_utils.TestCooperativeIterator.test_close test.unit.common.test_utils.TestCooperativeIterator.test_close ... ok test.unit.common.test_utils.TestCooperativeIterator.test_init test.unit.common.test_utils.TestCooperativeIterator.test_init ... ok test.unit.common.test_utils.TestCooperativeIterator.test_iter test.unit.common.test_utils.TestCooperativeIterator.test_iter ... ok test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps ... ok test.unit.common.test_utils.TestCooperativeIterator.test_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_sleeps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345777.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345781.83531', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info test.unit.common.middleware.test_container_sync.TestContainerSync.test_info ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:49 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o_0', 'created_at': '1717345789.15758', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c/o_1', 'created_at': '1717345789.15852', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c/o_2', 'created_at': '1717345789.15883', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c/o_3', 'created_at': '1717345789.15910', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c/o_4', 'created_at': '1717345789.15937', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c/o_5', 'created_at': '1717345789.15962', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c/o_6', 'created_at': '1717345789.15988', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c/o_7', 'created_at': '1717345789.16013', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c/o_8', 'created_at': '1717345789.16038', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c/o_9', 'created_at': '1717345789.16063', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing', 'created_at': '1717345789.16088', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '870373c3-339a-4269-9abc-e5b73e2042b6-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpag2l617l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fee7f5c1-29c8-4106-b816-3c75d6370c16-sda', '1717345791.71042', '1717345794', '0', '', '1717345794', 0, 1) test DEBUG: /tmp/tmpag2l617l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fee7f5c1-29c8-4106-b816-3c75d6370c16-sda', '1717345791.71042', '1717345794', '0', '', '1717345794', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fee7f5c1-29c8-4106-b816-3c75d6370c16-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04780 seconds (20.91897/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp977s2g18/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f79a5e45f4b1ca2f05e1bba2f5ba8b90', 'baffb4db-d7be-44a9-a449-0090fbff8b50-sda', '1717345791.80738', '1717345794', '0', '', '1717345794', 1, 1) test DEBUG: /tmp/tmp977s2g18/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345795.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'baffb4db-d7be-44a9-a449-0090fbff8b50-sda') test DEBUG: /tmp/tmp977s2g18/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0172348s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f79a5e45f4b1ca2f05e1bba2f5ba8b90', 'baffb4db-d7be-44a9-a449-0090fbff8b50-sda', '1717345791.80738', '1717345794', '0', '', '1717345794', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('baffb4db-d7be-44a9-a449-0090fbff8b50-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp977s2g18/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.19032 seconds (5.25438/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp977s2g18/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e5282c216a82fd68b54af8373b020694', 'b3dc0a23-d381-4a80-99e0-5032e8995baf-sda', '1717345791.91004', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('b3dc0a23-d381-4a80-99e0-5032e8995baf-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e5282c216a82fd68b54af8373b020694', 'b3dc0a23-d381-4a80-99e0-5032e8995baf-sda', '1717345791.91004', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('b3dc0a23-d381-4a80-99e0-5032e8995baf-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1bg7o2o5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '709c8210-8de3-4339-b4b2-810637dfab53-sda', '1717345792.10145', '1717345795', '0', '', '1717345795', 0, 1) test DEBUG: /tmp/tmp1bg7o2o5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '709c8210-8de3-4339-b4b2-810637dfab53-sda', '1717345792.10145', '1717345795', '0', '', '1717345795', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('709c8210-8de3-4339-b4b2-810637dfab53-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04491 seconds (22.26626/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp4xbc1g7m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f943608dbfae930c5211851e8ba2abf2', '76d7db61-52d5-4d7d-8295-76350e28cba3-sda', '1717345792.19001', '1717345795', '0', '', '1717345795', 1, 1) test DEBUG: /tmp/tmp4xbc1g7m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f943608dbfae930c5211851e8ba2abf2', '76d7db61-52d5-4d7d-8295-76350e28cba3-sda', '1717345792.19001', '1717345795', '0', '', '1717345795', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('76d7db61-52d5-4d7d-8295-76350e28cba3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test ERROR: Invalid current //REALM/CLUSTER (foo) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get test.unit.common.test_internal_client.TestSimpleClient.test_get ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param ... ok test.unit.common.test_internal_client.TestSimpleClient.test_head test.unit.common.test_internal_client.TestSimpleClient.test_head ... ok test.unit.common.test_internal_client.TestSimpleClient.test_proxy test.unit.common.test_internal_client.TestSimpleClient.test_proxy ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError ... ok test.unit.common.test_splice.TestSplice.test_available test.unit.common.test_splice.TestSplice.test_available ... ok test.unit.common.test_splice.TestSplice.test_errno test.unit.common.test_splice.TestSplice.test_errno ... ok test.unit.common.test_splice.TestSplice.test_fileno test.unit.common.test_splice.TestSplice.test_fileno ... ok test.unit.common.test_splice.TestSplice.test_flags test.unit.common.test_splice.TestSplice.test_flags ... ok test.unit.common.test_splice.TestSplice.test_flags_list test.unit.common.test_splice.TestSplice.test_flags_list ... ok test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe ... ok test.unit.common.test_splice.TestSplice.test_unavailable test.unit.common.test_splice.TestSplice.test_unavailable ... ok test.unit.common.test_splice.TestSplice.test_unavailable_in_libc test.unit.common.test_splice.TestSplice.test_unavailable_in_libc ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_del test.unit.common.test_swob.TestHeaderEnvironProxy.test_del ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy ... ok test.unit.common.test_utils.ResellerConfReader.test_defaults test.unit.common.test_utils.ResellerConfReader.test_defaults ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers ... ok test.unit.common.test_utils.ResellerConfReader.test_same_as_default test.unit.common.test_utils.ResellerConfReader.test_same_as_default ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf ... ok test.unit.common.test_utils.ResellerConfReader.test_stray_comma test.unit.common.test_utils.ResellerConfReader.test_stray_comma ... ok test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options ... ok test.unit.common.test_utils.TestContextPool.test_close test.unit.common.test_utils.TestContextPool.test_close ... ok test.unit.common.test_utils.TestContextPool.test_context_manager test.unit.common.test_utils.TestContextPool.test_context_manager ... ok test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided ... ok test.unit.common.test_utils.TestDistributeEvenly.test_just_one test.unit.common.test_utils.TestDistributeEvenly.test_just_one ... ok test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items ... ok test.unit.common.test_utils.TestDistributeEvenly.test_uneven test.unit.common.test_utils.TestDistributeEvenly.test_uneven ... ok test.unit.common.test_utils.TestNamespace.test_bounds_serialization test.unit.common.test_utils.TestNamespace.test_bounds_serialization ... ok test.unit.common.test_utils.TestNamespace.test_comparisons test.unit.common.test_utils.TestNamespace.test_comparisons ... ok test.unit.common.test_utils.TestNamespace.test_contains test.unit.common.test_utils.TestNamespace.test_contains ... ok test.unit.common.test_utils.TestNamespace.test_end_marker test.unit.common.test_utils.TestNamespace.test_end_marker ... ok test.unit.common.test_utils.TestNamespace.test_entire_namespace test.unit.common.test_utils.TestNamespace.test_entire_namespace ... ok test.unit.common.test_utils.TestNamespace.test_expand test.unit.common.test_utils.TestNamespace.test_expand ... ok test.unit.common.test_utils.TestNamespace.test_includes test.unit.common.test_utils.TestNamespace.test_includes ... ok test.unit.common.test_utils.TestNamespace.test_lower_setter test.unit.common.test_utils.TestNamespace.test_lower_setter ... ok test.unit.common.test_utils.TestNamespace.test_name test.unit.common.test_utils.TestNamespace.test_name ... ok test.unit.common.test_utils.TestNamespace.test_name_unexpected_format test.unit.common.test_utils.TestNamespace.test_name_unexpected_format ... ok test.unit.common.test_utils.TestNamespace.test_total_ordering test.unit.common.test_utils.TestNamespace.test_total_ordering ... ok test.unit.common.test_utils.TestNamespace.test_unicode_name test.unit.common.test_utils.TestNamespace.test_unicode_name ... ok test.unit.common.test_utils.TestNamespace.test_upper_setter test.unit.common.test_utils.TestNamespace.test_upper_setter ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_closes test.unit.common.test_utils.TestReiterate.test_reiterate_closes ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple ... ok test.unit.common.test_utils.TestShardRangeList.test_filter test.unit.common.test_utils.TestShardRangeList.test_filter ... ok test.unit.common.test_utils.TestShardRangeList.test_find_lower test.unit.common.test_utils.TestShardRangeList.test_find_lower ... ok test.unit.common.test_utils.TestShardRangeList.test_includes test.unit.common.test_utils.TestShardRangeList.test_includes ... ok test.unit.common.test_utils.TestShardRangeList.test_init test.unit.common.test_utils.TestShardRangeList.test_init ... ok test.unit.common.test_utils.TestShardRangeList.test_init_with_list test.unit.common.test_utils.TestShardRangeList.test_init_with_list ... ok test.unit.common.test_utils.TestShardRangeList.test_pop test.unit.common.test_utils.TestShardRangeList.test_pop ... ok test.unit.common.test_utils.TestShardRangeList.test_slice test.unit.common.test_utils.TestShardRangeList.test_slice ... ok test.unit.common.test_utils.TestShardRangeList.test_states test.unit.common.test_utils.TestShardRangeList.test_states ... ok test.unit.common.test_utils.TestShardRangeList.test_timestamps test.unit.common.test_utils.TestShardRangeList.test_timestamps ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345790.22642 <= '409 (1717345790.22642), 409 (1717345790.22642), 409 (1717345790.22642)' (txn: txe0e6a8555f224ba585817-00665c9dfe) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345790.23213 <= '409 (1717345790.23213), 409 (1717345790.23213), 409 (1717345790.23213)' (txn: tx4cd44cb276c04edb93cdf-00665c9dfe) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345790.23626 <= '409 (1717345790.23626), 409 (1717345790.23626), 409 (1717345790.23626)' (txn: tx91c902bedbab469fad0e7-00665c9dfe) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345793.00000 <= '409 (1717345794.00000), 409 (1717345794.00000), 409 (1717345794.00000)' (txn: txd4dce8e7812a44f9b3de5-00665c9e01) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345795.00000 <= '409 (1717345796.00000), 409 (1717345796.00000), 409 (1717345796.00000)' (txn: tx04de978ad72e4878a8c36-00665c9e01) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345797.00000 <= '409 (1717345798.00000), 409 (1717345798.00000), 409 (1717345798.00000)' (txn: tx149e8b4ac6d54f2ea106d-00665c9e01) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345793.00000 <= '409 (1717345793.00000), 409 (1717345793.00000), 409 (1717345793.00000)' (txn: tx21ecdf9f1acf4d57aded8-00665c9e01) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345794.00000 <= '409 (1717345794.00000), 409 (1717345794.00000), 409 (1717345794.00000)' (txn: txfb78dd01cb1a4582a7f1f-00665c9e01) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345795.00000 <= '409 (1717345795.00000), 409 (1717345795.00000), 409 (1717345795.00000)' (txn: tx9327f3e6cd2a4277aa474-00665c9e01) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345793.00000 <= '409 (1717345793.00000)' (txn: tx9e23c22da4714866a1fba-00665c9e01) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345794.00000 <= '409 (1717345794.00000)' (txn: tx5659bd688f5a4d1ea37ab-00665c9e01) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345795.00000 <= '409 (1717345795.00000)' (txn: txd79ebad063ac4acd83082-00665c9e01) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345784.81269', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345784.82977', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345788.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345788.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345788.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345788.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txb8d4265cbdcd4d5585712-00665c9e07) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345799.00000 <= '409 (1717345800.00000)' (txn: txe64e451a590c4e1fa3c06-00665c9e07) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [409, 412, 412] (txn: tx19e03ca5607544ce8843d-00665c9e07) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345799.00000 <= '409 (unknown), 409 (unknown), 409 (unknown)' (txn: txfbe5906c84ee483c9ac41-00665c9e07) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717345799.00000 <= '409 (1717345800.00000), 201 (unknown), 201 (unknown)' (txn: tx5f4ac2d5a0d54d3cae1d5-00665c9e07) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden ... ok test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp4xbc1g7m/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.60384 seconds (0.38405/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp4xbc1g7m/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e39d8a1f32ba3b88b89aba079540883c', '47a2b4bb-12ed-4835-99f8-d3e126392442-sda', '1717345792.27147', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('47a2b4bb-12ed-4835-99f8-d3e126392442-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e39d8a1f32ba3b88b89aba079540883c', '47a2b4bb-12ed-4835-99f8-d3e126392442-sda', '1717345792.27147', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('47a2b4bb-12ed-4835-99f8-d3e126392442-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpq5oqctek/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e5eaee93-b1a1-4168-b376-8ddeb75be78f-sda', '1717345797.22777', '1717345797', '1717345801', '', '1717345801', 0, 0) test DEBUG: /tmp/tmpq5oqctek/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e5eaee93-b1a1-4168-b376-8ddeb75be78f-sda', '1717345797.22777', '1717345797', '1717345801', '', '1717345801', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e5eaee93-b1a1-4168-b376-8ddeb75be78f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04869 seconds (20.53661/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp476bguaj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '43892a13b4935f6558328ca569f63382', 'ec31227f-b895-4d98-a85c-e9dabeeb3117-sda', '1717345797.33190', '1717345797', '1717345801', '', '1717345801', 1, 0) test DEBUG: /tmp/tmp476bguaj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345802.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'ec31227f-b895-4d98-a85c-e9dabeeb3117-sda') test DEBUG: /tmp/tmp476bguaj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0116396s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '43892a13b4935f6558328ca569f63382', 'ec31227f-b895-4d98-a85c-e9dabeeb3117-sda', '1717345797.33190', '1717345797', '1717345801', '', '1717345801', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ec31227f-b895-4d98-a85c-e9dabeeb3117-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp476bguaj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09923 seconds (10.07739/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp476bguaj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '3cd5e910c7ab5ff9c049cc3a5d05ff8c', '378cd6e5-eb7c-4548-b9ea-063212b9f596-sda', '1717345797.46536', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('378cd6e5-eb7c-4548-b9ea-063212b9f596-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '3cd5e910c7ab5ff9c049cc3a5d05ff8c', '378cd6e5-eb7c-4548-b9ea-063212b9f596-sda', '1717345797.46536', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('378cd6e5-eb7c-4548-b9ea-063212b9f596-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_spigsis/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd130dff8-1584-446d-9840-cbea9c18ce7e-sda', '1717345797.54011', '1717345797', '1717345801', '', '1717345801', 0, 1) test DEBUG: /tmp/tmp_spigsis/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd130dff8-1584-446d-9840-cbea9c18ce7e-sda', '1717345797.54011', '1717345797', '1717345801', '', '1717345801', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d130dff8-1584-446d-9840-cbea9c18ce7e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04259 seconds (23.48219/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkmasg1zr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '43892a13b4935f6558328ca569f63382', 'ea52b26d-0d09-4655-8d53-a01976da9152-sda', '1717345797.65281', '1717345797', '1717345801', '', '1717345801', 1, 1) test DEBUG: /tmp/tmpkmasg1zr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '43892a13b4935f6558328ca569f63382', 'ea52b26d-0d09-4655-8d53-a01976da9152-sda', '1717345797.65281', '1717345797', '1717345801', '', '1717345801', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ea52b26d-0d09-4655-8d53-a01976da9152-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpkmasg1zr/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09382 seconds (10.65915/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpkmasg1zr/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345790.23177', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:29:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345790.25123', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission ... ok test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test-ic DEBUG: -> 2014-05-27T20:54:11 GET http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - test-ic DEBUG: -> 2014-05-27T20:54:11 HEAD http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test INFO: Client disconnected on read of '/a/c' (txn: txbb49d3ae0a5e4aaf9d2ce-00665c9e0c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx1d98406a773a42a78c933-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx1d98406a773a42a78c933-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx1d98406a773a42a78c933-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx1d98406a773a42a78c933-00665c9e0c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: tx6074dc4d9f2e4e10bbd0e-00665c9e0c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: txb6b98274557744feb1a8a-00665c9e0c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx19d15d011748498889926-00665c9e0c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx2e81a4ccae124085b3854-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: tx6aa9e805e0ae42979ee24-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx1e660bfea09c4d40b19f8-00665c9e0c) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx74a6d8e4c5fe410886bab-00665c9e0c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: tx9b13b6551fce4b81a739d-00665c9e0c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: txa534692c940a4d2d9c5fb-00665c9e0c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx5567ef8932b04e13b1652-00665c9e0c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx65875a458ccc49e1a3750-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: txd649450618f0452bba333-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx07991596b68e4b4b85d3f-00665c9e0c) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx4a541c3a96e8449d8527c-00665c9e0c) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: tx650189d6a1a4496fac829-00665c9e0c) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txe7d6d253ee7d42238631a-00665c9e0c) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: txfbd6fd5e6fac46c29932b-00665c9e0c) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345791.44092', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:59 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed ... ok test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion ... ok test.unit.common.ring.test_ring.TestRingData.test_attrs test.unit.common.ring.test_ring.TestRingData.test_attrs ... ok test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization ... ok test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data ... ok test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization ... ok test.unit.common.ring.test_ring.TestRingData.test_load_closes_file test.unit.common.ring.test_ring.TestRingData.test_load_closes_file ... ok test.unit.common.ring.test_ring.TestRingData.test_permissions test.unit.common.ring.test_ring.TestRingData.test_permissions ... ok test.unit.common.ring.test_ring.TestRingData.test_replica_count test.unit.common.ring.test_ring.TestRingData.test_replica_count ... ok test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization ... ok test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval ... ok test.unit.common.test_container_sync_realms.TestUtils.test_empty test.unit.common.test_container_sync_realms.TestUtils.test_empty ... ok test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm ... ok test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing ... ok test.unit.common.test_container_sync_realms.TestUtils.test_get_sig test.unit.common.test_container_sync_realms.TestUtils.test_get_sig ... ok test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there ... ok test.unit.common.test_container_sync_realms.TestUtils.test_one_realm test.unit.common.test_container_sync_realms.TestUtils.test_one_realm ... ok test.unit.common.test_container_sync_realms.TestUtils.test_os_error test.unit.common.test_container_sync_realms.TestUtils.test_os_error ... ok test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default ... ok test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers ... ok test.unit.common.test_swob.TestResponse.test_507 test.unit.common.test_swob.TestResponse.test_507 ... ok test.unit.common.test_swob.TestResponse.test_app_iter test.unit.common.test_swob.TestResponse.test_app_iter ... ok test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk ... ok test.unit.common.test_swob.TestResponse.test_call_preserves_closeability test.unit.common.test_swob.TestResponse.test_call_preserves_closeability ... ok test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary ... ok test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called ... ok test.unit.common.test_swob.TestResponse.test_charset test.unit.common.test_swob.TestResponse.test_charset ... ok test.unit.common.test_swob.TestResponse.test_charset_content_type test.unit.common.test_swob.TestResponse.test_charset_content_type ... ok test.unit.common.test_swob.TestResponse.test_content_type test.unit.common.test_swob.TestResponse.test_content_type ... ok test.unit.common.test_swob.TestResponse.test_empty_body test.unit.common.test_swob.TestResponse.test_empty_body ... ok test.unit.common.test_swob.TestResponse.test_etag test.unit.common.test_swob.TestResponse.test_etag ... ok test.unit.common.test_swob.TestResponse.test_host_url_default test.unit.common.test_swob.TestResponse.test_host_url_default ... ok test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_https test.unit.common.test_swob.TestResponse.test_host_url_host_https ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_override test.unit.common.test_swob.TestResponse.test_host_url_host_override ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_port_override test.unit.common.test_swob.TestResponse.test_host_url_host_port_override ... ok test.unit.common.test_swob.TestResponse.test_host_url_https test.unit.common.test_swob.TestResponse.test_host_url_https ... ok test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched ... ok test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite test.unit.common.test_swob.TestResponse.test_location_no_rewrite ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite test.unit.common.test_swob.TestResponse.test_location_rewrite ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host ... ok test.unit.common.test_swob.TestResponse.test_multi_range_body test.unit.common.test_swob.TestResponse.test_multi_range_body ... ok test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges ... ok test.unit.common.test_swob.TestResponse.test_multi_response_iter test.unit.common.test_swob.TestResponse.test_multi_response_iter ... ok test.unit.common.test_swob.TestResponse.test_properties test.unit.common.test_swob.TestResponse.test_properties ... ok test.unit.common.test_swob.TestResponse.test_range_body test.unit.common.test_swob.TestResponse.test_range_body ... ok test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range ... ok test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter ... ok test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter ... ok test.unit.common.test_swob.TestResponse.test_unicode_body test.unit.common.test_swob.TestResponse.test_unicode_body ... ok test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission ... ok REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '9faca47df24c68c1b4bda38e6056a577', '09b742eb-6904-4648-955d-73ecb786b089-sda', '1717345797.76905', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('09b742eb-6904-4648-955d-73ecb786b089-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '9faca47df24c68c1b4bda38e6056a577', '09b742eb-6904-4648-955d-73ecb786b089-sda', '1717345797.76905', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('09b742eb-6904-4648-955d-73ecb786b089-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpzu0u1pn6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'cf26b397-a4d5-45fd-8fc3-25af3b388768-sda', '1717345801.59933', '1717345801', '0', '', '1717345801', 0, 1) test DEBUG: /tmp/tmpzu0u1pn6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'cf26b397-a4d5-45fd-8fc3-25af3b388768-sda', '1717345801.59933', '1717345801', '0', '', '1717345801', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('cf26b397-a4d5-45fd-8fc3-25af3b388768-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.03016 seconds (33.16142/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6d63hh9k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1bd4b802ece07755fe1089b3936136e4', 'a4e23b0a-53b1-4f06-9ab1-dba5650e77cd-sda', '1717345801.66850', '1717345801', '0', '', '1717345801', 1, 0) test DEBUG: /tmp/tmp6d63hh9k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345805.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'a4e23b0a-53b1-4f06-9ab1-dba5650e77cd-sda') test DEBUG: /tmp/tmp6d63hh9k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00881553s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1bd4b802ece07755fe1089b3936136e4', 'a4e23b0a-53b1-4f06-9ab1-dba5650e77cd-sda', '1717345801.66850', '1717345801', '0', '', '1717345801', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a4e23b0a-53b1-4f06-9ab1-dba5650e77cd-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp6d63hh9k/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06790 seconds (14.72800/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp6d63hh9k/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57320d23aecdbc5759dff37f31ee9f4a', '92585be3-6acd-4347-80ae-77b1022e7824-sda', '1717345801.75125', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('92585be3-6acd-4347-80ae-77b1022e7824-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57320d23aecdbc5759dff37f31ee9f4a', '92585be3-6acd-4347-80ae-77b1022e7824-sda', '1717345801.75125', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('92585be3-6acd-4347-80ae-77b1022e7824-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp3gylcg2g/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9b3cdcde-f63d-4a1b-8289-226e216e9556-sda', '1717345801.80712', '1717345801', '0', '', '1717345801', 0, 1) test DEBUG: /tmp/tmp3gylcg2g/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9b3cdcde-f63d-4a1b-8289-226e216e9556-sda', '1717345801.80712', '1717345801', '0', '', '1717345801', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('9b3cdcde-f63d-4a1b-8289-226e216e9556-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05164 seconds (19.36452/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgjmqmw_q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1bd4b802ece07755fe1089b3936136e4', '37ddf449-062c-4a07-a56a-c776a9b09a37-sda', '1717345801.90325', '1717345801', '0', '', '1717345801', 1, 0) test DEBUG: /tmp/tmpgjmqmw_q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1bd4b802ece07755fe1089b3936136e4', '37ddf449-062c-4a07-a56a-c776a9b09a37-sda', '1717345801.90325', '1717345801', '0', '', '1717345801', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('37ddf449-062c-4a07-a56a-c776a9b09a37-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpgjmqmw_q/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09504 seconds (10.52208/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpgjmqmw_q/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57320d23aecdbc5759dff37f31ee9f4a', '06197b04-4ea6-4ae8-bee0-5cd9430f6b28-sda', '1717345802.01437', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('06197b04-4ea6-4ae8-bee0-5cd9430f6b28-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57320d23aecdbc5759dff37f31ee9f4a', '06197b04-4ea6-4ae8-bee0-5cd9430f6b28-sda', '1717345802.01437', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('06197b04-4ea6-4ae8-bee0-5cd9430f6b28-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:29:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.47032', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim ... ok proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp4zy6p3fb/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp4zy6p3fb/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp4zy6p3fb/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp4zy6p3fb/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:05 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0129 "-" 3543386 - HTTP PERF: 0.01790 seconds to PUT 127.0.0.1:38727 /sda1/0/a) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:07 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0135 "-" 3543386 - HTTP PERF: 0.01641 seconds to PUT 127.0.0.1:52887 /sdb1/0/a) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:07 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0116 "-" 3543386 - HTTP PERF: 0.01443 seconds to PUT 127.0.0.1:38727 /sda1/2/a1) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:07 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0120 "-" 3543386 - HTTP PERF: 0.01455 seconds to PUT 127.0.0.1:52887 /sdb1/2/a1) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:07 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx4f50eac748784eaf821ab-00665c9e0f" "proxy-server 3543386" 0.0023 "-" 3543386 - HTTP PERF: 0.00462 seconds to HEAD 127.0.0.1:38727 /sda1/0/a?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:54491/sdb1/3/a/c" "tx4f50eac748784eaf821ab-00665c9e0f" "container-server 3543386" 0.0021 "-" 3543386 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:59505/sda1/3/a/c" "tx4f50eac748784eaf821ab-00665c9e0f" "container-server 3543386" 0.0024 "-" 3543386 0 HTTP PERF: 0.02468 seconds to PUT 127.0.0.1:38727 /sda1/0/a/c) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx4f50eac748784eaf821ab-00665c9e0f" "proxy-server 3543386" 0.0442 "-" 3543386 0 HTTP PERF: 0.01047 seconds to PUT 127.0.0.1:52887 /sdb1/0/a/c) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx4f50eac748784eaf821ab-00665c9e0f" "proxy-server 3543386" 0.0254 "-" 3543386 0 HTTP PERF: 0.04899 seconds to PUT 127.0.0.1:59505 /sda1/3/a/c) HTTP PERF: 0.05047 seconds to PUT 127.0.0.1:54491 /sdb1/3/a/c) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/10 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx4f50eac748784eaf821ab-00665c9e0f - 2.6572 - - 1717345807.799688339 1717345810.456896544 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx41971dd10db74ae6b776b-00665c9e12" "proxy-server 3543386" 0.0023 "-" 3543386 - HTTP PERF: 0.00485 seconds to HEAD 127.0.0.1:38727 /sda1/2/a1?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:54491/sdb1/0/a1/c1" "tx41971dd10db74ae6b776b-00665c9e12" "container-server 3543386" 0.0022 "-" 3543386 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:59505/sda1/0/a1/c1" "tx41971dd10db74ae6b776b-00665c9e12" "container-server 3543386" 0.0023 "-" 3543386 0 HTTP PERF: 0.02311 seconds to PUT 127.0.0.1:38727 /sda1/2/a1/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx41971dd10db74ae6b776b-00665c9e12" "proxy-server 3543386" 0.0377 "-" 3543386 0 HTTP PERF: 0.01016 seconds to PUT 127.0.0.1:52887 /sdb1/2/a1/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx41971dd10db74ae6b776b-00665c9e12" "proxy-server 3543386" 0.0238 "-" 3543386 0 HTTP PERF: 0.04172 seconds to PUT 127.0.0.1:54491 /sdb1/0/a1/c1) HTTP PERF: 0.04284 seconds to PUT 127.0.0.1:59505 /sda1/0/a1/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/10 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx41971dd10db74ae6b776b-00665c9e12 - 0.0530 - - 1717345810.459432364 1717345810.512476683 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa6ae9492472647598ec0d-00665c9e12" "proxy-server 3543386" 0.0092 "-" 3543386 - HTTP PERF: 0.01175 seconds to HEAD 127.0.0.1:52887 /sdb1/0/a?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:54491/sdb1/3/a/c1" "txa6ae9492472647598ec0d-00665c9e12" "container-server 3543386" 0.0056 "-" 3543386 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:59505/sda1/3/a/c1" "txa6ae9492472647598ec0d-00665c9e12" "container-server 3543386" 0.0104 "-" 3543386 1 HTTP PERF: 0.02068 seconds to PUT 127.0.0.1:38727 /sda1/0/a/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa6ae9492472647598ec0d-00665c9e12" "proxy-server 3543386" 0.0344 "-" 3543386 1 HTTP PERF: 0.03662 seconds to PUT 127.0.0.1:52887 /sdb1/0/a/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa6ae9492472647598ec0d-00665c9e12" "proxy-server 3543386" 0.0495 "-" 3543386 1 HTTP PERF: 0.05287 seconds to PUT 127.0.0.1:59505 /sda1/3/a/c1) HTTP PERF: 0.05319 seconds to PUT 127.0.0.1:54491 /sdb1/3/a/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/10 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txa6ae9492472647598ec0d-00665c9e12 - 0.0711 - - 1717345810.514885664 1717345810.585950136 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx5c1f08be4c2041ab9c5d2-00665c9e12" "proxy-server 3543386" 0.0085 "-" 3543386 - HTTP PERF: 0.01130 seconds to HEAD 127.0.0.1:38727 /sda1/0/a?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:54491/sdb1/0/a/c2" "tx5c1f08be4c2041ab9c5d2-00665c9e12" "container-server 3543386" 0.0073 "-" 3543386 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:59505/sda1/0/a/c2" "tx5c1f08be4c2041ab9c5d2-00665c9e12" "container-server 3543386" 0.0017 "-" 3543386 2 HTTP PERF: 0.02634 seconds to PUT 127.0.0.1:52887 /sdb1/0/a/c2) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx5c1f08be4c2041ab9c5d2-00665c9e12" "proxy-server 3543386" 0.0391 "-" 3543386 2 HTTP PERF: 0.01478 seconds to PUT 127.0.0.1:38727 /sda1/0/a/c2) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:10 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx5c1f08be4c2041ab9c5d2-00665c9e12" "proxy-server 3543386" 0.0273 "-" 3543386 2 HTTP PERF: 0.04344 seconds to PUT 127.0.0.1:54491 /sdb1/0/a/c2) HTTP PERF: 0.04463 seconds to PUT 127.0.0.1:59505 /sda1/0/a/c2) test.unit.proxy.test_server.TestContainerController.test_CORS_valid test.unit.proxy.test_server.TestContainerController.test_CORS_valid ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE test.unit.proxy.test_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_GET_no_content test.unit.proxy.test_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_OPTIONS test.unit.proxy.test_server.TestContainerController.test_OPTIONS ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. fake INFO: - - 02/Jun/2024/16/30/13 GET /v1/a/c/o%3Ftemp_url_sig%3Ddeadbeefdeadbeef...%26temp_url_expires%3D1717346813 HTTP/1.0 404 - - - - - - - - 0.0002 - - 1717345813.380563974 1717345813.380790472 - test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test ERROR: Error in '/tmp/tmptqdm0p78/container-sync-realms.conf' with mtime_check_interval: could not convert string to float: 'invalid' test ERROR: Could not load '/tmp/tmp8tedqnxs/container-sync-realms.conf': File contains no section headers. file: '/tmp/tmp8tedqnxs/container-sync-realms.conf', line: 1 'invalid' test DEBUG: Could not load 'dc0d1c497b3d47ef807738c41cc7914d': [Errno 2] No such file or directory: 'dc0d1c497b3d47ef807738c41cc7914d' test ERROR: Could not load '/tmp/tmp8ys3zl7q/container-sync-realms.conf': [Errno 13] Permission denied: '/tmp/tmp8ys3zl7q/container-sync-realms.conf' test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Unexpected response while deleting object account container obj: 404 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 412 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 500 test-expirer DEBUG: b'' test-expirer ERROR: Exception while deleting object account container obj : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer ERROR: Exception while deleting object .expiring_objects 1717259432 1717259432-a0/c0/o0 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717259432 1717259432-a1/c1/o1 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717259432 1717259432-a2/c2/o2 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717259432 1717259432-a3/c3/o3 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717259432 1717259432-a4/c4/o4 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345831 1717345831-a5/c5/o5 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init test.unit.obj.test_expirer.TestObjectExpirer.test_init ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_report test.unit.obj.test_expirer.TestObjectExpirer.test_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission ... ok test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00072 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp32bs8n8u/sda/fake/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00396 seconds (252.63209/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00059 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00061 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00031 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test ERROR: Problem finding shard upper in '/tmp/tmppb6y6h9s/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmppb6y6h9s/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpvfqdm6us/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpvfqdm6us/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.18248', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.18614', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.18945', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.19382', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.19705', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.20163', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.20583', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.20957', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.21272', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.22861', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.23318', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta ... ok test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd (txn: txf8eddd0f9c0a464aa0892-00665c9e0c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: txea94116d457846398ae29-00665c9e10) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txf079b88874ea4efc96e7d-00665c9e10) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx57876b20986a4724802f2-00665c9e10) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx79197645757c4550805b2-00665c9e12) test INFO: Client disconnected on read of '/a/c' (txn: tx8aff918ff84b47a185c05-00665c9e12) test INFO: Client disconnected on read of '/a/c' (txn: tx0a2f211665524559876ea-00665c9e13) test INFO: Client disconnected on read of '/a/c' (txn: tx0a2f211665524559876ea-00665c9e13) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE ... ok proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sda (txn: tx5d2fae40c6f947ca871da-00665c9de6) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5b95247f58374c068c030-00665c9dee) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txd75caf75c121482ba514f-00665c9dee) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txda5d55da2c4b42bb82180-00665c9dee) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx4528ad0cdef84497b4e83-00665c9dee) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txc41e84599a1f42e5a1c7d-00665c9dee) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345800.48723', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345803.16730', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy ... ok test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6b8y6xu3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f0d0a1aa-f0a1-4745-8275-4aeac93ea9fc-sda', '1717345806.36436', '1717345806', '1717345808', '', '1717345808', 0, 1) test DEBUG: /tmp/tmp6b8y6xu3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f0d0a1aa-f0a1-4745-8275-4aeac93ea9fc-sda', '1717345806.36436', '1717345806', '1717345808', '', '1717345808', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f0d0a1aa-f0a1-4745-8275-4aeac93ea9fc-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04445 seconds (22.49917/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpdi6vbg66/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2072a9bd7fd6ec7e2b1334845fe425eb', '1cd94827-041c-4580-a725-287d367d243c-sda', '1717345806.45914', '1717345806', '1717345808', '', '1717345808', 1, 1) test DEBUG: /tmp/tmpdi6vbg66/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345811.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '1cd94827-041c-4580-a725-287d367d243c-sda') test DEBUG: /tmp/tmpdi6vbg66/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00870776s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2072a9bd7fd6ec7e2b1334845fe425eb', '1cd94827-041c-4580-a725-287d367d243c-sda', '1717345806.45914', '1717345806', '1717345808', '', '1717345808', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('1cd94827-041c-4580-a725-287d367d243c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpdi6vbg66/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10356 seconds (9.65663/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpdi6vbg66/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'd1f3ba1f718bd44b518e32ed7cc0cfd6', '733629d0-7fe6-414a-b46d-fb8aa688262a-sda', '1717345806.58475', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('733629d0-7fe6-414a-b46d-fb8aa688262a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'd1f3ba1f718bd44b518e32ed7cc0cfd6', '733629d0-7fe6-414a-b46d-fb8aa688262a-sda', '1717345806.58475', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('733629d0-7fe6-414a-b46d-fb8aa688262a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpp7nqqnvp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f46b9da0-3ab3-416b-8438-fd23ae9fcf8e-sda', '1717345806.65629', '1717345806', '1717345808', '', '1717345808', 0, 0) test DEBUG: /tmp/tmpp7nqqnvp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f46b9da0-3ab3-416b-8438-fd23ae9fcf8e-sda', '1717345806.65629', '1717345806', '1717345808', '', '1717345808', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f46b9da0-3ab3-416b-8438-fd23ae9fcf8e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06069 seconds (16.47756/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpupxou1i3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2072a9bd7fd6ec7e2b1334845fe425eb', '8c53eef7-5e56-4283-898d-25f988544091-sda', '1717345806.77798', '1717345806', '1717345808', '', '1717345808', 1, 1) test DEBUG: /tmp/tmpupxou1i3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2072a9bd7fd6ec7e2b1334845fe425eb', '8c53eef7-5e56-4283-898d-25f988544091-sda', '1717345806.77798', '1717345806', '1717345808', '', '1717345808', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8c53eef7-5e56-4283-898d-25f988544091-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpupxou1i3/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10782 seconds (9.27466/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpupxou1i3/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'd1f3ba1f718bd44b518e32ed7cc0cfd6', 'e85d7076-5caa-47c5-b85a-209297f8a577-sda', '1717345806.90094', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('e85d7076-5caa-47c5-b85a-209297f8a577-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'd1f3ba1f718bd44b518e32ed7cc0cfd6', 'e85d7076-5caa-47c5-b85a-209297f8a577-sda', '1717345806.90094', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('e85d7076-5caa-47c5-b85a-209297f8a577-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpso_1ql6w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9d14c186-b910-45b8-82d0-7f8a1c0dd31f-sda', '1717345806.97489', '1717345809', '1717345808', '', '1717345809', 0, 1) test DEBUG: /tmp/tmpso_1ql6w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings ... ok proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/10 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx5c1f08be4c2041ab9c5d2-00665c9e12 - 0.0619 - - 1717345810.588556051 1717345810.650471926 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx985726392a3347198eac7-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx5a233efd6b4c4c48b3c8f-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx07d4fba43f17493a99157-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx07d4fba43f17493a99157-00665c9e12) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx07d4fba43f17493a99157-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx374621c463d44660971fa-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx374621c463d44660971fa-00665c9e12) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx374621c463d44660971fa-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: txb642ad7bf24a47f8a957a-00665c9e12) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txb642ad7bf24a47f8a957a-00665c9e12) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txcbd2a918132d45febbc1f-00665c9e12) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txcbd2a918132d45febbc1f-00665c9e12) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx545c7f5dcacd43b1b3a0c-00665c9e12) test INFO: Client disconnected on read of '/a/c' (txn: tx545c7f5dcacd43b1b3a0c-00665c9e12) test INFO: Client disconnected on read of '/a/c' (txn: txfc990d22a16e45e0bf7f0-00665c9e12) test INFO: Client disconnected on read of '/a/c' (txn: txfc990d22a16e45e0bf7f0-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx0f7cd38f316f4b7cb33d9-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txa88c5aa8754741028c6e6-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf1118f723a15418e99534-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txf1118f723a15418e99534-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf1118f723a15418e99534-00665c9e12) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: txf1118f723a15418e99534-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txd0e5eea5989d4e1f90fcb-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txd0e5eea5989d4e1f90fcb-00665c9e12) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txd0e5eea5989d4e1f90fcb-00665c9e12) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: txd0e5eea5989d4e1f90fcb-00665c9e12) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.proxy.test_server.TestContainerController.test_POST test.unit.proxy.test_server.TestContainerController.test_POST ... ok test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_server.TestContainerController.test_POST_metadata test.unit.proxy.test_server.TestContainerController.test_POST_metadata ... ok test-expirer ERROR: Exception while deleting object .expiring_objects 1717345831 1717345831-a6/c6/o6 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345831 1717345831-a7/c7/o7 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345831 1717345831-a8/c8/o8♡ TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717345831 1717345831-a9/c9/o9ø TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer ERROR: Unexcepted error handling task '1717259432-invalid0': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid0 test-expirer ERROR: Unexcepted error handling task '1717259432-invalid1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test-expirer INFO: Pass so far 0s; 0 objects expired test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0000', 'task_object': '0000-invalid0', 'delete_timestamp': 0000000000.00000_0000000000000000, 'target_path': 'invalid0', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0001', 'task_object': '0001-invalid1', 'delete_timestamp': 0000000001.00000_0000000000000000, 'target_path': 'invalid1', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0002', 'task_object': '0002-invalid2', 'delete_timestamp': 0000000002.00000_0000000000000000, 'target_path': 'invalid2', 'is_async_delete': False} test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 380, in run_forever self.run_once(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/test/unit/obj/test_expirer.py", line 907, in raise_exceptions raise Exception('exception %d' % raises[0]) Exception: exception 1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 330, in run_once self.swift.get_account_info(task_account) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'get_account_info' test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: Object returning 503 for [] (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Ignoring 404 from primary (txn: tx6175350df859462d9dfbf-00665c9dee) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: txe70f45f801494e4fb385a-00665c9e0b) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: txe70f45f801494e4fb385a-00665c9e0b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 0, Lengths: [490, 490, 490, 490, 490, 490, 490, 490, 490, 490]: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345808.29803', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T163008Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345808.31630', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash ... ok test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: txf139239810fe4dc49a6b0-00665c9e15) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: txf139239810fe4dc49a6b0-00665c9e15) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: txf139239810fe4dc49a6b0-00665c9e15) test ERROR: Account HEAD returning 503 for [] (txn: txf139239810fe4dc49a6b0-00665c9e15) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx0be4b0daad44457883e02-00665c9e1a) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx2ccf9d4dc54f406b821b6-00665c9e1a) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx2ccf9d4dc54f406b821b6-00665c9e1a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4d39fc93125342d5afe28-00665c9e1a) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa10eb35d86d04963a5c9c-00665c9e1a) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbc8f425f253f4c438bf0c-00665c9e1a) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx950bb19297ba4be6bdf21-00665c9e1a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx10b23619b3ce4e2191393-00665c9e1a) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx21c46a73b4d449ec9eeee-00665c9e1a) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx21c46a73b4d449ec9eeee-00665c9e1a) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx53b2073195c6463db284b-00665c9e1a) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx53b2073195c6463db284b-00665c9e1a) test WARNING: Could not autocreate account '/account' (txn: tx53b2073195c6463db284b-00665c9e1a) test INFO: autocreate account '/account' (txn: tx781fe1a1fa024ecb8139c-00665c9e1a) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: txcf9413d674f04b5aa6565-00665c9e1a) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txcf9413d674f04b5aa6565-00665c9e1a) test ERROR: Account HEAD returning 503 for [503, 503] (txn: txcf9413d674f04b5aa6565-00665c9e1a) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: txcf9413d674f04b5aa6565-00665c9e1a) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: txcf9413d674f04b5aa6565-00665c9e1a) test.unit.proxy.test_server.TestContainerController.test_PUT test.unit.proxy.test_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission ... ok test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (10.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3543410] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3543410] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/2 pid=pid-1] Starting object reconstructor in daemon mode. test INFO: [worker 1/2 pid=pid-1] Starting object reconstruction pass. test INFO: [worker 1/2 pid=pid-1] Object reconstruction complete. (0.10 minutes) test DEBUG: [worker 1/2 pid=pid-1] reconstruction sleeping for 30.0 seconds. test INFO: [worker 2/4 pid=20641] Running object reconstructor in script mode. test DEBUG: [worker 2/4 pid=20641] debug message test INFO: [worker 2/4 pid=20641] info message test WARNING: [worker 2/4 pid=20641] warning message test ERROR: [worker 2/4 pid=20641] error message test INFO: [worker 2/4 pid=20641] Object reconstruction complete (once). (0.00 minutes) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpzjmy13xu/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpzjmy13xu/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpzjmy13xu/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpzjmy13xu/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:33 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0160 "-" 3543410 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:33 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0111 "-" 3543410 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0129 "-" 3543410 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0121 "-" 3543410 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx364c031fa6514d5a985f0-00665c9e2a" "proxy-server 3543410" 0.0024 "-" 3543410 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:37719/sda1/0/a/c" "tx364c031fa6514d5a985f0-00665c9e2a" "container-server 3543410" 0.0021 "-" 3543410 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:57965/sdb1/0/a/c" "tx364c031fa6514d5a985f0-00665c9e2a" "container-server 3543410" 0.0016 "-" 3543410 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx364c031fa6514d5a985f0-00665c9e2a" "proxy-server 3543410" 0.0653 "-" 3543410 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx364c031fa6514d5a985f0-00665c9e2a" "proxy-server 3543410" 0.0276 "-" 3543410 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/34 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx364c031fa6514d5a985f0-00665c9e2a - 0.0835 - - 1717345834.021858931 1717345834.105379581 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "HEAD /sda1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txfd492b4dd78b4f3986446-00665c9e2a" "proxy-server 3543410" 0.0021 "-" 3543410 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:37719/sda1/2/a1/c1" "txfd492b4dd78b4f3986446-00665c9e2a" "container-server 3543410" 0.0026 "-" 3543410 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:57965/sdb1/2/a1/c1" "txfd492b4dd78b4f3986446-00665c9e2a" "container-server 3543410" 0.0022 "-" 3543410 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfd492b4dd78b4f3986446-00665c9e2a" "proxy-server 3543410" 0.0391 "-" 3543410 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfd492b4dd78b4f3986446-00665c9e2a" "proxy-server 3543410" 0.0240 "-" 3543410 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/34 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txfd492b4dd78b4f3986446-00665c9e2a - 0.0532 - - 1717345834.107679129 1717345834.160864353 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx0c6c29f64a134475925e5-00665c9e2a" "proxy-server 3543410" 0.0091 "-" 3543410 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:37719/sda1/0/a/c1" "tx0c6c29f64a134475925e5-00665c9e2a" "container-server 3543410" 0.0118 "-" 3543410 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:57965/sdb1/0/a/c1" "tx0c6c29f64a134475925e5-00665c9e2a" "container-server 3543410" 0.0019 "-" 3543410 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx0c6c29f64a134475925e5-00665c9e2a" "proxy-server 3543410" 0.0297 "-" 3543410 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx0c6c29f64a134475925e5-00665c9e2a" "proxy-server 3543410" 0.0473 "-" 3543410 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/34 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx0c6c29f64a134475925e5-00665c9e2a - 0.0673 - - 1717345834.163138390 1717345834.230435610 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb43e6f53c8d541f5ba7df-00665c9e2a" "proxy-server 3543410" 0.0075 "-" 3543410 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:37719/sda1/2/a/c2" "txb43e6f53c8d541f5ba7df-00665c9e2a" "container-server 3543410" 0.0017 "-" 3543410 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:57965/sdb1/2/a/c2" "txb43e6f53c8d541f5ba7df-00665c9e2a" "container-server 3543410" 0.0098 "-" 3543410 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txb43e6f53c8d541f5ba7df-00665c9e2a" "proxy-server 3543410" 0.0446 "-" 3543410 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:30:34 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txb43e6f53c8d541f5ba7df-00665c9e2a" "proxy-server 3543410" 0.0286 "-" 3543410 2 test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/30/34 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txb43e6f53c8d541f5ba7df-00665c9e2a - 0.0643 - - 1717345834.232792377 1717345834.297087431 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345805.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission ... ok test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: txcf9413d674f04b5aa6565-00665c9e1a) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txcf9413d674f04b5aa6565-00665c9e1a) test WARNING: Could not autocreate account '/account' (txn: txcf9413d674f04b5aa6565-00665c9e1a) test INFO: autocreate account '/account' (txn: txfc9fcb70eb0d4c7a95663-00665c9e1c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txe7dc13dce2194f9da4164-00665c9e1c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txe7dc13dce2194f9da4164-00665c9e1c) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txe7dc13dce2194f9da4164-00665c9e1c) test INFO: autocreate account '/account' (txn: txf120338736564c408e4c2-00665c9e1c) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx6868d38d65624f52a2bb0-00665c9e1c) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx6868d38d65624f52a2bb0-00665c9e1c) test INFO: autocreate account '/account' (txn: tx6868d38d65624f52a2bb0-00665c9e1c) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: txde3aaa52cc104c5fafcd4-00665c9e1c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txde3aaa52cc104c5fafcd4-00665c9e1c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: txf932f2ea946047578fbcb-00665c9e1e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: txbc5451486096437b81c33-00665c9e1e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfd6e61cc21b54f219aa9f-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3a41f457e3444c70b1123-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txabeb32aad03f43b787b75-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf7afa305e9af4712a9654-00665c9e1e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx7f5092e5390a463cbfc5f-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx3785617af52343bb9d830-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txf4cf9d21d42b451896b43-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txf4cf9d21d42b451896b43-00665c9e1e) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txf4cf9d21d42b451896b43-00665c9e1e) test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl ... ok proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server ERROR: ERROR Unhandled exception in request: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/server.py", line 598, in handle_request return handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 375, in wrapped return func(*a, **kw) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 275, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 263, in GETorHEAD resp = self._get_or_head_response(req, node_iter, partition, policy) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2930, in _get_or_head_response app_iter.kickoff(req, resp) File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1184, in kickoff self.stashed_iter = reiterate(self._real_iter(req, resp.headers)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3749, in reiterate chunk = next(iterator) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4452, in _get_next_item return super(StringAlong, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1470, in _iter_one_range for i, next_seg in enumerate(segment_iter): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1549, in _decode_segments_from_fragments segment = self.policy.pyeclib_driver.decode(fragments) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/ec_iface.py", line 300, in decode return self.ec_lib_reference.decode(fragment_payloads, ranges, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/core.py", line 91, in decode return pyeclib_c.decode(self.handle, _fragment_payloads, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pyeclib.ec_iface.ECInvalidFragmentMetadata: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx6a71e41139b6482988e21-00665c9e0b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server ERROR: Object returning 503 for [] (txn: tx24fbc8e43c4042479fd83-00665c9e13) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 ... ok test ERROR: Failed to unlink '/tmp/tmpw1brimdb/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpw1brimdb/containers/part/suffix/hash/container_1717345837.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpw1brimdb/containers/part/suffix/hash/container_1717345837.00000.db': ['/tmp/tmpw1brimdb/containers/part/suffix/hash/container.db', '/tmp/tmpw1brimdb/containers/part/suffix/hash/container_1717345838.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpw1brimdb/containers/part/suffix/hash/container_1717345838.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0085 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0074 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0120 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0099 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0146 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0216 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0125 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0131 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0089 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0116 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3543390 2 test ERROR: ERROR Account update failed with 127.0.0.1:58931/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0120 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0082 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0130 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0066 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0195 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:39 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0106 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0011 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0047 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0052 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0265 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0120 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0092 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0012 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:39 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0051 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0018 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmpo3vnhnk5/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0066 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T163013Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345813.18367', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T163013Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345813.20036', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission ... ok test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx024dd53c753a41338961a-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx024dd53c753a41338961a-00665c9e1e) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx024dd53c753a41338961a-00665c9e1e) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx69cd7c65c53a4c759758f-00665c9e20) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx69cd7c65c53a4c759758f-00665c9e20) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx69cd7c65c53a4c759758f-00665c9e20) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx69cd7c65c53a4c759758f-00665c9e20) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx32bf208125b54a3d83477-00665c9e20) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx32bf208125b54a3d83477-00665c9e20) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx32bf208125b54a3d83477-00665c9e20) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx32bf208125b54a3d83477-00665c9e20) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txf5acce8278e94df2b0001-00665c9e23) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4e5ce1fbe0084843bf238-00665c9e23) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4e5ce1fbe0084843bf238-00665c9e23) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4e5ce1fbe0084843bf238-00665c9e23) test ERROR: Account HEAD returning 503 for [] (txn: tx4e5ce1fbe0084843bf238-00665c9e23) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txd88a03c826bc4f1189081-00665c9e23) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txd88a03c826bc4f1189081-00665c9e23) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txd88a03c826bc4f1189081-00665c9e23) test ERROR: Account HEAD returning 503 for [] (txn: txd88a03c826bc4f1189081-00665c9e23) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txc83d5b7c6ab04412bcee4-00665c9e23) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txe48beea560e74f8ca2542-00665c9e23) test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.test_db.TestDatabaseConnectionError.test_str test.unit.common.test_db.TestDatabaseConnectionError.test_str ... ok test.unit.common.test_db.TestDictFactory.test_normal_case test.unit.common.test_db.TestDictFactory.test_normal_case ... ok test.unit.common.test_utils.TestClosingIterator.test_close test.unit.common.test_utils.TestClosingIterator.test_close ... ok test.unit.common.test_utils.TestClosingIterator.test_close_gen test.unit.common.test_utils.TestClosingIterator.test_close_gen ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_exception test.unit.common.test_utils.TestClosingIterator.test_close_on_exception ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration ... ok test.unit.common.test_utils.TestClosingIterator.test_close_others test.unit.common.test_utils.TestClosingIterator.test_close_others ... ok test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter ... ok test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable ... ok test.unit.common.test_utils.TestClosingIterator.test_nested_iters test.unit.common.test_utils.TestClosingIterator.test_nested_iters ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync ... ok test.unit.common.test_utils.TestWatchdog.test_start_stop test.unit.common.test_utils.TestWatchdog.test_start_stop ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_expire test.unit.common.test_utils.TestWatchdog.test_timeout_expire ... ok test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_info test.unit.container.test_backend.TestCommonContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345813.47524', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345813.49342', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345818.28022', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345818.29901', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5111710f329249b390335-00665c9e14) proxy-server ERROR: Object returning 503 for [] (txn: tx5111710f329249b390335-00665c9e14) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server ERROR: Object returning 503 for [] (txn: tx18e2125724e64ad481aeb-00665c9e19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb705de0231744e8682a12-00665c9e19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from primary (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf64936053415457e944f9-00665c9e19) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: txd636169eae284954a466e-00665c9e19) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body ... ok sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345891.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/a26/1f715828bf49901dc02b4b77f8b43a26/1f715828bf49901dc02b4b77f8b43a26.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345891.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/a26/1f715828bf49901dc02b4b77f8b43a26/1f715828bf49901dc02b4b77f8b43a26.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345900.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/f8c/d7c31c53e6d51c081a27642fd71a5f8c/d7c31c53e6d51c081a27642fd71a5f8c.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345900.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/f8c/d7c31c53e6d51c081a27642fd71a5f8c/d7c31c53e6d51c081a27642fd71a5f8c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345900.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/f8c/d7c31c53e6d51c081a27642fd71a5f8c/d7c31c53e6d51c081a27642fd71a5f8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345909.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/865/74b5c04da8a3bb483b802d336b03e865/74b5c04da8a3bb483b802d336b03e865.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345909.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/865/74b5c04da8a3bb483b802d336b03e865/74b5c04da8a3bb483b802d336b03e865.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345909.00000-2, db: /tmp/tmppxjmt7i8/sda/containers/0/865/74b5c04da8a3bb483b802d336b03e865/74b5c04da8a3bb483b802d336b03e865.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717345782.00000, (0, 0) as of 1717345782.00000, active as of 1717345782.00000> to ShardRange<'a' to 'b' as of 1717345782.00000, (0, 0) as of 1717345782.00000, sharding as of 1717345787.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0_1717345782.00000.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0_1717345782.00000.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0_1717345782.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0_1717345782.00000.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717345782.00000, (0, 0) as of 1717345782.00000, sharding as of 1717345788.00000> to ShardRange<'a' to 'b' as of 1717345782.00000, (0, 0) as of 1717345782.00000, sharded as of 1717345791.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717345782.00000-2, db: /tmp/tmpy0xw7voh/sda/containers/0/7c0/212197483a521b82fd08c28c1a58a7c0/212197483a521b82fd08c28c1a58a7c0_1717345782.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345782.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/0aa/d0dfea9038036e628c801ffaf6a1a0aa/d0dfea9038036e628c801ffaf6a1a0aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345784.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d2a/e6f7095dd2a3d6a35272d4be3ff95d2a/e6f7095dd2a3d6a35272d4be3ff95d2a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345783.00000, (0, 0) as of 1717345783.00000, found as of 1717345783.00000> to ShardRange<'k' to 't' as of 1717345784.00000, (0, 0) as of 1717345784.00000, created as of 1717345784.00000>, path: .shards_a/shard_c_1717345784.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d2a/e6f7095dd2a3d6a35272d4be3ff95d2a/e6f7095dd2a3d6a35272d4be3ff95d2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345786.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d10/539d6f80a2f83d7b43717d99a41ebd10/539d6f80a2f83d7b43717d99a41ebd10.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345785.00000, (0, 0) as of 1717345785.00000, found as of 1717345785.00000> to ShardRange<'k' to 't' as of 1717345786.00000, (0, 0) as of 1717345786.00000, cleaved as of 1717345786.00000>, path: .shards_a/shard_c_1717345786.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d10/539d6f80a2f83d7b43717d99a41ebd10/539d6f80a2f83d7b43717d99a41ebd10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345788.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/89f/7219a9c5e32013a65b352d2e822d089f/7219a9c5e32013a65b352d2e822d089f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345787.00000, (0, 0) as of 1717345787.00000, found as of 1717345787.00000> to ShardRange<'k' to 't' as of 1717345788.00000, (0, 0) as of 1717345788.00000, active as of 1717345788.00000>, path: .shards_a/shard_c_1717345788.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/89f/7219a9c5e32013a65b352d2e822d089f/7219a9c5e32013a65b352d2e822d089f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345790.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e00/f3f3671d13390b035c9a7cd671f72e00/f3f3671d13390b035c9a7cd671f72e00.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345789.00000, (0, 0) as of 1717345789.00000, found as of 1717345789.00000> to ShardRange<'k' to 't' as of 1717345790.00000, (0, 0) as of 1717345790.00000, shrinking as of 1717345790.00000>, path: .shards_a/shard_c_1717345790.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e00/f3f3671d13390b035c9a7cd671f72e00/f3f3671d13390b035c9a7cd671f72e00.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345790.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e00/f3f3671d13390b035c9a7cd671f72e00/f3f3671d13390b035c9a7cd671f72e00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345792.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fb9/a433ed598b3bfaffb60aed8c47decfb9/a433ed598b3bfaffb60aed8c47decfb9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345791.00000, (0, 0) as of 1717345791.00000, found as of 1717345791.00000> to ShardRange<'k' to 't' as of 1717345792.00000, (0, 0) as of 1717345792.00000, sharding as of 1717345792.00000>, path: .shards_a/shard_c_1717345792.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fb9/a433ed598b3bfaffb60aed8c47decfb9/a433ed598b3bfaffb60aed8c47decfb9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345792.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fb9/a433ed598b3bfaffb60aed8c47decfb9/a433ed598b3bfaffb60aed8c47decfb9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345794.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3ea/1240af13ca1b6172fdc53e8f78ef13ea/1240af13ca1b6172fdc53e8f78ef13ea.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345793.00000, (0, 0) as of 1717345793.00000, found as of 1717345793.00000> to ShardRange<'k' to 't' as of 1717345794.00000, (0, 0) as of 1717345794.00000, sharded as of 1717345794.00000>, path: .shards_a/shard_c_1717345794.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3ea/1240af13ca1b6172fdc53e8f78ef13ea/1240af13ca1b6172fdc53e8f78ef13ea.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345794.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3ea/1240af13ca1b6172fdc53e8f78ef13ea/1240af13ca1b6172fdc53e8f78ef13ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345796.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b5a/bba41b20e41eb33a29237935a82bab5a/bba41b20e41eb33a29237935a82bab5a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345795.00000, (0, 0) as of 1717345795.00000, found as of 1717345795.00000> to ShardRange<'k' to 't' as of 1717345796.00000, (0, 0) as of 1717345796.00000, shrunk as of 1717345796.00000>, path: .shards_a/shard_c_1717345796.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b5a/bba41b20e41eb33a29237935a82bab5a/bba41b20e41eb33a29237935a82bab5a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345796.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b5a/bba41b20e41eb33a29237935a82bab5a/bba41b20e41eb33a29237935a82bab5a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345798.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/160/77a0481df86b7c6f1e300723026a8160/77a0481df86b7c6f1e300723026a8160.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345797.00000, (0, 0) as of 1717345797.00000, created as of 1717345797.00000> to ShardRange<'k' to 't' as of 1717345798.00000, (0, 0) as of 1717345798.00000, found as of 1717345798.00000>, path: .shards_a/shard_c_1717345798.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/160/77a0481df86b7c6f1e300723026a8160/77a0481df86b7c6f1e300723026a8160.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345800.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d84/be1cdb9675dd70875806717d9bb19d84/be1cdb9675dd70875806717d9bb19d84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345802.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/350/73a442b253a4e6c9bf8825ee223b3350/73a442b253a4e6c9bf8825ee223b3350.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345801.00000, (0, 0) as of 1717345801.00000, created as of 1717345801.00000> to ShardRange<'k' to 't' as of 1717345802.00000, (0, 0) as of 1717345802.00000, cleaved as of 1717345802.00000>, path: .shards_a/shard_c_1717345802.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/350/73a442b253a4e6c9bf8825ee223b3350/73a442b253a4e6c9bf8825ee223b3350.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345804.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/9bb/22ebd21b208a91d2d6ddfc3a131019bb/22ebd21b208a91d2d6ddfc3a131019bb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345803.00000, (0, 0) as of 1717345803.00000, created as of 1717345803.00000> to ShardRange<'k' to 't' as of 1717345804.00000, (0, 0) as of 1717345804.00000, active as of 1717345804.00000>, path: .shards_a/shard_c_1717345804.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/9bb/22ebd21b208a91d2d6ddfc3a131019bb/22ebd21b208a91d2d6ddfc3a131019bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345806.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f68/7b081b938fb0e48ab58af3817d4b5f68/7b081b938fb0e48ab58af3817d4b5f68.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345805.00000, (0, 0) as of 1717345805.00000, created as of 1717345805.00000> to ShardRange<'k' to 't' as of 1717345806.00000, (0, 0) as of 1717345806.00000, shrinking as of 1717345806.00000>, path: .shards_a/shard_c_1717345806.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f68/7b081b938fb0e48ab58af3817d4b5f68/7b081b938fb0e48ab58af3817d4b5f68.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345806.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f68/7b081b938fb0e48ab58af3817d4b5f68/7b081b938fb0e48ab58af3817d4b5f68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345808.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fa2/92d687791c7317167ea506a2b8c9cfa2/92d687791c7317167ea506a2b8c9cfa2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345807.00000, (0, 0) as of 1717345807.00000, created as of 1717345807.00000> to ShardRange<'k' to 't' as of 1717345808.00000, (0, 0) as of 1717345808.00000, sharding as of 1717345808.00000>, path: .shards_a/shard_c_1717345808.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fa2/92d687791c7317167ea506a2b8c9cfa2/92d687791c7317167ea506a2b8c9cfa2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345808.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fa2/92d687791c7317167ea506a2b8c9cfa2/92d687791c7317167ea506a2b8c9cfa2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345810.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c3f/8b8b384b7c78f8466f434052f84e4c3f/8b8b384b7c78f8466f434052f84e4c3f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345809.00000, (0, 0) as of 1717345809.00000, created as of 1717345809.00000> to ShardRange<'k' to 't' as of 1717345810.00000, (0, 0) as of 1717345810.00000, sharded as of 1717345810.00000>, path: .shards_a/shard_c_1717345810.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c3f/8b8b384b7c78f8466f434052f84e4c3f/8b8b384b7c78f8466f434052f84e4c3f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345810.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c3f/8b8b384b7c78f8466f434052f84e4c3f/8b8b384b7c78f8466f434052f84e4c3f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345812.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/373/f2258d170aaa814b28882229bd47d373/f2258d170aaa814b28882229bd47d373.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345811.00000, (0, 0) as of 1717345811.00000, created as of 1717345811.00000> to ShardRange<'k' to 't' as of 1717345812.00000, (0, 0) as of 1717345812.00000, shrunk as of 1717345812.00000>, path: .shards_a/shard_c_1717345812.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/373/f2258d170aaa814b28882229bd47d373/f2258d170aaa814b28882229bd47d373.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345812.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/373/f2258d170aaa814b28882229bd47d373/f2258d170aaa814b28882229bd47d373.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345814.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b34/0eeff8231f9567f86af467561ad67b34/0eeff8231f9567f86af467561ad67b34.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345813.00000, (0, 0) as of 1717345813.00000, cleaved as of 1717345813.00000> to ShardRange<'k' to 't' as of 1717345814.00000, (0, 0) as of 1717345814.00000, found as of 1717345814.00000>, path: .shards_a/shard_c_1717345814.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b34/0eeff8231f9567f86af467561ad67b34/0eeff8231f9567f86af467561ad67b34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345816.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fc5/9b194352c7b5b75080546bcf2e53afc5/9b194352c7b5b75080546bcf2e53afc5.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345815.00000, (0, 0) as of 1717345815.00000, cleaved as of 1717345815.00000> to ShardRange<'k' to 't' as of 1717345816.00000, (0, 0) as of 1717345816.00000, created as of 1717345816.00000>, path: .shards_a/shard_c_1717345816.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fc5/9b194352c7b5b75080546bcf2e53afc5/9b194352c7b5b75080546bcf2e53afc5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345818.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/0d0/cb1a2e6ba7d08d7b57a4e79ce96ce0d0/cb1a2e6ba7d08d7b57a4e79ce96ce0d0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345820.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ff1/4fa4e71692b2ccc82711077753312ff1/4fa4e71692b2ccc82711077753312ff1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345819.00000, (0, 0) as of 1717345819.00000, cleaved as of 1717345819.00000> to ShardRange<'k' to 't' as of 1717345820.00000, (0, 0) as of 1717345820.00000, active as of 1717345820.00000>, path: .shards_a/shard_c_1717345820.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ff1/4fa4e71692b2ccc82711077753312ff1/4fa4e71692b2ccc82711077753312ff1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345822.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/5fc/e404d5e913b48a7a1895f2fc497f95fc/e404d5e913b48a7a1895f2fc497f95fc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345821.00000, (0, 0) as of 1717345821.00000, cleaved as of 1717345821.00000> to ShardRange<'k' to 't' as of 1717345822.00000, (0, 0) as of 1717345822.00000, shrinking as of 1717345822.00000>, path: .shards_a/shard_c_1717345822.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/5fc/e404d5e913b48a7a1895f2fc497f95fc/e404d5e913b48a7a1895f2fc497f95fc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345822.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/5fc/e404d5e913b48a7a1895f2fc497f95fc/e404d5e913b48a7a1895f2fc497f95fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345824.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/0e9/72aa68b621c74cca5fa0bbfb533420e9/72aa68b621c74cca5fa0bbfb533420e9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345823.00000, (0, 0) as of 1717345823.00000, cleaved as of 1717345823.00000> to ShardRange<'k' to 't' as of 1717345824.00000, (0, 0) as of 1717345824.00000, sharding as of 1717345824.00000>, path: .shards_a/shard_c_1717345824.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/0e9/72aa68b621c74cca5fa0bbfb533420e9/72aa68b621c74cca5fa0bbfb533420e9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345824.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/0e9/72aa68b621c74cca5fa0bbfb533420e9/72aa68b621c74cca5fa0bbfb533420e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345826.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/934/f7e5f2dffbc7c5336d1baf1e4e77c934/f7e5f2dffbc7c5336d1baf1e4e77c934.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345825.00000, (0, 0) as of 1717345825.00000, cleaved as of 1717345825.00000> to ShardRange<'k' to 't' as of 1717345826.00000, (0, 0) as of 1717345826.00000, sharded as of 1717345826.00000>, path: .shards_a/shard_c_1717345826.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/934/f7e5f2dffbc7c5336d1baf1e4e77c934/f7e5f2dffbc7c5336d1baf1e4e77c934.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345826.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/934/f7e5f2dffbc7c5336d1baf1e4e77c934/f7e5f2dffbc7c5336d1baf1e4e77c934.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345828.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f0b/6e3b994656a976b69f762d0949c7bf0b/6e3b994656a976b69f762d0949c7bf0b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345827.00000, (0, 0) as of 1717345827.00000, cleaved as of 1717345827.00000> to ShardRange<'k' to 't' as of 1717345828.00000, (0, 0) as of 1717345828.00000, shrunk as of 1717345828.00000>, path: .shards_a/shard_c_1717345828.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f0b/6e3b994656a976b69f762d0949c7bf0b/6e3b994656a976b69f762d0949c7bf0b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345828.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f0b/6e3b994656a976b69f762d0949c7bf0b/6e3b994656a976b69f762d0949c7bf0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345830.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dbe/cec7bfc38b55dab0877ad75e46048dbe/cec7bfc38b55dab0877ad75e46048dbe.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345829.00000, (0, 0) as of 1717345829.00000, active as of 1717345829.00000> to ShardRange<'k' to 't' as of 1717345830.00000, (0, 0) as of 1717345830.00000, found as of 1717345830.00000>, path: .shards_a/shard_c_1717345830.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dbe/cec7bfc38b55dab0877ad75e46048dbe/cec7bfc38b55dab0877ad75e46048dbe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345832.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/02d/d122358e409306a3fe6f9af25f2db02d/d122358e409306a3fe6f9af25f2db02d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345831.00000, (0, 0) as of 1717345831.00000, active as of 1717345831.00000> to ShardRange<'k' to 't' as of 1717345832.00000, (0, 0) as of 1717345832.00000, created as of 1717345832.00000>, path: .shards_a/shard_c_1717345832.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/02d/d122358e409306a3fe6f9af25f2db02d/d122358e409306a3fe6f9af25f2db02d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345834.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ea8/0ee6894084802bc579d7c7a131790ea8/0ee6894084802bc579d7c7a131790ea8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345833.00000, (0, 0) as of 1717345833.00000, active as of 1717345833.00000> to ShardRange<'k' to 't' as of 1717345834.00000, (0, 0) as of 1717345834.00000, cleaved as of 1717345834.00000>, path: .shards_a/shard_c_1717345834.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ea8/0ee6894084802bc579d7c7a131790ea8/0ee6894084802bc579d7c7a131790ea8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345836.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d3d/488f7293ab8be28777c305e53a20dd3d/488f7293ab8be28777c305e53a20dd3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345838.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/39b/025cf07a64c122606d0740228028939b/025cf07a64c122606d0740228028939b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345837.00000, (0, 0) as of 1717345837.00000, active as of 1717345837.00000> to ShardRange<'k' to 't' as of 1717345838.00000, (0, 0) as of 1717345838.00000, shrinking as of 1717345838.00000>, path: .shards_a/shard_c_1717345838.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/39b/025cf07a64c122606d0740228028939b/025cf07a64c122606d0740228028939b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345838.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/39b/025cf07a64c122606d0740228028939b/025cf07a64c122606d0740228028939b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345840.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f2d/92ca7185aeabc98aff771545bad6df2d/92ca7185aeabc98aff771545bad6df2d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345839.00000, (0, 0) as of 1717345839.00000, active as of 1717345839.00000> to ShardRange<'k' to 't' as of 1717345840.00000, (0, 0) as of 1717345840.00000, sharding as of 1717345840.00000>, path: .shards_a/shard_c_1717345840.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f2d/92ca7185aeabc98aff771545bad6df2d/92ca7185aeabc98aff771545bad6df2d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345840.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f2d/92ca7185aeabc98aff771545bad6df2d/92ca7185aeabc98aff771545bad6df2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345842.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7dd/e706813afa9a939408c5ede9913617dd/e706813afa9a939408c5ede9913617dd.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345841.00000, (0, 0) as of 1717345841.00000, active as of 1717345841.00000> to ShardRange<'k' to 't' as of 1717345842.00000, (0, 0) as of 1717345842.00000, sharded as of 1717345842.00000>, path: .shards_a/shard_c_1717345842.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7dd/e706813afa9a939408c5ede9913617dd/e706813afa9a939408c5ede9913617dd.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345842.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7dd/e706813afa9a939408c5ede9913617dd/e706813afa9a939408c5ede9913617dd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345844.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ad1/e8f378a279e2244036d8490796ce2ad1/e8f378a279e2244036d8490796ce2ad1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345843.00000, (0, 0) as of 1717345843.00000, active as of 1717345843.00000> to ShardRange<'k' to 't' as of 1717345844.00000, (0, 0) as of 1717345844.00000, shrunk as of 1717345844.00000>, path: .shards_a/shard_c_1717345844.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ad1/e8f378a279e2244036d8490796ce2ad1/e8f378a279e2244036d8490796ce2ad1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345844.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ad1/e8f378a279e2244036d8490796ce2ad1/e8f378a279e2244036d8490796ce2ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345846.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/4f9/ca8c0059e8308777396b773435fd44f9/ca8c0059e8308777396b773435fd44f9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345845.00000, (0, 0) as of 1717345845.00000, shrinking as of 1717345845.00000> to ShardRange<'k' to 't' as of 1717345846.00000, (0, 0) as of 1717345846.00000, found as of 1717345846.00000>, path: .shards_a/shard_c_1717345846.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/4f9/ca8c0059e8308777396b773435fd44f9/ca8c0059e8308777396b773435fd44f9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345848.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7ab/0b314fa1b29cdeaaf1d3287838e977ab/0b314fa1b29cdeaaf1d3287838e977ab.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345847.00000, (0, 0) as of 1717345847.00000, shrinking as of 1717345847.00000> to ShardRange<'k' to 't' as of 1717345848.00000, (0, 0) as of 1717345848.00000, created as of 1717345848.00000>, path: .shards_a/shard_c_1717345848.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7ab/0b314fa1b29cdeaaf1d3287838e977ab/0b314fa1b29cdeaaf1d3287838e977ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345850.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a62/adb54d02852b5ce25cccd5be334cda62/adb54d02852b5ce25cccd5be334cda62.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345849.00000, (0, 0) as of 1717345849.00000, shrinking as of 1717345849.00000> to ShardRange<'k' to 't' as of 1717345850.00000, (0, 0) as of 1717345850.00000, cleaved as of 1717345850.00000>, path: .shards_a/shard_c_1717345850.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a62/adb54d02852b5ce25cccd5be334cda62/adb54d02852b5ce25cccd5be334cda62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345852.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e84/90fd57c60d60a860dd1afa48f6204e84/90fd57c60d60a860dd1afa48f6204e84.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345851.00000, (0, 0) as of 1717345851.00000, shrinking as of 1717345851.00000> to ShardRange<'k' to 't' as of 1717345852.00000, (0, 0) as of 1717345852.00000, active as of 1717345852.00000>, path: .shards_a/shard_c_1717345852.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e84/90fd57c60d60a860dd1afa48f6204e84/90fd57c60d60a860dd1afa48f6204e84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345854.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/682/47be01186cfc8b8458b25830c5913682/47be01186cfc8b8458b25830c5913682.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345854.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/682/47be01186cfc8b8458b25830c5913682/47be01186cfc8b8458b25830c5913682.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345856.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a3d/bf524be0f42fbe7e4178d24d4415da3d/bf524be0f42fbe7e4178d24d4415da3d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345855.00000, (0, 0) as of 1717345855.00000, shrinking as of 1717345855.00000> to ShardRange<'k' to 't' as of 1717345856.00000, (0, 0) as of 1717345856.00000, sharding as of 1717345856.00000>, path: .shards_a/shard_c_1717345856.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a3d/bf524be0f42fbe7e4178d24d4415da3d/bf524be0f42fbe7e4178d24d4415da3d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345856.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a3d/bf524be0f42fbe7e4178d24d4415da3d/bf524be0f42fbe7e4178d24d4415da3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345858.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c19/c622b0eda5ac447c224fcbb18f23cc19/c622b0eda5ac447c224fcbb18f23cc19.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345857.00000, (0, 0) as of 1717345857.00000, shrinking as of 1717345857.00000> to ShardRange<'k' to 't' as of 1717345858.00000, (0, 0) as of 1717345858.00000, sharded as of 1717345858.00000>, path: .shards_a/shard_c_1717345858.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c19/c622b0eda5ac447c224fcbb18f23cc19/c622b0eda5ac447c224fcbb18f23cc19.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345858.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c19/c622b0eda5ac447c224fcbb18f23cc19/c622b0eda5ac447c224fcbb18f23cc19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345860.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/2c9/94f225e50c3d3bc013a762fe2e1022c9/94f225e50c3d3bc013a762fe2e1022c9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345859.00000, (0, 0) as of 1717345859.00000, shrinking as of 1717345859.00000> to ShardRange<'k' to 't' as of 1717345860.00000, (0, 0) as of 1717345860.00000, shrunk as of 1717345860.00000>, path: .shards_a/shard_c_1717345860.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/2c9/94f225e50c3d3bc013a762fe2e1022c9/94f225e50c3d3bc013a762fe2e1022c9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345860.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/2c9/94f225e50c3d3bc013a762fe2e1022c9/94f225e50c3d3bc013a762fe2e1022c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345862.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3aa/83ba8c04ba9e4613bed069bc99cec3aa/83ba8c04ba9e4613bed069bc99cec3aa.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345861.00000, (0, 0) as of 1717345861.00000, sharding as of 1717345861.00000> to ShardRange<'k' to 't' as of 1717345862.00000, (0, 0) as of 1717345862.00000, found as of 1717345862.00000>, path: .shards_a/shard_c_1717345862.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3aa/83ba8c04ba9e4613bed069bc99cec3aa/83ba8c04ba9e4613bed069bc99cec3aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345864.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/df1/abb8bbd452b4808949238fd5604bbdf1/abb8bbd452b4808949238fd5604bbdf1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345863.00000, (0, 0) as of 1717345863.00000, sharding as of 1717345863.00000> to ShardRange<'k' to 't' as of 1717345864.00000, (0, 0) as of 1717345864.00000, created as of 1717345864.00000>, path: .shards_a/shard_c_1717345864.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/df1/abb8bbd452b4808949238fd5604bbdf1/abb8bbd452b4808949238fd5604bbdf1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345866.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/1e9/1be9734108b4af2eedab6bb1d0d2d1e9/1be9734108b4af2eedab6bb1d0d2d1e9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345865.00000, (0, 0) as of 1717345865.00000, sharding as of 1717345865.00000> to ShardRange<'k' to 't' as of 1717345866.00000, (0, 0) as of 1717345866.00000, cleaved as of 1717345866.00000>, path: .shards_a/shard_c_1717345866.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/1e9/1be9734108b4af2eedab6bb1d0d2d1e9/1be9734108b4af2eedab6bb1d0d2d1e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345868.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c09/1d2c7000cd326eb3132a57c063003c09/1d2c7000cd326eb3132a57c063003c09.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345867.00000, (0, 0) as of 1717345867.00000, sharding as of 1717345867.00000> to ShardRange<'k' to 't' as of 1717345868.00000, (0, 0) as of 1717345868.00000, active as of 1717345868.00000>, path: .shards_a/shard_c_1717345868.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c09/1d2c7000cd326eb3132a57c063003c09/1d2c7000cd326eb3132a57c063003c09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345870.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/126/fc693c4d591f24e9cccd8bb2f788a126/fc693c4d591f24e9cccd8bb2f788a126.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345869.00000, (0, 0) as of 1717345869.00000, sharding as of 1717345869.00000> to ShardRange<'k' to 't' as of 1717345870.00000, (0, 0) as of 1717345870.00000, shrinking as of 1717345870.00000>, path: .shards_a/shard_c_1717345870.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/126/fc693c4d591f24e9cccd8bb2f788a126/fc693c4d591f24e9cccd8bb2f788a126.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345870.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/126/fc693c4d591f24e9cccd8bb2f788a126/fc693c4d591f24e9cccd8bb2f788a126.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345872.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/89b/4f1b4a6eb2c50ad6d3f1d02ac744589b/4f1b4a6eb2c50ad6d3f1d02ac744589b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345872.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/89b/4f1b4a6eb2c50ad6d3f1d02ac744589b/4f1b4a6eb2c50ad6d3f1d02ac744589b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345874.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/4ac/8a1d5e4c6eaa51a9619f2502526604ac/8a1d5e4c6eaa51a9619f2502526604ac.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345873.00000, (0, 0) as of 1717345873.00000, sharding as of 1717345873.00000> to ShardRange<'k' to 't' as of 1717345874.00000, (0, 0) as of 1717345874.00000, sharded as of 1717345874.00000>, path: .shards_a/shard_c_1717345874.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/4ac/8a1d5e4c6eaa51a9619f2502526604ac/8a1d5e4c6eaa51a9619f2502526604ac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345874.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/4ac/8a1d5e4c6eaa51a9619f2502526604ac/8a1d5e4c6eaa51a9619f2502526604ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345876.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a25/2a521685dab2915d8f94d1957aa16a25/2a521685dab2915d8f94d1957aa16a25.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345875.00000, (0, 0) as of 1717345875.00000, sharding as of 1717345875.00000> to ShardRange<'k' to 't' as of 1717345876.00000, (0, 0) as of 1717345876.00000, shrunk as of 1717345876.00000>, path: .shards_a/shard_c_1717345876.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a25/2a521685dab2915d8f94d1957aa16a25/2a521685dab2915d8f94d1957aa16a25.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345876.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a25/2a521685dab2915d8f94d1957aa16a25/2a521685dab2915d8f94d1957aa16a25.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345878.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/406/81d33b357b344e611370a652952a3406/81d33b357b344e611370a652952a3406.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345877.00000, (0, 0) as of 1717345877.00000, sharded as of 1717345877.00000> to ShardRange<'k' to 't' as of 1717345878.00000, (0, 0) as of 1717345878.00000, found as of 1717345878.00000>, path: .shards_a/shard_c_1717345878.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/406/81d33b357b344e611370a652952a3406/81d33b357b344e611370a652952a3406.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345880.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/2af/3a80c3e331948006f086b14984fe32af/3a80c3e331948006f086b14984fe32af.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345879.00000, (0, 0) as of 1717345879.00000, sharded as of 1717345879.00000> to ShardRange<'k' to 't' as of 1717345880.00000, (0, 0) as of 1717345880.00000, created as of 1717345880.00000>, path: .shards_a/shard_c_1717345880.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/2af/3a80c3e331948006f086b14984fe32af/3a80c3e331948006f086b14984fe32af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345882.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/680/df66e138663dd1bbdb7658e62dc33680/df66e138663dd1bbdb7658e62dc33680.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345881.00000, (0, 0) as of 1717345881.00000, sharded as of 1717345881.00000> to ShardRange<'k' to 't' as of 1717345882.00000, (0, 0) as of 1717345882.00000, cleaved as of 1717345882.00000>, path: .shards_a/shard_c_1717345882.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/680/df66e138663dd1bbdb7658e62dc33680/df66e138663dd1bbdb7658e62dc33680.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345884.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/684/756e24fa87c19fdf1d00b679f03bc684/756e24fa87c19fdf1d00b679f03bc684.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345883.00000, (0, 0) as of 1717345883.00000, sharded as of 1717345883.00000> to ShardRange<'k' to 't' as of 1717345884.00000, (0, 0) as of 1717345884.00000, active as of 1717345884.00000>, path: .shards_a/shard_c_1717345884.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/684/756e24fa87c19fdf1d00b679f03bc684/756e24fa87c19fdf1d00b679f03bc684.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345886.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/acb/03ada642bfb4f80457e93a9dcac95acb/03ada642bfb4f80457e93a9dcac95acb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345885.00000, (0, 0) as of 1717345885.00000, sharded as of 1717345885.00000> to ShardRange<'k' to 't' as of 1717345886.00000, (0, 0) as of 1717345886.00000, shrinking as of 1717345886.00000>, path: .shards_a/shard_c_1717345886.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/acb/03ada642bfb4f80457e93a9dcac95acb/03ada642bfb4f80457e93a9dcac95acb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345886.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/acb/03ada642bfb4f80457e93a9dcac95acb/03ada642bfb4f80457e93a9dcac95acb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345888.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dd3/cd6cb810de80d32dba8e2e11916e3dd3/cd6cb810de80d32dba8e2e11916e3dd3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345887.00000, (0, 0) as of 1717345887.00000, sharded as of 1717345887.00000> to ShardRange<'k' to 't' as of 1717345888.00000, (0, 0) as of 1717345888.00000, sharding as of 1717345888.00000>, path: .shards_a/shard_c_1717345888.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dd3/cd6cb810de80d32dba8e2e11916e3dd3/cd6cb810de80d32dba8e2e11916e3dd3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345888.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dd3/cd6cb810de80d32dba8e2e11916e3dd3/cd6cb810de80d32dba8e2e11916e3dd3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345890.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dcf/4d394e0464b28793c92b4319561efdcf/4d394e0464b28793c92b4319561efdcf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345890.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/dcf/4d394e0464b28793c92b4319561efdcf/4d394e0464b28793c92b4319561efdcf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345892.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d5c/236937d7a078dfc6eba9335a7678dd5c/236937d7a078dfc6eba9335a7678dd5c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345891.00000, (0, 0) as of 1717345891.00000, sharded as of 1717345891.00000> to ShardRange<'k' to 't' as of 1717345892.00000, (0, 0) as of 1717345892.00000, shrunk as of 1717345892.00000>, path: .shards_a/shard_c_1717345892.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d5c/236937d7a078dfc6eba9335a7678dd5c/236937d7a078dfc6eba9335a7678dd5c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345892.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d5c/236937d7a078dfc6eba9335a7678dd5c/236937d7a078dfc6eba9335a7678dd5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345894.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/89e/8ea2815e930818fd0abd1b759ba5689e/8ea2815e930818fd0abd1b759ba5689e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345893.00000, (0, 0) as of 1717345893.00000, shrunk as of 1717345893.00000> to ShardRange<'k' to 't' as of 1717345894.00000, (0, 0) as of 1717345894.00000, found as of 1717345894.00000>, path: .shards_a/shard_c_1717345894.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/89e/8ea2815e930818fd0abd1b759ba5689e/8ea2815e930818fd0abd1b759ba5689e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345896.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b2e/fb3aacab15f1d8d6630a2add16efbb2e/fb3aacab15f1d8d6630a2add16efbb2e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345895.00000, (0, 0) as of 1717345895.00000, shrunk as of 1717345895.00000> to ShardRange<'k' to 't' as of 1717345896.00000, (0, 0) as of 1717345896.00000, created as of 1717345896.00000>, path: .shards_a/shard_c_1717345896.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b2e/fb3aacab15f1d8d6630a2add16efbb2e/fb3aacab15f1d8d6630a2add16efbb2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345898.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3e7/36526bdf73e4c16a307256de414c43e7/36526bdf73e4c16a307256de414c43e7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345897.00000, (0, 0) as of 1717345897.00000, shrunk as of 1717345897.00000> to ShardRange<'k' to 't' as of 1717345898.00000, (0, 0) as of 1717345898.00000, cleaved as of 1717345898.00000>, path: .shards_a/shard_c_1717345898.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/3e7/36526bdf73e4c16a307256de414c43e7/36526bdf73e4c16a307256de414c43e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345900.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7ae/ba9bed263516a49ff367b989197117ae/ba9bed263516a49ff367b989197117ae.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345899.00000, (0, 0) as of 1717345899.00000, shrunk as of 1717345899.00000> to ShardRange<'k' to 't' as of 1717345900.00000, (0, 0) as of 1717345900.00000, active as of 1717345900.00000>, path: .shards_a/shard_c_1717345900.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7ae/ba9bed263516a49ff367b989197117ae/ba9bed263516a49ff367b989197117ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345902.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ae3/32cbe17e152b8469da67255fe65ddae3/32cbe17e152b8469da67255fe65ddae3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345901.00000, (0, 0) as of 1717345901.00000, shrunk as of 1717345901.00000> to ShardRange<'k' to 't' as of 1717345902.00000, (0, 0) as of 1717345902.00000, shrinking as of 1717345902.00000>, path: .shards_a/shard_c_1717345902.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ae3/32cbe17e152b8469da67255fe65ddae3/32cbe17e152b8469da67255fe65ddae3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345902.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/ae3/32cbe17e152b8469da67255fe65ddae3/32cbe17e152b8469da67255fe65ddae3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345904.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/770/dc730df607feec4ef35121e646165770/dc730df607feec4ef35121e646165770.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345903.00000, (0, 0) as of 1717345903.00000, shrunk as of 1717345903.00000> to ShardRange<'k' to 't' as of 1717345904.00000, (0, 0) as of 1717345904.00000, sharding as of 1717345904.00000>, path: .shards_a/shard_c_1717345904.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/770/dc730df607feec4ef35121e646165770/dc730df607feec4ef35121e646165770.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345904.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/770/dc730df607feec4ef35121e646165770/dc730df607feec4ef35121e646165770.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345906.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b53/ae4e2beee7b9130beab5cea7d121bb53/ae4e2beee7b9130beab5cea7d121bb53.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345905.00000, (0, 0) as of 1717345905.00000, shrunk as of 1717345905.00000> to ShardRange<'k' to 't' as of 1717345906.00000, (0, 0) as of 1717345906.00000, sharded as of 1717345906.00000>, path: .shards_a/shard_c_1717345906.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b53/ae4e2beee7b9130beab5cea7d121bb53/ae4e2beee7b9130beab5cea7d121bb53.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345906.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b53/ae4e2beee7b9130beab5cea7d121bb53/ae4e2beee7b9130beab5cea7d121bb53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345908.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/57d/eb5ba4c07c6e20f9b7676e1113b8257d/eb5ba4c07c6e20f9b7676e1113b8257d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345908.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/57d/eb5ba4c07c6e20f9b7676e1113b8257d/eb5ba4c07c6e20f9b7676e1113b8257d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345909.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/608/89ebb577b05ca43cbda6b70c58bc5608/89ebb577b05ca43cbda6b70c58bc5608.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345911.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/20a/945a7307e5e3b3ce6045c6a5020f320a/945a7307e5e3b3ce6045c6a5020f320a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345913.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/285/c88a291ce3ddbcea672f6625a16ad285/c88a291ce3ddbcea672f6625a16ad285.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345915.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/9f4/4bfdbe8acb6161c35686833cabf3f9f4/4bfdbe8acb6161c35686833cabf3f9f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345917.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/9c9/9654fa1fa4d1cd54d6df78787b4629c9/9654fa1fa4d1cd54d6df78787b4629c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345919.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/37c/a9c2bdb64efe7b0672bae134b667637c/a9c2bdb64efe7b0672bae134b667637c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345921.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/5f7/e8c19816b6c6f8a1f3c6f625ba85b5f7/e8c19816b6c6f8a1f3c6f625ba85b5f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345923.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/1a8/e695be8117e042fa69d573c5afd751a8/e695be8117e042fa69d573c5afd751a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345925.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/495/12c8d78ae9f69d3c0ecbebd249ac5495/12c8d78ae9f69d3c0ecbebd249ac5495.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345927.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f39/0ba6a695bc4c3b56b7bba6280ae49f39/0ba6a695bc4c3b56b7bba6280ae49f39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345929.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/6a7/fa246a26d216dcaae96c2c4cc0eac6a7/fa246a26d216dcaae96c2c4cc0eac6a7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345931.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/6e4/7d20a300ab219be613617a21f899a6e4/7d20a300ab219be613617a21f899a6e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345933.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/285/597d8b67df97b05cafa30f35b9c2e285/597d8b67df97b05cafa30f35b9c2e285.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345935.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7b2/f7e0d73f291883b8ae1fb291a19ba7b2/f7e0d73f291883b8ae1fb291a19ba7b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345937.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/609/52f9b89c9909adf17028890f383df609/52f9b89c9909adf17028890f383df609.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345939.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c9d/1f073d50aff4d05182cd23a38d020c9d/1f073d50aff4d05182cd23a38d020c9d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345941.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/204/324c567b94b19753771a541991de2204/324c567b94b19753771a541991de2204.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345943.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/81a/9cfc91887749fa2c4fcde5317a70481a/9cfc91887749fa2c4fcde5317a70481a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345945.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/1d4/1e9217d14a87cfbaedb3b572f7ffe1d4/1e9217d14a87cfbaedb3b572f7ffe1d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345947.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/a3e/e3193e22127da6e35b199d5b3c636a3e/e3193e22127da6e35b199d5b3c636a3e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345949.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c22/d86e385b4aaaa921ce00efbea1172c22/d86e385b4aaaa921ce00efbea1172c22.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345951.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/615/02d2f6635235f4716f950356ab5fc615/02d2f6635235f4716f950356ab5fc615.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345953.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/cb9/15c17a64cf116e8d10bbcd6e36219cb9/15c17a64cf116e8d10bbcd6e36219cb9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345955.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/732/c33c95a9e114a9a71275bc11f7e78732/c33c95a9e114a9a71275bc11f7e78732.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345957.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/267/f221ea6ae5500182680aab99312aa267/f221ea6ae5500182680aab99312aa267.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345959.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f82/02bc54f362c969be1720037ed8f3bf82/02bc54f362c969be1720037ed8f3bf82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345961.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/49d/f626e041c454a4356faa513656c5549d/f626e041c454a4356faa513656c5549d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345963.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d73/c899320f90c6b11598b587ab7230bd73/c899320f90c6b11598b587ab7230bd73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345965.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/fba/6ebf20a37c134dd109927b9f63d0efba/6ebf20a37c134dd109927b9f63d0efba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345967.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/2b6/74e919399c702cde9b5881236825d2b6/74e919399c702cde9b5881236825d2b6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345969.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/284/0004b1988b9692935d48512842833284/0004b1988b9692935d48512842833284.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345971.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/784/2b809766900c4ded14351a5ffc066784/2b809766900c4ded14351a5ffc066784.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345973.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/556/e18fde4f33e29c4599ab83e43ec76556/e18fde4f33e29c4599ab83e43ec76556.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345973.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/556/e18fde4f33e29c4599ab83e43ec76556/e18fde4f33e29c4599ab83e43ec76556.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345975.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c54/8e86ca6d8292f9e9003c15035ac7cc54/8e86ca6d8292f9e9003c15035ac7cc54.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345975.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/c54/8e86ca6d8292f9e9003c15035ac7cc54/8e86ca6d8292f9e9003c15035ac7cc54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345977.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d83/bdd27a49f34acdfc8fa9eb8a93fd8d83/bdd27a49f34acdfc8fa9eb8a93fd8d83.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345977.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d83/bdd27a49f34acdfc8fa9eb8a93fd8d83/bdd27a49f34acdfc8fa9eb8a93fd8d83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345979.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b4c/e28b1f4852c4fa8e0aace9f0566c2b4c/e28b1f4852c4fa8e0aace9f0566c2b4c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345979.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/b4c/e28b1f4852c4fa8e0aace9f0566c2b4c/e28b1f4852c4fa8e0aace9f0566c2b4c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345981.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e53/955332a31bd75b46593ac2544d648e53/955332a31bd75b46593ac2544d648e53.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345981.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e53/955332a31bd75b46593ac2544d648e53/955332a31bd75b46593ac2544d648e53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345983.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e4c/08cfff59cd22fe2b47c1f073bcb8ee4c/08cfff59cd22fe2b47c1f073bcb8ee4c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345983.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e4c/08cfff59cd22fe2b47c1f073bcb8ee4c/08cfff59cd22fe2b47c1f073bcb8ee4c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345985.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/8ff/c0fd989aaab4dc3944151ad77e1e88ff/c0fd989aaab4dc3944151ad77e1e88ff.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345985.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/8ff/c0fd989aaab4dc3944151ad77e1e88ff/c0fd989aaab4dc3944151ad77e1e88ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345987.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/53b/366a3d693250c6c830b5fc65eb8a953b/366a3d693250c6c830b5fc65eb8a953b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345987.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/53b/366a3d693250c6c830b5fc65eb8a953b/366a3d693250c6c830b5fc65eb8a953b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345989.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e6a/ff4c30c14c63d3c4a4352e6f167bee6a/ff4c30c14c63d3c4a4352e6f167bee6a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345989.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e6a/ff4c30c14c63d3c4a4352e6f167bee6a/ff4c30c14c63d3c4a4352e6f167bee6a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345991.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/498/00ebe15bb52b7a87b7a79d1ac4689498/00ebe15bb52b7a87b7a79d1ac4689498.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345991.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/498/00ebe15bb52b7a87b7a79d1ac4689498/00ebe15bb52b7a87b7a79d1ac4689498.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345993.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e20/41af3a648e0f673e5330aa4f3bd1be20/41af3a648e0f673e5330aa4f3bd1be20.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345993.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e20/41af3a648e0f673e5330aa4f3bd1be20/41af3a648e0f673e5330aa4f3bd1be20.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345995.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/09f/b305934c24fb759fe5582df563ab909f/b305934c24fb759fe5582df563ab909f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345995.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/09f/b305934c24fb759fe5582df563ab909f/b305934c24fb759fe5582df563ab909f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345997.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e5f/f7530b6b578bf87bf7735f205797fe5f/f7530b6b578bf87bf7735f205797fe5f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345997.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e5f/f7530b6b578bf87bf7735f205797fe5f/f7530b6b578bf87bf7735f205797fe5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717345999.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/6d1/6137ca5db90dbe58f38cce655f0226d1/6137ca5db90dbe58f38cce655f0226d1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717345999.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/6d1/6137ca5db90dbe58f38cce655f0226d1/6137ca5db90dbe58f38cce655f0226d1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346001.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e72/49d6ee6ba0adb9fbd0cabee015684e72/49d6ee6ba0adb9fbd0cabee015684e72.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346001.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e72/49d6ee6ba0adb9fbd0cabee015684e72/49d6ee6ba0adb9fbd0cabee015684e72.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346003.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/5cc/0e7c112454eb14ef5c25f2989ae6a5cc/0e7c112454eb14ef5c25f2989ae6a5cc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346003.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/5cc/0e7c112454eb14ef5c25f2989ae6a5cc/0e7c112454eb14ef5c25f2989ae6a5cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346005.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/db3/bf6efb479c0c42ffd9e0d25640d44db3/bf6efb479c0c42ffd9e0d25640d44db3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346005.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/db3/bf6efb479c0c42ffd9e0d25640d44db3/bf6efb479c0c42ffd9e0d25640d44db3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346007.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/050/42a1427b4aba882f289e10b9ffbd5050/42a1427b4aba882f289e10b9ffbd5050.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346007.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/050/42a1427b4aba882f289e10b9ffbd5050/42a1427b4aba882f289e10b9ffbd5050.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346009.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7ec/908c4ecb38fea553b0baa7b0dafad7ec/908c4ecb38fea553b0baa7b0dafad7ec.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346009.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/7ec/908c4ecb38fea553b0baa7b0dafad7ec/908c4ecb38fea553b0baa7b0dafad7ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346011.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d73/8ba5b319539e24d2f4741f00846f6d73/8ba5b319539e24d2f4741f00846f6d73.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346011.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/d73/8ba5b319539e24d2f4741f00846f6d73/8ba5b319539e24d2f4741f00846f6d73.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container ... ok test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0069 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0045 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0010 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0197 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0120 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0116 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0004 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0161 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0060 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0028 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0042 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0090 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0050 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0041 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0048 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0048 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmpc8seve_m/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0035 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0084 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0046 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0180 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0132 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0091 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0038 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable ... ok test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9d14c186-b910-45b8-82d0-7f8a1c0dd31f-sda', '1717345806.97489', '1717345809', '1717345808', '', '1717345809', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('9d14c186-b910-45b8-82d0-7f8a1c0dd31f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.15344 seconds (0.46437/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp73ug1oaj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6c7158f2a70c5b9d140e0979427e5ac0', '23d8c6c7-72c7-4135-913e-fa241e7b0821-sda', '1717345809.20835', '1717345812', '1717345811', '', '1717345812', 1, 0) test DEBUG: /tmp/tmp73ug1oaj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717345815.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '23d8c6c7-72c7-4135-913e-fa241e7b0821-sda') test DEBUG: /tmp/tmp73ug1oaj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0099771s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6c7158f2a70c5b9d140e0979427e5ac0', '23d8c6c7-72c7-4135-913e-fa241e7b0821-sda', '1717345809.20835', '1717345812', '1717345811', '', '1717345812', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('23d8c6c7-72c7-4135-913e-fa241e7b0821-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp73ug1oaj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10366 seconds (9.64728/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp73ug1oaj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '404d933fe2877884781a12ce6704857d', '3cd1d334-bab4-4b73-a0b7-d263bb41a4a2-sda', '1717345809.35002', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('3cd1d334-bab4-4b73-a0b7-d263bb41a4a2-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '404d933fe2877884781a12ce6704857d', '3cd1d334-bab4-4b73-a0b7-d263bb41a4a2-sda', '1717345809.35002', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('3cd1d334-bab4-4b73-a0b7-d263bb41a4a2-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpyocan84a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '42d560e3-f02d-42aa-abce-54a462037d57-sda', '1717345809.42910', '1717345812', '1717345811', '', '1717345812', 0, 0) test DEBUG: /tmp/tmpyocan84a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '42d560e3-f02d-42aa-abce-54a462037d57-sda', '1717345809.42910', '1717345812', '1717345811', '', '1717345812', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('42d560e3-f02d-42aa-abce-54a462037d57-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06090 seconds (16.42124/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpu_typvyy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6c7158f2a70c5b9d140e0979427e5ac0', 'c29c5be4-bb8f-4701-921f-0ecc7cbaacdc-sda', '1717345809.56384', '1717345812', '1717345811', '', '1717345812', 1, 0) test DEBUG: /tmp/tmpu_typvyy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6c7158f2a70c5b9d140e0979427e5ac0', 'c29c5be4-bb8f-4701-921f-0ecc7cbaacdc-sda', '1717345809.56384', '1717345812', '1717345811', '', '1717345812', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c29c5be4-bb8f-4701-921f-0ecc7cbaacdc-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpu_typvyy/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.11084 seconds (9.02208/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpu_typvyy/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '404d933fe2877884781a12ce6704857d', '0ebc97d1-b7ea-47ae-8c5e-1407a082b88d-sda', '1717345809.71635', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0ebc97d1-b7ea-47ae-8c5e-1407a082b88d-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '404d933fe2877884781a12ce6704857d', '0ebc97d1-b7ea-47ae-8c5e-1407a082b88d-sda', '1717345809.71635', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0ebc97d1-b7ea-47ae-8c5e-1407a082b88d-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a5b91babcf97e1e71bb1658b0890e272', '5157e6f4-6e28-43c4-a447-dc1be48f2f8b-sda', '1717345812.17934', '1717345812.00000', '0', '{"x-container-meta-test": ["foo", "1717345812.00000"]}', '1717345812.00000', 1, 0) test DEBUG: replicator-rpc-sync time for info: 0.01s test DEBUG: replicator-rpc-sync time for update_metadata: 0.00s test DEBUG: replicator-rpc-sync time for merge_timestamps: 0.01s test DEBUG: replicator-rpc-sync time for get_sync: 0.00s test DEBUG: replicator-rpc-sync time for merge_syncs: 0.01s REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345812.30581', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345812.30581', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30581', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345812.30587', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345812.30590', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345812.30590', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30590', 'epoch': None, 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345812.33997', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.00893569s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345812.30581', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345812.30581', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30581', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345812.30587', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345812.30590', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345812.30590', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30590', 'epoch': None, 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345812.30581', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345812.30581', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30581', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 13, 'bytes_used': 123, 'meta_timestamp': '1717345816.64477', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345812.30590', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345812.30590', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30590', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345812.30581', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345812.30581', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30581', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345816.68977', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345812.30590', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345812.30590', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30590', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345816.72140', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345816.72140', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345816.72140', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345816.68977', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345812.30590', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345812.30590', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30590', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345816.72140', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345816.72140', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345816.72140', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345816.68977', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345818.87388', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 30, 'meta_timestamp': '1717345818.87388', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345818.87388', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345816.72140', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345816.72140', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345816.72140', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345816.68977', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345818.87388', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717345818.91964', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345818.87388', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345818.95380', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345816.72140', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345816.72140', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345816.68977', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345818.87388', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717345818.91964', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345818.87388', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpopo9pg9l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ba9f95ad7e0517e7bc47f46444258f16', '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda', '1717345812.27795', '1717345812.27273', '0', '', '1717345812.27273', 1, 0) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345818.36904', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345818.38568', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.20175', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.21679', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record ... ok test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txe48beea560e74f8ca2542-00665c9e23) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe48beea560e74f8ca2542-00665c9e23) test ERROR: Account HEAD returning 503 for [] (txn: txe48beea560e74f8ca2542-00665c9e23) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txe3123665eac84c87ae98a-00665c9e24) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txe3123665eac84c87ae98a-00665c9e24) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txe3123665eac84c87ae98a-00665c9e24) test ERROR: Account HEAD returning 503 for [] (txn: txe3123665eac84c87ae98a-00665c9e24) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txbf98f396669e46db9ccde-00665c9e27) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: txab72a04a17fa45a7a8322-00665c9e27) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: txab72a04a17fa45a7a8322-00665c9e27) test INFO: Client disconnected on read of '/a/c' (txn: txab72a04a17fa45a7a8322-00665c9e27) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txfd9c990d049f4423a6606-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx679fc3dceee247b7b60d8-00665c9e27) test INFO: Client disconnected on read of '/a/c' (txn: tx679fc3dceee247b7b60d8-00665c9e27) test INFO: Client disconnected on read of '/a/c/' (txn: tx679fc3dceee247b7b60d8-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx5c6c493e2ce24b8da122c-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx5c6c493e2ce24b8da122c-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx5c6c493e2ce24b8da122c-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx5c6c493e2ce24b8da122c-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx65704bddab4841c19ff2a-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx65704bddab4841c19ff2a-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx65704bddab4841c19ff2a-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx65704bddab4841c19ff2a-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txb44b7a112f6c4ac5948b6-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txb44b7a112f6c4ac5948b6-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txb44b7a112f6c4ac5948b6-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txb44b7a112f6c4ac5948b6-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx9208653a9cc04ae78cfa3-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx9208653a9cc04ae78cfa3-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx9208653a9cc04ae78cfa3-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx9208653a9cc04ae78cfa3-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txcf0131a768d9469dab552-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txcf0131a768d9469dab552-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txcf0131a768d9469dab552-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txcf0131a768d9469dab552-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx5ffd4adbc8274e4ea0f4a-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx5ffd4adbc8274e4ea0f4a-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx5ffd4adbc8274e4ea0f4a-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx5ffd4adbc8274e4ea0f4a-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx9c3510f2e702460098275-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx9c3510f2e702460098275-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx9c3510f2e702460098275-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx9c3510f2e702460098275-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx8db0de0c327b42a8a8e7d-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx8db0de0c327b42a8a8e7d-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx8db0de0c327b42a8a8e7d-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx8db0de0c327b42a8a8e7d-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txcc1f7073ca8a493584582-00665c9e27) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: txcc1f7073ca8a493584582-00665c9e27) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txcc1f7073ca8a493584582-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txcc1f7073ca8a493584582-00665c9e27) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txcc1f7073ca8a493584582-00665c9e27) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txcc1f7073ca8a493584582-00665c9e27) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txadcab4eceaa34988a7602-00665c9e2b) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txadcab4eceaa34988a7602-00665c9e2b) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txadcab4eceaa34988a7602-00665c9e2b) test ERROR: Container GET returning 503 for [503, 503] (txn: txadcab4eceaa34988a7602-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx47487a436dc0469d9545d-00665c9e2b) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx47487a436dc0469d9545d-00665c9e2b) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx47487a436dc0469d9545d-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx47487a436dc0469d9545d-00665c9e2b) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx47487a436dc0469d9545d-00665c9e2b) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx47487a436dc0469d9545d-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx47487a436dc0469d9545d-00665c9e2b) test ERROR: Container GET returning 503 for [503, 503] (txn: tx47487a436dc0469d9545d-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx15e7fba582a245d2a6a9d-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx15e7fba582a245d2a6a9d-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx15e7fba582a245d2a6a9d-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx15e7fba582a245d2a6a9d-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx60671f6be2b94fbba88a5-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx60671f6be2b94fbba88a5-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx60671f6be2b94fbba88a5-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx60671f6be2b94fbba88a5-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txccf46408514d4e138e3dd-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txccf46408514d4e138e3dd-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txccf46408514d4e138e3dd-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: txccf46408514d4e138e3dd-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7e67dbac8dc8473c88c24-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7e67dbac8dc8473c88c24-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7e67dbac8dc8473c88c24-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx7e67dbac8dc8473c88c24-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx68f4c0e1e0544eda9fb65-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx68f4c0e1e0544eda9fb65-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx68f4c0e1e0544eda9fb65-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx68f4c0e1e0544eda9fb65-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txba482b2a5bce4a79b1666-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txba482b2a5bce4a79b1666-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txba482b2a5bce4a79b1666-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: txba482b2a5bce4a79b1666-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx4855fb86ec614cc697380-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx4855fb86ec614cc697380-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx4855fb86ec614cc697380-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx4855fb86ec614cc697380-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx9ded6d12d3c44d52a5409-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx9ded6d12d3c44d52a5409-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx9ded6d12d3c44d52a5409-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx9ded6d12d3c44d52a5409-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx2c1e7673eaba4a8e8ff29-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx2c1e7673eaba4a8e8ff29-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx2c1e7673eaba4a8e8ff29-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx2c1e7673eaba4a8e8ff29-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx8c999da543484f749e8f4-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx8c999da543484f749e8f4-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx8c999da543484f749e8f4-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: tx8c999da543484f749e8f4-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa0dd9c43f1c441aab47d7-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa0dd9c43f1c441aab47d7-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa0dd9c43f1c441aab47d7-00665c9e2b) test ERROR: Container GET returning 503 for [] (txn: txa0dd9c43f1c441aab47d7-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx3037c7e0761f41029dee5-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx3037c7e0761f41029dee5-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx3037c7e0761f41029dee5-00665c9e2b) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx3037c7e0761f41029dee5-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txf8478ab2fa904de29523f-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txf8478ab2fa904de29523f-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txf8478ab2fa904de29523f-00665c9e2b) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txf8478ab2fa904de29523f-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa518447166c44755960c0-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa518447166c44755960c0-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa518447166c44755960c0-00665c9e2b) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txa518447166c44755960c0-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx472376207aa440d588faa-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx472376207aa440d588faa-00665c9e2b) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx472376207aa440d588faa-00665c9e2b) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx472376207aa440d588faa-00665c9e2b) test INFO: Client disconnected on read of '/a/c' (txn: txaea209a7fc924170ae98f-00665c9e2b) test INFO: Client disconnected on read of '/a/c/' (txn: txaea209a7fc924170ae98f-00665c9e2b) test.unit.proxy.test_server.TestContainerController.test_error_limiting test.unit.proxy.test_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_transfer_headers test.unit.proxy.test_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346013.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/6d7/ed6ce9b069e2f0968995e2a49419c6d7/ed6ce9b069e2f0968995e2a49419c6d7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346013.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/6d7/ed6ce9b069e2f0968995e2a49419c6d7/ed6ce9b069e2f0968995e2a49419c6d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346015.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e6f/d6c8a4180993f06d294c2412b8e53e6f/d6c8a4180993f06d294c2412b8e53e6f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346015.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e6f/d6c8a4180993f06d294c2412b8e53e6f/d6c8a4180993f06d294c2412b8e53e6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346017.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/609/37af6c6afd1e241ea5560983f90e2609/37af6c6afd1e241ea5560983f90e2609.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346017.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/609/37af6c6afd1e241ea5560983f90e2609/37af6c6afd1e241ea5560983f90e2609.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346019.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/480/dae6e63c41a59d0e07bd3fddbe6c7480/dae6e63c41a59d0e07bd3fddbe6c7480.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346019.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/480/dae6e63c41a59d0e07bd3fddbe6c7480/dae6e63c41a59d0e07bd3fddbe6c7480.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346021.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/120/3d791edbea77f331a158c62742b16120/3d791edbea77f331a158c62742b16120.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346021.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/120/3d791edbea77f331a158c62742b16120/3d791edbea77f331a158c62742b16120.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346023.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/226/e0d9f0200de68aae8ba99ce2952b6226/e0d9f0200de68aae8ba99ce2952b6226.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346023.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/226/e0d9f0200de68aae8ba99ce2952b6226/e0d9f0200de68aae8ba99ce2952b6226.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346025.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/785/3d3fab948ecee6655707729566a22785/3d3fab948ecee6655707729566a22785.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346025.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/785/3d3fab948ecee6655707729566a22785/3d3fab948ecee6655707729566a22785.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346027.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f69/669acf5f64ea564d99287a5a66cfff69/669acf5f64ea564d99287a5a66cfff69.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346027.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/f69/669acf5f64ea564d99287a5a66cfff69/669acf5f64ea564d99287a5a66cfff69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346029.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/011/1dd97d00eeb3062f4bd91ea1cd7f3011/1dd97d00eeb3062f4bd91ea1cd7f3011.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346029.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/011/1dd97d00eeb3062f4bd91ea1cd7f3011/1dd97d00eeb3062f4bd91ea1cd7f3011.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346031.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/727/45653eb69159c2047e3a85e15d8f0727/45653eb69159c2047e3a85e15d8f0727.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346031.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/727/45653eb69159c2047e3a85e15d8f0727/45653eb69159c2047e3a85e15d8f0727.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346033.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/9ce/6c4d0c467afe5875018699495f5f19ce/6c4d0c467afe5875018699495f5f19ce.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346033.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/9ce/6c4d0c467afe5875018699495f5f19ce/6c4d0c467afe5875018699495f5f19ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346035.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e75/28ec3daecfb350f9d9c78f329820de75/28ec3daecfb350f9d9c78f329820de75.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346035.00000, db: /tmp/tmp_nt7pg3_/sda/containers/0/e75/28ec3daecfb350f9d9c78f329820de75/28ec3daecfb350f9d9c78f329820de75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp177dmtgc/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717345816.00000, (0, 0) as of 1717345816.00000, active as of 1717345816.00000> to ShardRange<'k' to 't' as of 1717345817.00000, (0, 0) as of 1717345817.00000, sharded as of 1717345818.00000>, path: .shards_a/shard_c, db: /tmp/tmp177dmtgc/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmp177dmtgc/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpijlpzilv/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345817.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/78b/d2890ae006c29aefd498a5f5c51e678b/d2890ae006c29aefd498a5f5c51e678b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345820.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/208/b31bc16f2543d08f4a0670f5937a0208/b31bc16f2543d08f4a0670f5937a0208.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345823.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/e03/e8af4b8e29298723c43e8a2b6decce03/e8af4b8e29298723c43e8a2b6decce03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345826.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/6c2/09ba571dadaea6baf83522cbaea666c2/09ba571dadaea6baf83522cbaea666c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345829.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/7d8/8ba98416400f3ae3bbb9f5637c6a57d8/8ba98416400f3ae3bbb9f5637c6a57d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345832.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/563/d6e30c3b808e6abbe01e7f8629cc6563/d6e30c3b808e6abbe01e7f8629cc6563.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345835.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/025/600568b26e686e91b350099bbf036025/600568b26e686e91b350099bbf036025.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345838.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/536/641871e462efeab0b814048fe58ad536/641871e462efeab0b814048fe58ad536.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx3598a46c785d4e559e097-00665c9e1e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server ERROR: Object returning 503 for [] (txn: tx0d29cb9a501b4e3098b15-00665c9e20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.7:1007/sdh (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from primary (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server ERROR: Exception fetching fragments for '/a/c/o': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1496, in put_fragments_in_queue for fragment in frag_iter: File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2518, in _iter_bytes_from_response_part six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2490, in _iter_bytes_from_response_part chunk = part_file.read(self.app.object_chunk_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1133, in read raise ShortReadError( swift.common.exceptions.ShortReadError: Too few bytes; read 415, expecting 490 (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx46f4019c1ea442d39ed3c-00665c9e20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files ... ok test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0097 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0053 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0164 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0125 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0130 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0046 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0248 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0108 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0166 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0089 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0095 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543390 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0109 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0036 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0285 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0036 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container ... ok test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx360a95920302488f92b14-00665c9e2b) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx360a95920302488f92b14-00665c9e2b) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx360a95920302488f92b14-00665c9e2b) test ERROR: Account HEAD returning 503 for [] (txn: tx360a95920302488f92b14-00665c9e2b) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9f8cad0418634efab5ba1-00665c9e2c) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9f8cad0418634efab5ba1-00665c9e2c) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9f8cad0418634efab5ba1-00665c9e2c) test ERROR: Account HEAD returning 503 for [] (txn: tx9f8cad0418634efab5ba1-00665c9e2c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx9f4245b294f0471e9e5f3-00665c9e2e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx7d5d5f7598e8476bb750f-00665c9e2e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx4c667f3d4bd54981bb7ca-00665c9e2e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers ... ok test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type ... ok test.unit.common.test_daemon.TestRunDaemon.test_fork_workers test.unit.common.test_daemon.TestRunDaemon.test_fork_workers ... ok test.unit.common.test_daemon.TestRunDaemon.test_forked_worker test.unit.common.test_daemon.TestRunDaemon.test_forked_worker ... ok test.unit.common.test_daemon.TestRunDaemon.test_restart_workers test.unit.common.test_daemon.TestRunDaemon.test_restart_workers ... ok test.unit.common.test_daemon.TestRunDaemon.test_run test.unit.common.test_daemon.TestRunDaemon.test_run ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon test.unit.common.test_daemon.TestRunDaemon.test_run_daemon ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file ... ok test.unit.common.test_daemon.TestRunDaemon.test_signal test.unit.common.test_daemon.TestRunDaemon.test_signal ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup ... ok test.unit.common.test_db.TestGetDBConnection.test_invalid_path test.unit.common.test_db.TestGetDBConnection.test_invalid_path ... ok test.unit.common.test_db.TestGetDBConnection.test_locked_db test.unit.common.test_db.TestGetDBConnection.test_locked_db ... ok test.unit.common.test_db.TestGetDBConnection.test_normal_case test.unit.common.test_db.TestGetDBConnection.test_normal_case ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header ... ok test.unit.common.test_utils.TestShardRange.test_constants test.unit.common.test_utils.TestShardRange.test_constants ... ok test.unit.common.test_utils.TestShardRange.test_copy test.unit.common.test_utils.TestShardRange.test_copy ... ok test.unit.common.test_utils.TestShardRange.test_deleted_setter test.unit.common.test_utils.TestShardRange.test_deleted_setter ... ok test.unit.common.test_utils.TestShardRange.test_epoch_setter test.unit.common.test_utils.TestShardRange.test_epoch_setter ... ok test.unit.common.test_utils.TestShardRange.test_find_ancestors test.unit.common.test_utils.TestShardRange.test_find_ancestors ... ok test.unit.common.test_utils.TestShardRange.test_find_root test.unit.common.test_utils.TestShardRange.test_find_root ... ok test.unit.common.test_utils.TestShardRange.test_increment_meta test.unit.common.test_utils.TestShardRange.test_increment_meta ... ok test.unit.common.test_utils.TestShardRange.test_is_child_of test.unit.common.test_utils.TestShardRange.test_is_child_of ... ok test.unit.common.test_utils.TestShardRange.test_make_path test.unit.common.test_utils.TestShardRange.test_make_path ... ok test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_min_max_bounds test.unit.common.test_utils.TestShardRange.test_min_max_bounds ... ok test.unit.common.test_utils.TestShardRange.test_name test.unit.common.test_utils.TestShardRange.test_name ... ok test.unit.common.test_utils.TestShardRange.test_name_validation test.unit.common.test_utils.TestShardRange.test_name_validation ... ok test.unit.common.test_utils.TestShardRange.test_repr test.unit.common.test_utils.TestShardRange.test_repr ... ok test.unit.common.test_utils.TestShardRange.test_resolve_state test.unit.common.test_utils.TestShardRange.test_resolve_state ... ok test.unit.common.test_utils.TestShardRange.test_row_count test.unit.common.test_utils.TestShardRange.test_row_count ... ok test.unit.common.test_utils.TestShardRange.test_set_deleted test.unit.common.test_utils.TestShardRange.test_set_deleted ... ok test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation ... ok test.unit.common.test_utils.TestShardRange.test_sort_key test.unit.common.test_utils.TestShardRange.test_sort_key ... ok test.unit.common.test_utils.TestShardRange.test_sort_key_order test.unit.common.test_utils.TestShardRange.test_sort_key_order ... ok test.unit.common.test_utils.TestShardRange.test_state_setter test.unit.common.test_utils.TestShardRange.test_state_setter ... ok test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_timestamp_setter test.unit.common.test_utils.TestShardRange.test_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_to_from_dict test.unit.common.test_utils.TestShardRange.test_to_from_dict ... ok test.unit.common.test_utils.TestShardRange.test_update_meta test.unit.common.test_utils.TestShardRange.test_update_meta ... ok test.unit.common.test_utils.TestShardRange.test_update_state test.unit.common.test_utils.TestShardRange.test_update_state ... ok test.unit.common.test_utils.TestShardRange.test_update_tombstones test.unit.common.test_utils.TestShardRange.test_update_tombstones ... ok test.unit.common.test_utils.TestUTC.test_tzname test.unit.common.test_utils.TestUTC.test_tzname ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function ... ok test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible ... ok test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test WARNING: The following digest algorithms are configured but not supported: md5, not-a-valid-digest test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Worker mock-pid-0 exited my-daemon DEBUG: Worker mock-pid-1 exited my-daemon DEBUG: Worker mock-pid-2 exited my-daemon NOTICE: Finished 3543412 my-daemon DEBUG: Forked worker 3543412 finished my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Daemon needs to change options, aborting workers my-daemon DEBUG: Cleaned up worker mock-pid-0 my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-3 with {} my-daemon DEBUG: Spawned worker mock-pid-4 with {} my-daemon DEBUG: Spawned worker mock-pid-5 with {} my-daemon DEBUG: Cleaned up worker mock-pid-3 my-daemon DEBUG: Cleaned up worker mock-pid-4 my-daemon DEBUG: Cleaned up worker mock-pid-5 my-daemon NOTICE: SIGTERM received () my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Worker mock-pid exited my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon NOTICE: Worker mock-pid died my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 sharder-test DEBUG: Unlinked retiring db '/tmp/tmpj2ekx48p/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.11:1011/sdl (txn: tx0c24aeef52db48d7a5d71-00665c9e20) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.6:1006/sdg (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbd3d505c0782433894658-00665c9e27) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: txbd3d505c0782433894658-00665c9e27) proxy-server ERROR: ERROR Unhandled exception in request: ChunkReadTimeout (0.01s after 0.02s) (txn: txbd3d505c0782433894658-00665c9e27) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.10:1010/sdk (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from primary (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: tx7a3de3e3bc1841e5addc0-00665c9e27) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4f719ac48c5c44668457f-00665c9e27) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries ... ok test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0034 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0486 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0019 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:40 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:41 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0102 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0020 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0152 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0021 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0020 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0025 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0143 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0142 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmprz_r4ah7/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0041 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0155 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.1716 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmp125dxze0/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0041 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0130 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0058 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0034 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0035 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0034 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0123 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0175 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0035 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0122 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0117 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0034 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.23982', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.24419', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.24827', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.25155', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.25595', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.26180', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.26601', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.27025', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.27418', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.29184', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345825.29590', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile ... ok REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345818.95380', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345816.72140', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345816.72140', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345812.30587', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717345816.68977', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345812.30587', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345818.87388', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717345818.91964', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345818.87388', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345816.63780', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345816.63780', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345816.63763', 'epoch': '1717345816.63763', 'reported': 0, 'tombstones': -1}], '93e74562-c5eb-4b83-abdb-54fee3a6a7a1-sda') test DEBUG: /tmp/tmpopo9pg9l/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.0:1000/sda test WARNING: /tmp/tmpopo9pg9l/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.0:1000/sda; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '242f950c96009eba7906789e27de18d4', 'bc7669b8-28ed-4290-b3d2-c2550070a216-sda', '1717345824.03414', '1717345824.02946', '0', '', '1717345824.02946', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': 'a/c', 'timestamp': '1717345824.05822', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.05822', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345824.05799', 'epoch': '1717345824.05799', 'reported': 0, 'tombstones': -1}], 'bc7669b8-28ed-4290-b3d2-c2550070a216-sda') test ERROR: ERROR Bad response 500 from 10.0.0.1 test DEBUG: /tmp/tmpb4yedjdo/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345824.05763', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], 'bc7669b8-28ed-4290-b3d2-c2550070a216-sda') test DEBUG: /tmp/tmpb4yedjdo/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0107222s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8114f04f-1148-4a9a-99c3-3b7b7cec2961-sda', '1717345824.12423', '1717345824.11976', '0', '', '1717345824.11976', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345824.14483', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345824.14483', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.14483', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345824.14489', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345824.14489', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.14489', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345824.14492', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345824.14492', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.14492', 'epoch': None, 'reported': 0, 'tombstones': -1}], '8114f04f-1148-4a9a-99c3-3b7b7cec2961-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '01f66ec9-40a4-48e0-97a3-c405707abf5a-sda', '1717345824.18227', '1717345824.17816', '0', '', '1717345824.17816', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345824.20037', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345824.20037', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.20037', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345824.20043', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345824.20043', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.20043', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345824.20045', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345824.20045', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.20045', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11929', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11929', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11929', 'epoch': None, 'reported': 0, 'tombstones': -1}], '01f66ec9-40a4-48e0-97a3-c405707abf5a-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ec804446-36e2-43b7-b724-b6cf52fba179-sda', '1717345824.24915', '1717345824.24513', '0', '', '1717345824.24513', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345824.26739', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345824.26739', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.26739', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345824.26746', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345824.26746', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.26746', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345824.26749', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345824.26749', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345824.26749', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11929', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11929', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11929', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'ec804446-36e2-43b7-b724-b6cf52fba179-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '143788e8-66f2-4c7a-a510-4b366ca832f8-sda', '1717345826.69524', '1717345826.69152', '0', '', '1717345826.69152', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345826.71542', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345826.71542', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.71542', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345826.71547', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345826.71547', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.71547', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345826.71550', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345826.71550', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.71550', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11929', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11929', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11929', 'epoch': None, 'reported': 0, 'tombstones': -1}], '143788e8-66f2-4c7a-a510-4b366ca832f8-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e18280eb-6feb-4049-8e8f-51de5ebac9ff-sda', '1717345826.77170', '1717345826.76755', '0', '', '1717345826.76755', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345826.78984', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345826.78984', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.78984', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345826.78990', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345826.78990', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.78990', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345826.78993', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345826.78993', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.78993', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11929', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11929', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11929', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'e18280eb-6feb-4049-8e8f-51de5ebac9ff-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3cb42acb-131d-4586-8c5d-fe741e3bd469-sda', '1717345826.84156', '1717345826.83756', '0', '', '1717345826.83756', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345826.85973', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345826.85973', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.85973', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345826.85979', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345826.85979', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.85979', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345826.85982', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345826.85982', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.85982', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11948', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11948', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11948', 'epoch': '1717345824.11951', 'reported': 0, 'tombstones': -1}], '3cb42acb-131d-4586-8c5d-fe741e3bd469-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd6c274c4-c31e-4fb9-a4c7-4f486b333e29-sda', '1717345826.90720', '1717345826.90318', '0', '', '1717345826.90318', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345826.92447', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345826.92447', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.92447', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345826.92453', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345826.92453', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.92453', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345826.92456', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345826.92456', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.92456', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11948', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11948', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11948', 'epoch': '1717345824.11951', 'reported': 0, 'tombstones': -1}], 'd6c274c4-c31e-4fb9-a4c7-4f486b333e29-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '63a497c5-9620-42fb-8a12-d3ac28f9ff48-sda', '1717345826.97612', '1717345826.97174', '0', '', '1717345826.97174', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345826.99347', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345826.99347', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.99347', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345826.99354', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345826.99354', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.99354', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345826.99358', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345826.99358', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345826.99358', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11948', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11948', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11948', 'epoch': '1717345824.11951', 'reported': 0, 'tombstones': -1}], '63a497c5-9620-42fb-8a12-d3ac28f9ff48-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c6af29ef-9a55-431f-8a84-31e8f3efbadf-sda', '1717345829.02869', '1717345829.02427', '0', '', '1717345829.02427', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345829.04750', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345829.04750', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.04750', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345829.04754', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345829.04754', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.04754', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345829.04757', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345829.04757', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.04757', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11962', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11962', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11962', 'epoch': '1717345824.11965', 'reported': 0, 'tombstones': -1}], 'c6af29ef-9a55-431f-8a84-31e8f3efbadf-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2b4eb88b-be8e-46a0-a325-35609cc57685-sda', '1717345829.11430', '1717345829.10848', '0', '', '1717345829.10848', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345829.13306', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345829.13306', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.13306', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345829.13312', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345829.13312', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.13312', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345829.13315', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345829.13315', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.13315', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11948', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11948', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11948', 'epoch': '1717345824.11951', 'reported': 0, 'tombstones': -1}], '2b4eb88b-be8e-46a0-a325-35609cc57685-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9ad8e8df-0c83-48a8-8cec-906b7e66f469-sda', '1717345829.24558', '1717345829.24146', '0', '', '1717345829.24146', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345829.26121', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345829.26121', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.26121', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345829.26126', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345829.26126', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.26126', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345829.26129', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345829.26129', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.26129', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11948', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11948', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11948', 'epoch': '1717345824.11951', 'reported': 0, 'tombstones': -1}], '9ad8e8df-0c83-48a8-8cec-906b7e66f469-sda') test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'dd009a8f-d160-404c-ae0c-427c1006d7fc-sda', '1717345829.31594', '1717345829.31273', '0', '', '1717345829.31273', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync ... ok test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (201, 404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (201, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from primary (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Ignoring 404 from handoff (txn: txaf26c8bed0c444a087f75-00665c9e28) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Ignoring 404 from primary (txn: tx40631c4e873d4776af8e7-00665c9e2f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txc93c1775aff24c208de55-00665c9e2f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to GET /a/c/o From Object Server 10.0.0.6:1006/sdg (txn: tx1bbb61309c4d43fd9ecec-00665c9e2f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3dfb26f641924a719e4e1-00665c9e30) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717345831.22670', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T163031Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345831.24532', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0174 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0112 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0041 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0008 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0006 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0007 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0007 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0006 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0006 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0007 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0007 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0006 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0006 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0006 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0007 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0007 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0006 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0148 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0046 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0044 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0018 "-" 3543390 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0007 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0137 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0101 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0124 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0036 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0093 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0089 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0041 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0036 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0042 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0037 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:41 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0011 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container ... ok swift: Client disconnected on read of '/a' (txn: tx21e905bb61304424be9cc-00665c9e49) test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345829.33061', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345829.33061', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.33061', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345829.33066', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345829.33066', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.33066', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345829.33068', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345829.33068', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345829.33068', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345824.11974', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11929', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11929', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'dd009a8f-d160-404c-ae0c-427c1006d7fc-sda') test WARNING: Ignoring remote osr w/o epoch, own_sr: {'name': 'a/c', 'timestamp': '1717345824.11948', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11948', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11948', 'epoch': '1717345824.11951', 'reported': 0, 'tombstones': -1}, remote_sr: {'name': 'a/c', 'timestamp': '1717345824.11974', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345824.11929', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345824.11929', 'epoch': None, 'reported': 0, 'tombstones': -1}, source: repl_req test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpuu0jovff/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '6e11d25f36dfa28853dcb7edf8e54224', '5af2550f-f1ff-47ff-8878-71e30f337769-sda', '1717345833.78178', '1717345833.77480', '0', '', '1717345833.77480', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345833.80407', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '5af2550f-f1ff-47ff-8878-71e30f337769-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test DEBUG: Replicating db /tmp/tmpszhr0nmb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'df344ad0-9c0e-42ab-a3d1-10b010d31fdb-sda', '1717345833.87391', '1717345833.86609', '0', '', '1717345833.86609', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test ERROR: ERROR syncing /tmp/tmpszhr0nmb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db with node {'ip': '10.0.0.1', 'replication_ip': '10.0.0.1', 'port': 1001, 'replication_port': 1001, 'device': 'sdb', 'zone': 1, 'region': 1, 'id': 1, 'weight': 1, 'index': 1}: Timeout (0.001s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '18d726dfd2f09ed97281c92ca11a6d31', '856fa515-d655-44c6-9426-d5a031332ec7-sda', '1717345833.94840', '1717345833.93995', '0', '', '1717345833.93995', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717345833.97974', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '856fa515-d655-44c6-9426-d5a031332ec7-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptizsj39h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '90de309b-547f-4fd5-a0ab-5a99f5f53137-sda', '1717345834.04752', '1717345834.03988', '0', '', '1717345834.03988', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('90de309b-547f-4fd5-a0ab-5a99f5f53137-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '90de309b-547f-4fd5-a0ab-5a99f5f53137-sda', '1717345834.04752', '1717345834.03988', '0', '', '1717345834.03988', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('90de309b-547f-4fd5-a0ab-5a99f5f53137-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04149 seconds (24.10320/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptizsj39h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '90de309b-547f-4fd5-a0ab-5a99f5f53137-sda', '1717345834.04752', '1717345834.03988', '0', '', '1717345834.03988', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345834.14044', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345834.14044', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345834.14044', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345834.07480', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345834.07480', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345834.07480', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345834.07484', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345834.07484', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345834.07484', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345834.07588', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345834.07588', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345834.07488', 'epoch': '1717345834.07488', 'reported': 0, 'tombstones': -1}], '90de309b-547f-4fd5-a0ab-5a99f5f53137-sda') test DEBUG: /tmp/tmptizsj39h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmptizsj39h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '90de309b-547f-4fd5-a0ab-5a99f5f53137-sda', '1717345834.04752', '1717345834.03988', '0', '', '1717345834.03988', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/30/42 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0016 - - 1717345842.516131639 1717345842.517775297 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/30/42 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0014 - - 1717345842.518824100 1717345842.520251036 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware Sun Jun 2 04:30:52 2024 /tmp/log/swift/profile/default.profile456 Sun Jun 2 04:30:52 2024 /tmp/log/swift/profile/default.profile123 Sun Jun 2 04:30:52 2024 /tmp/log/swift/profile/default.profile3543396 45 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 9 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 9 0.001 0.000 0.001 0.000 :0(exec) 9 0.000 0.000 0.000 0.000 :1() 9 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 9 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile123 Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile3543396 Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001Traceback (most recent call last): test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code ... ok File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render ... ok test.unit.common.test_db.TestExampleBroker.test_commit_pending test.unit.common.test_db.TestExampleBroker.test_commit_pending ... ok test.unit.common.test_db.TestExampleBroker.test_delete_db test.unit.common.test_db.TestExampleBroker.test_delete_db ... ok test.unit.common.test_db.TestExampleBroker.test_get_info test.unit.common.test_db.TestExampleBroker.test_get_info ... ok test.unit.common.test_db.TestExampleBroker.test_get_max_row test.unit.common.test_db.TestExampleBroker.test_get_max_row ... ok test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata ... ok test.unit.common.test_db.TestExampleBroker.test_get_syncs test.unit.common.test_db.TestExampleBroker.test_get_syncs ... ok test.unit.common.test_db.TestExampleBroker.test_maybe_get test.unit.common.test_db.TestExampleBroker.test_maybe_get ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_put_timestamp test.unit.common.test_db.TestExampleBroker.test_put_timestamp ... ok test.unit.common.test_db.TestExampleBroker.test_status_changed_at test.unit.common.test_db.TestExampleBroker.test_status_changed_at ... ok test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717345834.14044', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717345834.14044', 'deleted': 1, 'state': 10, 'state_timestamp': '1717345834.14044', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717345834.07480', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717345834.07480', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345834.07480', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717345834.07484', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717345834.07484', 'deleted': 0, 'state': 10, 'state_timestamp': '1717345834.07484', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345834.07588', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345834.07588', 'deleted': 0, 'state': 60, 'state_timestamp': '1717345834.07488', 'epoch': '1717345834.07488', 'reported': 0, 'tombstones': -1}], '90de309b-547f-4fd5-a0ab-5a99f5f53137-sda') test DEBUG: /tmp/tmptizsj39h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.2:1002/sdc test WARNING: /tmp/tmptizsj39h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.2:1002/sdc; have shard ranges and will wait for cleaving test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.39619 seconds (0.41733/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '521f8326-f67b-46b4-826a-1862c7ef4b99-sda', '1717345839.78039', '1717345839.77564', '1717345839.7994', '', '1717345839.7994', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkkexj_yl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8e672861-2f9f-4c8b-ba63-d32c0524c7e9-sda', '1717345839.83438', '1717345839.00000', '0', '', '1717345839.00000', 0, 1) test DEBUG: /tmp/tmpkkexj_yl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8e672861-2f9f-4c8b-ba63-d32c0524c7e9-sda', '1717345839.83438', '1717345839.00000', '0', '', '1717345839.00000', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8e672861-2f9f-4c8b-ba63-d32c0524c7e9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06397 seconds (15.63182/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp35_bksmh/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '5f480f92-1dec-469c-9d0f-b7758141fffd-sda', '1717345839.95587', '1717345840.00000', '0', '{"X-Container-Sync-To": ["a", "1717345841.00000"], "X-Container-Sync-Key": ["secret", "1717345842.00000"]}', '1717345840.00000', 0, 0) REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('5f480f92-1dec-469c-9d0f-b7758141fffd-sda', 'c48d89b33840539f9c5442ab3b10027d.db') REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '5f480f92-1dec-469c-9d0f-b7758141fffd-sda', '1717345839.95587', '1717345840.00000', '0', '{"X-Container-Sync-To": ["a", "1717345841.00000"], "X-Container-Sync-Key": ["secret", "1717345842.00000"]}', '1717345840.00000', 0, 0) REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('5f480f92-1dec-469c-9d0f-b7758141fffd-sda', 'c48d89b33840539f9c5442ab3b10027d.db') test DEBUG: Replicating db /tmp/tmp35_bksmh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7d8142a1-13fd-44b5-9e9c-d893f63bd8b0-sda', '1717345839.94296', '1717345839.00000', '0', '', '1717345839.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7d8142a1-13fd-44b5-9e9c-d893f63bd8b0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7d8142a1-13fd-44b5-9e9c-d893f63bd8b0-sda', '1717345839.94296', '1717345839.00000', '0', '', '1717345839.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7d8142a1-13fd-44b5-9e9c-d893f63bd8b0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.10633 seconds (18.80894/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:4 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp35_bksmh/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '5f480f92-1dec-469c-9d0f-b7758141fffd-sda', '1717345839.95587', '1717345840.00000', '0', '{"X-Container-Sync-To": ["", "1717345843.00000"], "X-Container-Sync-Key": ["secret", "1717345842.00000"]}', '1717345840.00000', 0, 0) test DEBUG: /tmp/tmp35_bksmh/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '5f480f92-1dec-469c-9d0f-b7758141fffd-sda', '1717345839.95587', '1717345840.00000', '0', '{"X-Container-Sync-To": ["", "1717345843.00000"], "X-Container-Sync-Key": ["secret", "1717345842.00000"]}', '1717345840.00000', 0, 0) test DEBUG: /tmp/tmp35_bksmh/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.2:1002/sdc, nothing to do test DEBUG: Replicating db /tmp/tmp35_bksmh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7d8142a1-13fd-44b5-9e9c-d893f63bd8b0-sda', '1717345839.94296', '1717345839.00000', '0', '', '1717345839.00000', 0, 0) test DEBUG: /tmp/tmp35_bksmh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7d8142a1-13fd-44b5-9e9c-d893f63bd8b0-sda', '1717345839.94296', '1717345839.00000', '0', '', '1717345839.00000', 0, 0) test DEBUG: /tmp/tmp35_bksmh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.04250 seconds (47.06397/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:4 remote_merge:0 rsync:0 ts_repl:0 test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception ... ok test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit ... ok test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0011 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0113 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0029 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0021 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:42 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0099 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0020 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0140 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0145 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0052 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0028 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0032 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0032 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0031 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmprw092qnf/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0078 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0041 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0036 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0153 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0128 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0035 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0064 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0035 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0156 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0124 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0151 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0059 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0154 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0096 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0153 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0086 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0028 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:42 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0026 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T163035Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345835.97867', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T163035Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345835.99549', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission ... ok 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile123 Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile3543396 Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile456 30 function calls in 0.001 seconds Ordered by: call count ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :0(setprofile) 0 0.000 0.000 profile:0(profiler) Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile123 Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile3543396 Sun Jun 2 04:30:52 2024 /tmp/tmprxdz54rw/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile3543396 Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile3543396 Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile123 Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile3543396 Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile3543396 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.000 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile3543396 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.000 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:30:56 2024 /tmp/tmpxovttoam/unittest.profile3543396 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.000 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple ... ok test.unit.common.test_utils.TestFileLikeIter.test_close test.unit.common.test_utils.TestFileLikeIter.test_close ... ok test.unit.common.test_utils.TestFileLikeIter.test_get_hub test.unit.common.test_utils.TestFileLikeIter.test_get_hub ... ok test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter ... ok test.unit.common.test_utils.TestFileLikeIter.test_next test.unit.common.test_utils.TestFileLikeIter.test_next ... ok test.unit.common.test_utils.TestFileLikeIter.test_read test.unit.common.test_utils.TestFileLikeIter.test_read ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size test.unit.common.test_utils.TestFileLikeIter.test_read_with_size ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline test.unit.common.test_utils.TestFileLikeIter.test_readline ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline2 test.unit.common.test_utils.TestFileLikeIter.test_readline2 ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline3 test.unit.common.test_utils.TestFileLikeIter.test_readline3 ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines test.unit.common.test_utils.TestFileLikeIter.test_readlines ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size ... ok test.unit.common.test_utils.TestPairs.test_pairs test.unit.common.test_utils.TestPairs.test_pairs ... ok test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_default test.unit.obj.test_server.TestConfigOptionHandling.test_default ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default ... ok test.unit.obj.test_ssync_sender.TestSender.test_call test.unit.obj.test_ssync_sender.TestSender.test_call ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect test.unit.obj.test_ssync_sender.TestSender.test_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect test.unit.obj.test_ssync_sender.TestSender.test_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all test.unit.obj.test_ssync_sender.TestSender.test_readline_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete test.unit.obj.test_ssync_sender.TestSender.test_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post test.unit.obj.test_ssync_sender.TestSender.test_send_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put test.unit.obj.test_ssync_sender.TestSender.test_send_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_post test.unit.obj.test_ssync_sender.TestSender.test_updates_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put test.unit.obj.test_ssync_sender.TestSender.test_updates_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345841.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/df5/9b5d5d50ed914c8dc3913b84e6045df5/9b5d5d50ed914c8dc3913b84e6045df5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345844.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/9ca/2983df72e4e269e3b94e72be5b0429ca/2983df72e4e269e3b94e72be5b0429ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345847.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/68e/02c60b29fedc06909056c2660953068e/02c60b29fedc06909056c2660953068e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345850.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/d1e/db49e2e99f7ef32b2ebe14c697d32d1e/db49e2e99f7ef32b2ebe14c697d32d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345853.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/ca1/a87083603adfbb345b30597f99732ca1/a87083603adfbb345b30597f99732ca1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345856.00000-0, db: /tmp/tmpu9bth0e7/sda/containers/0/46b/d9d6a0063ccf348d96d880b13688446b/d9d6a0063ccf348d96d880b13688446b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345825.00000-0, db: /tmp/tmpbd62x2dz/sda/containers/0/c2b/c5a98a82cdc33bd2aa5b48e63bddcc2b/c5a98a82cdc33bd2aa5b48e63bddcc2b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345829.00000-0, db: /tmp/tmpbd62x2dz/sda/containers/0/7d8/8ba98416400f3ae3bbb9f5637c6a57d8/8ba98416400f3ae3bbb9f5637c6a57d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345827.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/fdf/bc59eaa8477b3dc14f979f6d49281fdf/bc59eaa8477b3dc14f979f6d49281fdf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345831.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/648/5d054da4c811982b1ec46474cc885648/5d054da4c811982b1ec46474cc885648.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345835.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/025/600568b26e686e91b350099bbf036025/600568b26e686e91b350099bbf036025.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345839.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/5e3/aa48eb23cb4abb86c0917980853135e3/aa48eb23cb4abb86c0917980853135e3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345839.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/5e3/aa48eb23cb4abb86c0917980853135e3/aa48eb23cb4abb86c0917980853135e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345843.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/cc5/133b0ea6c0251a0aceb196fb46903cc5/133b0ea6c0251a0aceb196fb46903cc5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345847.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/68e/02c60b29fedc06909056c2660953068e/02c60b29fedc06909056c2660953068e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345851.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345855.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/0b3/7da809de52d6c0590cd8f1957c7d40b3/7da809de52d6c0590cd8f1957c7d40b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345859.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/82a/5c7d54d86457214ab2b267672a27a82a/5c7d54d86457214ab2b267672a27a82a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345863.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/f35/b7205c0fa015d4356e1a4e6c19164f35/b7205c0fa015d4356e1a4e6c19164f35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345867.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/10a/103cd16479c54c9e582618292a3c110a/103cd16479c54c9e582618292a3c110a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345871.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/577/ff75c4e36d977172a3d89f3beab16577/ff75c4e36d977172a3d89f3beab16577.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345871.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/577/ff75c4e36d977172a3d89f3beab16577/ff75c4e36d977172a3d89f3beab16577.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345875.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345879.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/277/b45fb34ccf3414c4389ce686a2cba277/b45fb34ccf3414c4389ce686a2cba277.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345883.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/a86/9f96247331cfe4b0e61fad020caada86/9f96247331cfe4b0e61fad020caada86.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345887.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/e71/1484e75f1d95065b67c23a8eff750e71/1484e75f1d95065b67c23a8eff750e71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345891.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/20d/4eb4de89a5b06c3f96b6b1e27ba4320d/4eb4de89a5b06c3f96b6b1e27ba4320d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345895.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345903.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/892/b50ba0993aeed6c54f87327f9a014892/b50ba0993aeed6c54f87327f9a014892.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345903.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/892/b50ba0993aeed6c54f87327f9a014892/b50ba0993aeed6c54f87327f9a014892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345907.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345915.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d41/740cb5d220733796133a1347f1910d41/740cb5d220733796133a1347f1910d41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345923.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/0bd/4cb334d4f01ecd9abfdf3bdca8df30bd/4cb334d4f01ecd9abfdf3bdca8df30bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345927.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/000/9cc39cf5e202188f3cb365ab1f84b000/9cc39cf5e202188f3cb365ab1f84b000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345939.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d69/82df5f6d5f930f5efcd5c422ac1bcd69/82df5f6d5f930f5efcd5c422ac1bcd69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345947.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/a17/d7ac936fa77aa2f44d71ebdc60230a17/d7ac936fa77aa2f44d71ebdc60230a17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345951.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/6f3/5145c6253305303db4393a56882626f3/5145c6253305303db4393a56882626f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345963.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/1c6/923db0dcb5f23439340d0336b4df81c6/923db0dcb5f23439340d0336b4df81c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345971.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345975.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/1c6/06727a10627bb589627c0835996451c6/06727a10627bb589627c0835996451c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345983.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/2fe/14e8e0360f977223aacbcdb1876582fe/14e8e0360f977223aacbcdb1876582fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345987.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/a2f/0976486a84408c4d0a9612b9c99f0a2f/0976486a84408c4d0a9612b9c99f0a2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345995.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345999.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345999.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346007.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d9a/cd426bceff16dee8a0035688912e7d9a/cd426bceff16dee8a0035688912e7d9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346011.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/9b1/cecec856fd6c654aee8d170536e519b1/cecec856fd6c654aee8d170536e519b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346023.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/f7e/c76a6f341fe0b5daa394696434d01f7e/c76a6f341fe0b5daa394696434d01f7e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346035.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/037/21fe1f4898825a6a856e1f376b464037/21fe1f4898825a6a856e1f376b464037.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346043.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/34e/65d7cd5609f1cd05bcfa5f850928b34e/65d7cd5609f1cd05bcfa5f850928b34e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346047.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/e10/0c21378f348cc36f4019f220b9ec9e10/0c21378f348cc36f4019f220b9ec9e10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346051.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346055.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/76b/8517e9828eb7eca20964951da6d8176b/8517e9828eb7eca20964951da6d8176b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346059.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/56f/9a8b64160e6ae421965eb79c4a6cc56f/9a8b64160e6ae421965eb79c4a6cc56f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346063.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c7a/87b0db4a3a68a6cad2f717ee23cacc7a/87b0db4a3a68a6cad2f717ee23cacc7a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346063.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c7a/87b0db4a3a68a6cad2f717ee23cacc7a/87b0db4a3a68a6cad2f717ee23cacc7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346067.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/0eb/8bffce534889c3e9760d4e4c7707f0eb/8bffce534889c3e9760d4e4c7707f0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346071.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/32d/2293f80a7b9db3f5f9ab86f42c9cb32d/2293f80a7b9db3f5f9ab86f42c9cb32d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346075.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/256/8c28cad3e3414d4c0358c374c3b65256/8c28cad3e3414d4c0358c374c3b65256.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346079.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/078/6433e58e0348b3f47247aa2b74b0e078/6433e58e0348b3f47247aa2b74b0e078.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346083.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/b8c/a4b071d63381c76854cda44089211b8c/a4b071d63381c76854cda44089211b8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346087.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/01f/413fde947036f52b7637fa6d4298c01f/413fde947036f52b7637fa6d4298c01f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346091.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d8a/ae0bf7520b1d386abbb62849a3aebd8a/ae0bf7520b1d386abbb62849a3aebd8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346095.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/391/8e831b661fc3b45c8576186e7931e391/8e831b661fc3b45c8576186e7931e391.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346099.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/a70/a7c571ebacba5fa57e73a3256a942a70/a7c571ebacba5fa57e73a3256a942a70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346103.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/afc/b972e9e7bc66ded2e87f447ac6b97afc/b972e9e7bc66ded2e87f447ac6b97afc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346107.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/fe4/0cb09fec16b7c585fb6d22a9e8161fe4/0cb09fec16b7c585fb6d22a9e8161fe4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346111.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/254/848a0bad6b01f855062fb20a421a5254/848a0bad6b01f855062fb20a421a5254.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346115.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/9d2/a40147a7d26491322ccdad84eee269d2/a40147a7d26491322ccdad84eee269d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346119.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/2d7/2128a81fd3c0376e7654966effa7b2d7/2128a81fd3c0376e7654966effa7b2d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346123.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/4b2/6d56bb7b7edde6f086c86832f377b4b2/6d56bb7b7edde6f086c86832f377b4b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346127.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/d8e/35b48c1b178f799b98530b86a361ed8e/35b48c1b178f799b98530b86a361ed8e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346131.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c8f/8b5fa0e2c718d6f7e374bea4bee66c8f/8b5fa0e2c718d6f7e374bea4bee66c8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346135.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/46a/563bf149d2efaedf49bff236b486a46a/563bf149d2efaedf49bff236b486a46a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346139.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/368/3b8b2bde8514da353a6b30929a3a8368/3b8b2bde8514da353a6b30929a3a8368.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346143.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/064/e5c13020b853f8d55b97881b42d09064/e5c13020b853f8d55b97881b42d09064.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346147.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/2cd/441bf842753d8554ae4a23602dfd52cd/441bf842753d8554ae4a23602dfd52cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346151.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/010/c2cb8f1945cc7ca7ac309d1e63ad4010/c2cb8f1945cc7ca7ac309d1e63ad4010.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346155.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/dc9/1eed52dc5f057b26476cc823ddeb4dc9/1eed52dc5f057b26476cc823ddeb4dc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346159.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/2e0/72f9effd13247b9a8a1bf5b5627fe2e0/72f9effd13247b9a8a1bf5b5627fe2e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346163.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/809/95d2799e5709d15d5b42ea0651164809/95d2799e5709d15d5b42ea0651164809.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346167.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/7e6/18a91654120765952e370dd788d8f7e6/18a91654120765952e370dd788d8f7e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346171.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/eee/81c953943458438b4df515817d309eee/81c953943458438b4df515817d309eee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346175.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/e3d/14d062c63df20c011263f7150f018e3d/14d062c63df20c011263f7150f018e3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346179.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/5a8/e8d306be8eace3b9ffde02c606c385a8/e8d306be8eace3b9ffde02c606c385a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346183.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/b86/532bdcf25265362d595d53c2cba21b86/532bdcf25265362d595d53c2cba21b86.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346187.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c04/e381e633b29687d112983524c9eaac04/e381e633b29687d112983524c9eaac04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346191.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/ae0/8b9df3d8cd8d9b4c5c356d4eb850bae0/8b9df3d8cd8d9b4c5c356d4eb850bae0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346195.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/42e/c05cabe948b3cb0061f0e3d393d9e42e/c05cabe948b3cb0061f0e3d393d9e42e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346199.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/013/5fc61e3cfcebf5a8ce3b99cc5a58b013/5fc61e3cfcebf5a8ce3b99cc5a58b013.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346203.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/87b/780605a312f5c656e97f9d3bc903387b/780605a312f5c656e97f9d3bc903387b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346207.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/3de/fa496eb72f9b121f30cd553ad21ff3de/fa496eb72f9b121f30cd553ad21ff3de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346211.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/177/d3c13864b9d488fdc3af082c89586177/d3c13864b9d488fdc3af082c89586177.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346215.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/5c1/233d754a88059a5c8eb5dcc0896045c1/233d754a88059a5c8eb5dcc0896045c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346219.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/367/a959f8f9577df2fbf75cd54314923367/a959f8f9577df2fbf75cd54314923367.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346223.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/39e/7cdcd3ce7e324952f2a64d13b5f1639e/7cdcd3ce7e324952f2a64d13b5f1639e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346227.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/be7/414c2e8c8ca30bbd28f259de4bc5ebe7/414c2e8c8ca30bbd28f259de4bc5ebe7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346231.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/13a/fefc29cd793b61103d1fb7f7a4ae613a/fefc29cd793b61103d1fb7f7a4ae613a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346235.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/dc8/1f37e5bc1ea5989578422387201c4dc8/1f37e5bc1ea5989578422387201c4dc8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346239.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/e6b/1cc7179e299f33ae2a340c8eba37ae6b/1cc7179e299f33ae2a340c8eba37ae6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346243.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/4d6/605050656b69728cea4ee398f8d874d6/605050656b69728cea4ee398f8d874d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346247.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c5a/54f02af01ed4e14764e8252a5d52cc5a/54f02af01ed4e14764e8252a5d52cc5a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346251.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/266/5e734352455c242c623aaa618c70c266/5e734352455c242c623aaa618c70c266.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346255.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/52c/d1d1a811935c2533bd66bb167fbf852c/d1d1a811935c2533bd66bb167fbf852c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346259.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/581/8a845be8899b80641d35e00cf0e96581/8a845be8899b80641d35e00cf0e96581.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346263.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/611/0a27c85eb68539b05fd5f878e51cf611/0a27c85eb68539b05fd5f878e51cf611.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346267.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/6cd/804bd840675e01600a916e77c7bb06cd/804bd840675e01600a916e77c7bb06cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346271.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/2a8/943e567eda81a60d82b9b99d180c12a8/943e567eda81a60d82b9b99d180c12a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346275.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/c33/323fbd3d1cc2d27bef21530ab3ab2c33/323fbd3d1cc2d27bef21530ab3ab2c33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346279.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/396/7272307e65b5d3207e58e03cf201f396/7272307e65b5d3207e58e03cf201f396.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346283.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/190/8a8bbfb2bf37e971728251fce764d190/8a8bbfb2bf37e971728251fce764d190.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346287.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/7a9/6c13e9e0a3c72208da9e94d8caf947a9/6c13e9e0a3c72208da9e94d8caf947a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346291.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/f19/095fd57f03a777a5e2018ff0701b4f19/095fd57f03a777a5e2018ff0701b4f19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346295.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/1c6/fe4ed434546ebda9d2613687075b01c6/fe4ed434546ebda9d2613687075b01c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346299.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/546/317e5fb0b11565af8533d73963d00546/317e5fb0b11565af8533d73963d00546.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346303.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/252/5a434ceb0a135b19e3d57d35ceb47252/5a434ceb0a135b19e3d57d35ceb47252.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking ... ok test ERROR: Failed to update sync_store /tmp/tmpmvrxb0_u/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/replicator.py", line 301, in _post_replicate_hook self.sync_store.update_sync_store(broker) File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_replicator.py", line 1226, in update_sync_store raise OSError(1, '1') PermissionError: [Errno 1] 1 proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to get commit status of PUT to /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1750, in await_response self.resp = self.conn.getexpect() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 854, in getexpect expect_status = self._status.get_expect_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 757, in get_expect_status raise expect_status Exception: kaboom! (txn: tx603ff0df90cc4c49982f8-00665c9e35) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get commit status of PUT to /v1/a/c/o: Timeout (Nones) (txn: txccdbfebfa175486ba92cf-00665c9e35) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: tx344b611989cb45949e165-00665c9e35) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: tx18077049ebbd491bad1dc-00665c9e35) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3173, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 6776, in read raise Exception('exception message') Exception: exception message (txn: tx1e4285df6b524fca8e08d-00665c9e35) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: tx255bc31d521c449b80e80-00665c9e35) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.13:1013/sdn (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: txf239c216bc9149a6b9b9d-00665c9e35) proxy-server ERROR: Not enough object servers ack'ed (got 0) (txn: txf239c216bc9149a6b9b9d-00665c9e35) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers ... ok proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET https://cluster.domain/bucket/key HTTP/1.0" 200 106 0.000105 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET http://cluster.domain/v1/acct/cont/obj HTTP/1.0" 200 112 0.000061 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET ftp://cluster.domain/bucket/key HTTP/1.0" 200 126 0.000074 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET https://cluster.domain HTTP/1.0" 200 117 0.000077 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET http:omg//wtf/bbq HTTP/1.0" 200 112 0.000061 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: txba3b211ce2104efd979bf-00665c9e44) proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) test ERROR: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD') (txn: test-trans-id) proxy INFO: wsgi exiting (txn: test-trans-id) proxy INFO: (3543396) wsgi exited, is_accepting=True (txn: test-trans-id) proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: test-trans-id) proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "PUT /path HTTP/1.0" 200 105 0.000121 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET ///some-leading-slashes HTTP/1.0" 200 118 0.000059 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET /ohÿboy%what$now%E2%80%bd HTTP/1.0" 200 120 0.000065 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET /some%fFpath%D8%AA HTTP/1.0" 200 113 0.000056 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET / HTTP/1.1" 200 120 0.000085 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 44\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET /%FF HTTP/1.1" 200 123 0.000073 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 47\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "/%FF", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET /ÿ HTTP/1.1" 200 126 0.000069 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 50\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "/\\u00ff", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "PUT /Here%20Is%20A%20SnowMan:☃ HTTP/1.0" 200 181 0.000091 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 86\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\nConnection: close\r\n\r\n{"RAW_PATH_INFO": "/Here%20Is%20A%20SnowMan:\\u00e2\\u0098\\u0083", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "POST /?and%20it=does+nothing+to+params&PALMTREE=ð%9fŒ%b4 HTTP/1.1" 200 178 0.000073 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 101\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": "and%20it=does+nothing+to+params&PALMTREE=\\u00f0%9f\\u008c%b4"}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET // HTTP/1.1" 200 121 0.000078 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 45\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "//", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET //bar HTTP/1.1" 200 124 0.000083 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 48\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "//bar", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET //////baz HTTP/1.1" 200 128 0.000071 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 52\r\nDate: Sun, 02 Jun 2024 16:31:00 GMT\r\n\r\n{"RAW_PATH_INFO": "//////baz", "QUERY_STRING": null}' proxy INFO: (3543396) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3543396) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:31:00] "GET /someurl HTTP/1.0" 200 102 0.000057 proxy INFO: wsgi exiting proxy INFO: (3543396) wsgi exited, is_accepting=True test NOTICE: Child 3543396 exiting normally test NOTICE: Started child 88 from parent 3543396 test NOTICE: Started child 89 from parent 3543396 test ERROR: Removing dead child 88 from parent 3543396 test NOTICE: Started child 90 from parent 3543396 test NOTICE: Started child 88 from parent 3543396 test NOTICE: Started child 89 from parent 3543396 test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345843.23966', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345843.25631', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission ... ok test INFO: - - - [02/Jun/2024:16:30:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:46 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0085 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:46 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:46 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0139 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0084 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0058 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0088 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543390 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0094 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0127 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0127 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0042 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0042 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0041 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0042 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0155 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:48 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0005 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0004 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543390 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0138 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0087 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0085 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0176 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0154 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0200 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:49 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0156 "-" 3543390 - test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.13:1013/sdn (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server ERROR: Not enough object servers ack'ed (got 3) (txn: tx4ba57758f4ff4f16ad31f-00665c9e39) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: txabe947d0bb524b9b858e9-00665c9e3b) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: txabe947d0bb524b9b858e9-00665c9e3b) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: txabe947d0bb524b9b858e9-00665c9e3b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx5cd5c2968d89400e804f7-00665c9e3b) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx5cd5c2968d89400e804f7-00665c9e3b) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx5cd5c2968d89400e804f7-00665c9e3b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Expect: 100-continue on /a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3038, in _make_putter return MIMEPutter.connect( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2001, in connect raise FooterNotSupported() swift.common.exceptions.FooterNotSupported (txn: tx334a74bb605741e9a39c3-00665c9e3b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345850.89256', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717345850.91382', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag 'a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx4d8c85b04acb436890131-00665c9e49) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx733da0ba026f4dfda5beb-00665c9e49) test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txb8ec3d0adf474f119d8e9-00665c9e3f) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: txb8ec3d0adf474f119d8e9-00665c9e3f) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: txb8ec3d0adf474f119d8e9-00665c9e3f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: tx19cd797d1269454cad43b-00665c9e3f) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx19cd797d1269454cad43b-00665c9e3f) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: tx19cd797d1269454cad43b-00665c9e3f) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx19cd797d1269454cad43b-00665c9e3f) proxy-server ERROR: Object PUT returning 503 for [503, 503, 201, 503, 201, 201, 201, 201, 201, 201, 201, 201, 503, 201] (txn: tx19cd797d1269454cad43b-00665c9e3f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txd0a405704c7346a4841dd-00665c9e42) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txd0a405704c7346a4841dd-00665c9e42) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txd0a405704c7346a4841dd-00665c9e42) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: txd0a405704c7346a4841dd-00665c9e42) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller ... ok test.unit.common.middleware.test_bulk.TestConfig.test_defaults test.unit.common.middleware.test_bulk.TestConfig.test_defaults ... ok test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set ... ok test.unit.common.test_daemon.TestDaemon.test_create test.unit.common.test_daemon.TestDaemon.test_create ... ok test.unit.common.test_daemon.TestDaemon.test_stubs test.unit.common.test_daemon.TestDaemon.test_stubs ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout ... ok test.unit.common.test_internal_client.TestInternalClient.test_container_exists test.unit.common.test_internal_client.TestInternalClient.test_container_exists ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_account test.unit.common.test_internal_client.TestInternalClient.test_create_account ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container test.unit.common.test_internal_client.TestInternalClient.test_create_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_account test.unit.common.test_internal_client.TestInternalClient.test_delete_account ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_container test.unit.common.test_internal_client.TestInternalClient.test_delete_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_object test.unit.common.test_internal_client.TestInternalClient.test_delete_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info test.unit.common.test_internal_client.TestInternalClient.test_get_account_info ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_object test.unit.common.test_internal_client.TestInternalClient.test_get_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_init test.unit.common.test_internal_client.TestInternalClient.test_init ... ok test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_containers test.unit.common.test_internal_client.TestInternalClient.test_iter_containers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items test.unit.common.test_internal_client.TestInternalClient.test_iter_items ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_objects test.unit.common.test_internal_client.TestInternalClient.test_iter_objects ... ok test.unit.common.test_internal_client.TestInternalClient.test_load_from_config test.unit.common.test_internal_client.TestInternalClient.test_load_from_config ... ok test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path test.unit.common.test_internal_client.TestInternalClient.test_make_path ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object test.unit.common.test_internal_client.TestInternalClient.test_upload_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing ... ok test.unit.common.test_manager.TestServer.test_conf_dir test.unit.common.test_manager.TestServer.test_conf_dir ... ok test.unit.common.test_manager.TestServer.test_conf_files test.unit.common.test_manager.TestServer.test_conf_files ... ok test.unit.common.test_manager.TestServer.test_create_server test.unit.common.test_manager.TestServer.test_create_server ... ok test.unit.common.test_manager.TestServer.test_expirer_conf_files test.unit.common.test_manager.TestServer.test_expirer_conf_files ... ok test.unit.common.test_manager.TestServer.test_get_conf_file_name test.unit.common.test_manager.TestServer.test_get_conf_file_name ... ok test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_pid_file_name test.unit.common.test_manager.TestServer.test_get_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_running_pids test.unit.common.test_manager.TestServer.test_get_running_pids ... ok test.unit.common.test_manager.TestServer.test_interact test.unit.common.test_manager.TestServer.test_interact ... ok test.unit.common.test_manager.TestServer.test_iter_pid_files test.unit.common.test_manager.TestServer.test_iter_pid_files ... ok test.unit.common.test_manager.TestServer.test_kill_running_pids test.unit.common.test_manager.TestServer.test_kill_running_pids ... ok test.unit.common.test_manager.TestServer.test_launch test.unit.common.test_manager.TestServer.test_launch ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir test.unit.common.test_manager.TestServer.test_named_conf_dir ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files ... ok test.unit.common.test_manager.TestServer.test_proxy_conf_dir test.unit.common.test_manager.TestServer.test_proxy_conf_dir ... ok test.unit.common.test_manager.TestServer.test_server_equality test.unit.common.test_manager.TestServer.test_server_equality ... ok test.unit.common.test_manager.TestServer.test_server_repr test.unit.common.test_manager.TestServer.test_server_repr ... ok test.unit.common.test_manager.TestServer.test_server_to_string test.unit.common.test_manager.TestServer.test_server_to_string ... ok test.unit.common.test_manager.TestServer.test_signal_pids test.unit.common.test_manager.TestServer.test_signal_pids ... ok test.unit.common.test_manager.TestServer.test_spawn test.unit.common.test_manager.TestServer.test_spawn ... ok test.unit.common.test_manager.TestServer.test_status test.unit.common.test_manager.TestServer.test_status ... ok test.unit.common.test_manager.TestServer.test_stop test.unit.common.test_manager.TestServer.test_stop ... ok test.unit.common.test_manager.TestServer.test_wait test.unit.common.test_manager.TestServer.test_wait ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same ... ok test.unit.common.test_swob.TestTransferEncoding.test_is_chunked test.unit.common.test_swob.TestTransferEncoding.test_is_chunked ... ok test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str test.unit.common.test_wsgi.TestPipelineWrapper.test_str ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container ... ok test INFO: - - - [02/Jun/2024:16:30:49 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0098 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:51 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0082 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:51 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:51 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0113 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0109 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0136 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0061 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0101 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0014 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0130 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0046 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0059 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0066 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0068 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0070 "-" 3543390 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0217 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0093 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0160 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0002 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0240 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0080 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0044 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0129 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3543390 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3543390 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0154 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0099 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:53 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0064 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0124 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0057 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0052 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0032 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0025 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0050 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0036 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0035 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmp_28_x3ac/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0032 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0056 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0030 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0034 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0032 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - deferring rejection downstream test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, deferring rejection based on error: 400 Bad Request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, deferring rejection based on error: 401 Unauthorized test WARNING: More than one part in a single-part response? test ERROR: Problem finding shard upper in '/tmp/tmph5z5yb0o/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmph5z5yb0o/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpmuofblud/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpmuofblud/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3dfb26f641924a719e4e1-00665c9e30) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.7:1007/sdh, error: ERROR Insufficient Storage (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.5:1005/sdf, error: ERROR Insufficient Storage (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.13:1013/sdn, error: ERROR Insufficient Storage (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.10:1010/sdk, error: ERROR Insufficient Storage (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server ERROR: Object returning 503 for [507, 507, 507, 507, 507] (txn: tx9b3d9d84c16f4e55ac811-00665c9e33) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx4233aa5f012840a3a0728-00665c9e33) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf806da3d54b848c7bc434-00665c9e34) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: Container GET returning 503 for [] (txn: txd062bd9e09604e49932d3-00665c9e25) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: tx325761807d4f49bc8a483-00665c9e36) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: tx325761807d4f49bc8a483-00665c9e36) test INFO: Client disconnected on read of '/a/c' (txn: txf40446c14a904e11ba73f-00665c9e36) test INFO: Client disconnected on read of '/a/c' (txn: tx402bbc5c54594f0cabce6-00665c9e36) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner ... ok swift: Client disconnected on read of '/a/\x00c' (txn: txcf4967addf9c452e93785-00665c9e57) swift: ERROR 500 b'Response from 127.0.0.1' Trying to GET /account From Account Server 127.0.0.1:6010/sda (txn: tx0f380def59004d0e97a3e-00665c9e57) swift: ERROR 500 b'Response from 127.0.0.1' Trying to GET /account From Account Server 127.0.0.1:6010/sda (txn: tx0f380def59004d0e97a3e-00665c9e57) test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging ... ok test.unit.container.test_server.TestContainerController.test_DELETE test.unit.container.test_server.TestContainerController.test_DELETE ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race ... ok test INFO: - - - [02/Jun/2024:16:30:55 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:57 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0038 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:57 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0071 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:57 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:30:57 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0032 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3543390 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3543390 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0194 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0060 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0089 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0144 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0104 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0143 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0066 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0193 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0143 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0080 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0125 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0119 "-" 3543390 - test ERROR: ERROR Account update failed with 127.0.0.1:51881/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0197 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0021 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0020 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0020 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0020 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3543390 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0005 "-" 3543390 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3543390 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0062 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0063 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0058 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3543390 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0128 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_account_update test.unit.container.test_server.TestContainerController.test_DELETE_account_update ... ok test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_DELETE_not_found test.unit.container.test_server.TestContainerController.test_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_object test.unit.container.test_server.TestContainerController.test_DELETE_object ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:30:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: User tenant_id with role(s) rdr1,rdr2 has project reader authorizing test-ic DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) fake-swift INFO: - - 02/Jun/2024/16/31/05 GET /cont/obj HTTP/1.0 200 - some_agent - - 19 - - - 0.0004 - - 1717345865.428590059 1717345865.428991318 - fake-swift INFO: - - 02/Jun/2024/16/31/05 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0003 - - 1717345865.431560755 1717345865.431868076 - fake-swift INFO: - - 02/Jun/2024/16/31/05 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717345865.433001995 1717345865.433243990 - fake-swift INFO: - - 02/Jun/2024/16/31/05 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717345865.433954477 1717345865.434187889 - test DEBUG: Before request test DEBUG: Inside of request (txn: foo) test DEBUG: After request fake-swift INFO: - - 02/Jun/2024/16/31/05 DELETE /container HTTP/1.0 409 - some_agent - - 19 - - - 0.0002 - - 1717345865.445868969 1717345865.446067095 - Unable to locate config number 5 for account-server Unable to locate config number 5 for object-server Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 1 Signal object-replicator pid: 11 signal: 15 Signal object-replicator pid: 12 signal: 15 Signal proxy-server pid: 1 signal: 1 output Removing pid file /tmp/tmp833wns_k/var/run/one-server.pid with wrong pid 3 output Removing pid file /tmp/tmp833wns_k/var/run/zero-server.pid with invalid pid output Removing pid file /tmp/tmp833wns_k/var/run/invalid-server.pid with invalid pid Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 3 signal: 15 test ERROR: Problem finding shard upper in '/tmp/tmp1xup0ya7/test_container.db': : LockTimeout (Nones) test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap ... ok swift: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Network unreachable (txn: txa81afb665d404c98a1f60-00665c9e58) swift: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Network unreachable (txn: txa81afb665d404c98a1f60-00665c9e58) swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Network unreachable (txn: txa81afb665d404c98a1f60-00665c9e58) swift: Account HEAD returning 503 for [] (txn: txa81afb665d404c98a1f60-00665c9e58) test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346307.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/7f4/31157dbcbd1e96434e5215a61c6677f4/31157dbcbd1e96434e5215a61c6677f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346311.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/866/b070e8a3a4d7f61003bbc208bc7ed866/b070e8a3a4d7f61003bbc208bc7ed866.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346315.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/e07/1e4ba19e7e558551eb7a80da03062e07/1e4ba19e7e558551eb7a80da03062e07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346319.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/804/07fb83d270b00e323713215e60b07804/07fb83d270b00e323713215e60b07804.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346323.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/dd0/27fe33f6b0099787cac3e5e751f1add0/27fe33f6b0099787cac3e5e751f1add0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346327.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/ac2/56b865284124565fe66911692f970ac2/56b865284124565fe66911692f970ac2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346331.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/e9a/d9e9a91d5aec77b73132a2144b514e9a/d9e9a91d5aec77b73132a2144b514e9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346335.00000-0, db: /tmp/tmpl8c12xer/sda/containers/0/865/157e42795d4a10e4f65bdff41236d865/157e42795d4a10e4f65bdff41236d865.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmpkpiklhr9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345848.00000.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmpkpiklhr9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345848.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345851.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3_1717345851.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345851.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/bf3/9855d6ea299de9955cabecadd7008bf3/9855d6ea299de9955cabecadd7008bf3_1717345851.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345854.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/044/67376554e8bf4c54e79f65848e0c7044/67376554e8bf4c54e79f65848e0c7044_1717345854.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345854.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/044/67376554e8bf4c54e79f65848e0c7044/67376554e8bf4c54e79f65848e0c7044_1717345854.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345857.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b6d/89b6bf5c3c71a1414f9ad7d155e47b6d/89b6bf5c3c71a1414f9ad7d155e47b6d_1717345857.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345857.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b6d/89b6bf5c3c71a1414f9ad7d155e47b6d/89b6bf5c3c71a1414f9ad7d155e47b6d_1717345857.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345860.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/88b/ad0749a6f0ec31d14698e3cf6c59788b/ad0749a6f0ec31d14698e3cf6c59788b_1717345860.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345860.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/88b/ad0749a6f0ec31d14698e3cf6c59788b/ad0749a6f0ec31d14698e3cf6c59788b_1717345860.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345863.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/f35/b7205c0fa015d4356e1a4e6c19164f35/b7205c0fa015d4356e1a4e6c19164f35_1717345863.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345863.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/f35/b7205c0fa015d4356e1a4e6c19164f35/b7205c0fa015d4356e1a4e6c19164f35_1717345863.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345866.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b4e/2db922433fe5a20d94ef41ec93212b4e/2db922433fe5a20d94ef41ec93212b4e_1717345866.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345866.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b4e/2db922433fe5a20d94ef41ec93212b4e/2db922433fe5a20d94ef41ec93212b4e_1717345866.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345869.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/99c/31380e3c31d5872649889a425a41399c/31380e3c31d5872649889a425a41399c_1717345869.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345869.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/99c/31380e3c31d5872649889a425a41399c/31380e3c31d5872649889a425a41399c_1717345869.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345872.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/37f/3d182409525d0d8fec485ee969b4337f/3d182409525d0d8fec485ee969b4337f_1717345872.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345872.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/37f/3d182409525d0d8fec485ee969b4337f/3d182409525d0d8fec485ee969b4337f_1717345872.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345875.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80_1717345875.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345875.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/c80/14defcd1862773716dffaaa379dbfc80/14defcd1862773716dffaaa379dbfc80_1717345875.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345878.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a20/a1f91de4d32020c8e67b5f9e95692a20/a1f91de4d32020c8e67b5f9e95692a20_1717345878.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345878.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a20/a1f91de4d32020c8e67b5f9e95692a20/a1f91de4d32020c8e67b5f9e95692a20_1717345878.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345881.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/cf9/7e546adec99e0fa281dfd403f52d1cf9/7e546adec99e0fa281dfd403f52d1cf9_1717345881.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345881.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/cf9/7e546adec99e0fa281dfd403f52d1cf9/7e546adec99e0fa281dfd403f52d1cf9_1717345881.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345884.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/92a/66e219a1b6e02d4ca1e7cfe50b65192a/66e219a1b6e02d4ca1e7cfe50b65192a_1717345884.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345884.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/92a/66e219a1b6e02d4ca1e7cfe50b65192a/66e219a1b6e02d4ca1e7cfe50b65192a_1717345884.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345887.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e71/1484e75f1d95065b67c23a8eff750e71/1484e75f1d95065b67c23a8eff750e71_1717345887.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345887.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e71/1484e75f1d95065b67c23a8eff750e71/1484e75f1d95065b67c23a8eff750e71_1717345887.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345890.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/55c/39f6658130a353f14e8a6c37ad95c55c/39f6658130a353f14e8a6c37ad95c55c_1717345890.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345890.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/55c/39f6658130a353f14e8a6c37ad95c55c/39f6658130a353f14e8a6c37ad95c55c_1717345890.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345893.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/31a/577112ec93243b6b38c3b6ba1a47331a/577112ec93243b6b38c3b6ba1a47331a_1717345893.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345893.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/31a/577112ec93243b6b38c3b6ba1a47331a/577112ec93243b6b38c3b6ba1a47331a_1717345893.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345896.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a09/6f3079015f6e806747b747aac873ea09/6f3079015f6e806747b747aac873ea09_1717345896.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345896.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a09/6f3079015f6e806747b747aac873ea09/6f3079015f6e806747b747aac873ea09_1717345896.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882_1717345899.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882_1717345899.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345902.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/410/b831e1521e42362aaf60eafad1dce410/b831e1521e42362aaf60eafad1dce410_1717345902.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345902.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/410/b831e1521e42362aaf60eafad1dce410/b831e1521e42362aaf60eafad1dce410_1717345902.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345905.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/0a3/059f7cd8d224c5897d1e7acf09ada0a3/059f7cd8d224c5897d1e7acf09ada0a3_1717345905.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345905.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/0a3/059f7cd8d224c5897d1e7acf09ada0a3/059f7cd8d224c5897d1e7acf09ada0a3_1717345905.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a_1717345908.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a_1717345908.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0_1717345911.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0_1717345911.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421_1717345914.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421_1717345914.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3_1717345917.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3_1717345917.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345920.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/deb/12ecad9a67c2a93914ed09a180d18deb/12ecad9a67c2a93914ed09a180d18deb_1717345920.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345920.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/deb/12ecad9a67c2a93914ed09a180d18deb/12ecad9a67c2a93914ed09a180d18deb_1717345920.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345923.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/0bd/4cb334d4f01ecd9abfdf3bdca8df30bd/4cb334d4f01ecd9abfdf3bdca8df30bd_1717345923.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345923.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/0bd/4cb334d4f01ecd9abfdf3bdca8df30bd/4cb334d4f01ecd9abfdf3bdca8df30bd_1717345923.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345926.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/1ab/bb86b1d8f3e41d16afaa2b1b5ba911ab/bb86b1d8f3e41d16afaa2b1b5ba911ab_1717345926.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345926.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/1ab/bb86b1d8f3e41d16afaa2b1b5ba911ab/bb86b1d8f3e41d16afaa2b1b5ba911ab_1717345926.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345929.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b65/e499cb8202d58e7ad5f94e5e6fe38b65/e499cb8202d58e7ad5f94e5e6fe38b65_1717345929.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345929.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b65/e499cb8202d58e7ad5f94e5e6fe38b65/e499cb8202d58e7ad5f94e5e6fe38b65_1717345929.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345932.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/9d9/189b6935f81e85d6a0a649eed16de9d9/189b6935f81e85d6a0a649eed16de9d9_1717345932.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345932.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/9d9/189b6935f81e85d6a0a649eed16de9d9/189b6935f81e85d6a0a649eed16de9d9_1717345932.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e_1717345935.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345935.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/65e/1b597c870aa71b9077cd6c32eda0f65e/1b597c870aa71b9077cd6c32eda0f65e_1717345935.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345938.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b2d/9cc05f1466be45fbde1e64f1b8fb9b2d/9cc05f1466be45fbde1e64f1b8fb9b2d_1717345938.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345938.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b2d/9cc05f1466be45fbde1e64f1b8fb9b2d/9cc05f1466be45fbde1e64f1b8fb9b2d_1717345938.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345941.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/ad7/9aebbbd44c6ca8b6a9e71960af3cead7/9aebbbd44c6ca8b6a9e71960af3cead7_1717345941.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345941.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/ad7/9aebbbd44c6ca8b6a9e71960af3cead7/9aebbbd44c6ca8b6a9e71960af3cead7_1717345941.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345944.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/7eb/5c7c29ada09e7f3d71c4282e565b37eb/5c7c29ada09e7f3d71c4282e565b37eb_1717345944.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345944.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/7eb/5c7c29ada09e7f3d71c4282e565b37eb/5c7c29ada09e7f3d71c4282e565b37eb_1717345944.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345947.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a17/d7ac936fa77aa2f44d71ebdc60230a17/d7ac936fa77aa2f44d71ebdc60230a17_1717345947.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345947.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a17/d7ac936fa77aa2f44d71ebdc60230a17/d7ac936fa77aa2f44d71ebdc60230a17_1717345947.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345950.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b69/b7866328e9ff41c34dfaa43b12317b69/b7866328e9ff41c34dfaa43b12317b69_1717345950.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345950.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b69/b7866328e9ff41c34dfaa43b12317b69/b7866328e9ff41c34dfaa43b12317b69_1717345950.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345953.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/8a2/4cccedc6e3a351672a95668f6866e8a2/4cccedc6e3a351672a95668f6866e8a2_1717345953.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345953.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/8a2/4cccedc6e3a351672a95668f6866e8a2/4cccedc6e3a351672a95668f6866e8a2_1717345953.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345956.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e14/10e5c1e149b25cf3b6522e6269a4be14/10e5c1e149b25cf3b6522e6269a4be14_1717345956.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345956.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e14/10e5c1e149b25cf3b6522e6269a4be14/10e5c1e149b25cf3b6522e6269a4be14_1717345956.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589_1717345959.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589_1717345959.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345962.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/ed3/7016679a2b3f47cf3c66666d5f6fbed3/7016679a2b3f47cf3c66666d5f6fbed3_1717345962.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345962.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/ed3/7016679a2b3f47cf3c66666d5f6fbed3/7016679a2b3f47cf3c66666d5f6fbed3_1717345962.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345965.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/32c/67c64dda2b60a66458c8b842222eb32c/67c64dda2b60a66458c8b842222eb32c_1717345965.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345965.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/32c/67c64dda2b60a66458c8b842222eb32c/67c64dda2b60a66458c8b842222eb32c_1717345965.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345968.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/37d/24c6b594077d8573ef0cc41ba2e1d37d/24c6b594077d8573ef0cc41ba2e1d37d_1717345968.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345968.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/37d/24c6b594077d8573ef0cc41ba2e1d37d/24c6b594077d8573ef0cc41ba2e1d37d_1717345968.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345971.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01_1717345971.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345971.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/d01/5a83a6a6064583f1a5b15d369dc0dd01/5a83a6a6064583f1a5b15d369dc0dd01_1717345971.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345974.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/469/6cc61909e7a11534cf9facccb2cd6469/6cc61909e7a11534cf9facccb2cd6469_1717345974.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345974.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/469/6cc61909e7a11534cf9facccb2cd6469/6cc61909e7a11534cf9facccb2cd6469_1717345974.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345977.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/4db/9937054b4ec061f8125fd60bf4dbf4db/9937054b4ec061f8125fd60bf4dbf4db_1717345977.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345977.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/4db/9937054b4ec061f8125fd60bf4dbf4db/9937054b4ec061f8125fd60bf4dbf4db_1717345977.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345980.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/ed5/2df33c1469286e6799495fb91d7a9ed5/2df33c1469286e6799495fb91d7a9ed5_1717345980.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345980.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/ed5/2df33c1469286e6799495fb91d7a9ed5/2df33c1469286e6799495fb91d7a9ed5_1717345980.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345983.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/2fe/14e8e0360f977223aacbcdb1876582fe/14e8e0360f977223aacbcdb1876582fe_1717345983.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345983.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/2fe/14e8e0360f977223aacbcdb1876582fe/14e8e0360f977223aacbcdb1876582fe_1717345983.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345986.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a86/cee9b0a96eb2f4124cf61e01ea590a86/cee9b0a96eb2f4124cf61e01ea590a86_1717345986.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345986.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/a86/cee9b0a96eb2f4124cf61e01ea590a86/cee9b0a96eb2f4124cf61e01ea590a86_1717345986.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345989.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/975/40c65bb6fef1f9af0ce0f9a889d51975/40c65bb6fef1f9af0ce0f9a889d51975_1717345989.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345989.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/975/40c65bb6fef1f9af0ce0f9a889d51975/40c65bb6fef1f9af0ce0f9a889d51975_1717345989.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345992.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/05d/bb3d0420c8cf0ea39b3e649287b4305d/bb3d0420c8cf0ea39b3e649287b4305d_1717345992.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345992.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/05d/bb3d0420c8cf0ea39b3e649287b4305d/bb3d0420c8cf0ea39b3e649287b4305d_1717345992.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345995.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35_1717345995.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345995.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/c35/dcbd05c8c7ac0d6d635d186c333e9c35/dcbd05c8c7ac0d6d635d186c333e9c35_1717345995.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345998.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/513/41db166df269eddb0664009737651513/41db166df269eddb0664009737651513_1717345998.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345998.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/513/41db166df269eddb0664009737651513/41db166df269eddb0664009737651513_1717345998.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346001.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/837/ba631526b18a59321d0855f186ee7837/ba631526b18a59321d0855f186ee7837_1717346001.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346001.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/837/ba631526b18a59321d0855f186ee7837/ba631526b18a59321d0855f186ee7837_1717346001.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346004.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e22/5dae92396724bfb4eee10f10de10de22/5dae92396724bfb4eee10f10de10de22_1717346004.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346004.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e22/5dae92396724bfb4eee10f10de10de22/5dae92396724bfb4eee10f10de10de22_1717346004.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346007.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/d9a/cd426bceff16dee8a0035688912e7d9a/cd426bceff16dee8a0035688912e7d9a_1717346007.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346007.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/d9a/cd426bceff16dee8a0035688912e7d9a/cd426bceff16dee8a0035688912e7d9a_1717346007.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346010.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/fc6/1398ecba34e0d89c184c83f06fa4cfc6/1398ecba34e0d89c184c83f06fa4cfc6_1717346010.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346010.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/fc6/1398ecba34e0d89c184c83f06fa4cfc6/1398ecba34e0d89c184c83f06fa4cfc6_1717346010.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346013.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/f2f/20c9689a0a9aea9e8e025b18b58fbf2f/20c9689a0a9aea9e8e025b18b58fbf2f_1717346013.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346013.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/f2f/20c9689a0a9aea9e8e025b18b58fbf2f/20c9689a0a9aea9e8e025b18b58fbf2f_1717346013.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346016.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/300/877a2ac6f2b81e47fd796b1879668300/877a2ac6f2b81e47fd796b1879668300_1717346016.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346016.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/300/877a2ac6f2b81e47fd796b1879668300/877a2ac6f2b81e47fd796b1879668300_1717346016.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e_1717346019.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e_1717346019.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346022.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/8c3/487d1c4493d129889b37ec9e80ed18c3/487d1c4493d129889b37ec9e80ed18c3_1717346022.00000.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info ... ok test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx914001f8d11e4ce4a1287-00665c9e36) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx914001f8d11e4ce4a1287-00665c9e36) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx914001f8d11e4ce4a1287-00665c9e36) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx914001f8d11e4ce4a1287-00665c9e36) test ERROR: Container GET returning 503 for [] (txn: tx914001f8d11e4ce4a1287-00665c9e36) test INFO: Client disconnected on read of '/a/c' (txn: tx914001f8d11e4ce4a1287-00665c9e36) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: Container GET returning 503 for [] (txn: txb5c8e3bdd3684e9d87f8e-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: Container GET returning 503 for [] (txn: txd2744cc0e69d49eaa3137-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx9435514e681f47088cae6-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx9435514e681f47088cae6-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx9435514e681f47088cae6-00665c9e3b) test ERROR: Container GET returning 503 for [] (txn: tx9435514e681f47088cae6-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4c8d550dc8df421aab376-00665c9e3b) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx4c8d550dc8df421aab376-00665c9e3b) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txf4d2e774c5fc4489b3ed1-00665c9e3b) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txf4d2e774c5fc4489b3ed1-00665c9e3b) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: txf4d2e774c5fc4489b3ed1-00665c9e3b) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: txf4d2e774c5fc4489b3ed1-00665c9e3b) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx0b4dee9efc9e4d819129a-00665c9e3b) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txf6f77d5874e74e6897507-00665c9e3b) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txf6f77d5874e74e6897507-00665c9e3b) test INFO: Client disconnected on read of '/a/c' (txn: txf6f77d5874e74e6897507-00665c9e3b) test ERROR: Container GET returning 503 for [] (txn: tx9a2d92282249431dbd36d-00665c9e3b) test INFO: Client disconnected on read of '/a/c' (txn: tx9a2d92282249431dbd36d-00665c9e3b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txd326d1bc25f247a292d90-00665c9e3d) test INFO: Client disconnected on read of '/a/c' (txn: txd326d1bc25f247a292d90-00665c9e3d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing ... ok swift: ERROR Unhandled exception in request: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/server.py", line 537, in handle_request#012 controller, path_parts = self.get_controller(req)#012 ^^^^^^^^^^^^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 664, in get_controller#012 raise Exception('this shouldn\'t be caught')#012Exception: this shouldn't be caught swift: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: txe56120c64da546698a627-00665c9e59) swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: txe56120c64da546698a627-00665c9e59) swift: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: txe56120c64da546698a627-00665c9e59) swift: Account HEAD returning 503 for [] (txn: txe56120c64da546698a627-00665c9e59) swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1.0/a: Network unreachable (txn: tx40df923246e44361becc7-00665c9e59) swift: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1.0/a: Network unreachable (txn: tx40df923246e44361becc7-00665c9e59) swift: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1.0/a: Network unreachable (txn: tx40df923246e44361becc7-00665c9e59) swift: Account HEAD returning 503 for [] (txn: tx40df923246e44361becc7-00665c9e59) test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps ... ok proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx523d7d9664704a6385967-00665c9e38) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default ... ok test INFO: - - - [02/Jun/2024:16:31:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0107 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0051 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543390 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0151 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0120 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0021 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0021 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0023 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0158 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0132 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0059 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0059 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0060 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:03 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0060 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0059 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0092 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0009 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0011 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0197 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0157 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0017 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container ... ok test ERROR: Failed to unlink '/tmp/tmp47amnxgt/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp47amnxgt/containers/part/suffix/hash/container_1717345878.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp47amnxgt/containers/part/suffix/hash/container_1717345878.00000.db': ['/tmp/tmp47amnxgt/containers/part/suffix/hash/container.db', '/tmp/tmp47amnxgt/containers/part/suffix/hash/container_1717345879.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp47amnxgt/containers/part/suffix/hash/container_1717345879.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0138 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0079 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0101 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0103 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0213 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0192 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0148 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0211 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0103 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0071 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0101 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3543406 0 test ERROR: ERROR Account update failed with 127.0.0.1:56293/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0162 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0206 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0089 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0044 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0145 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0348 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0111 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0144 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0048 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0019 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0024 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0050 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0018 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0018 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0038 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0020 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0107 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0024 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0019 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0177 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0013 "-" 3543406 - test DEBUG: Unlinked retiring db '/tmp/tmpaxz2bdgg/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0137 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0027 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestContainerController.test_GET_auto_record_type test.unit.container.test_server.TestContainerController.test_GET_auto_record_type ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter test.unit.container.test_server.TestContainerController.test_GET_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: txa2ebe01a0d2b4abf848c5-00665c9e3d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx988547f32568433bb0ce9-00665c9e43) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: tx988547f32568433bb0ce9-00665c9e43) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx0e89d40be2b440c68eb4e-00665c9e43) test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body ... ok test.unit.proxy.test_mem_server.TestController.test_account_info_200 test.unit.proxy.test_mem_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_404 test.unit.proxy.test_mem_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_container_count test.unit.proxy.test_mem_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_account test.unit.proxy.test_mem_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_200 test.unit.proxy.test_mem_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_404 test.unit.proxy.test_mem_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_make_requests test.unit.proxy.test_mem_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_transfer_headers test.unit.proxy.test_mem_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny ... ok swift: Client disconnected on read of '/a' (txn: tx8272c3dc3d2e44d0b408c-00665c9e5b) swift: Client disconnected on read of '/a' (txn: tx8272c3dc3d2e44d0b408c-00665c9e5b) test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx41533289c5194311b78c8-00665c9e41) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txa188778089d5422d91fdb-00665c9e46) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpt388o81l/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpt388o81l/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpt388o81l/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpt388o81l/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0163 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0127 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0116 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0131 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx7dd64b1205ce47f99dfba-00665c9e48" "proxy-server 3543400" 0.0028 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:33415/sda1/1/a/c" "tx7dd64b1205ce47f99dfba-00665c9e48" "container-server 3543400" 0.0025 "-" 3543400 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:58491/sdb1/1/a/c" "tx7dd64b1205ce47f99dfba-00665c9e48" "container-server 3543400" 0.0019 "-" 3543400 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx7dd64b1205ce47f99dfba-00665c9e48" "proxy-server 3543400" 0.0233 "-" 3543400 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx7dd64b1205ce47f99dfba-00665c9e48" "proxy-server 3543400" 0.0408 "-" 3543400 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/04 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx7dd64b1205ce47f99dfba-00665c9e48 - 0.0550 - - 1717345864.267323494 1717345864.322274923 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txe70b1f80b5504247ac7ea-00665c9e48" "proxy-server 3543400" 0.0022 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:58491/sdb1/1/a1/c1" "txe70b1f80b5504247ac7ea-00665c9e48" "container-server 3543400" 0.0314 "-" 3543400 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:33415/sda1/1/a1/c1" "txe70b1f80b5504247ac7ea-00665c9e48" "container-server 3543400" 0.0034 "-" 3543400 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe70b1f80b5504247ac7ea-00665c9e48" "proxy-server 3543400" 0.0557 "-" 3543400 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe70b1f80b5504247ac7ea-00665c9e48" "proxy-server 3543400" 0.0835 "-" 3543400 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/04 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txe70b1f80b5504247ac7ea-00665c9e48 - 0.0973 - - 1717345864.324970007 1717345864.422235727 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9f6fc65d18ea476c8493f-00665c9e48" "proxy-server 3543400" 0.0079 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:58491/sdb1/2/a/c1" "tx9f6fc65d18ea476c8493f-00665c9e48" "container-server 3543400" 0.0019 "-" 3543400 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:33415/sda1/2/a/c1" "tx9f6fc65d18ea476c8493f-00665c9e48" "container-server 3543400" 0.0076 "-" 3543400 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx9f6fc65d18ea476c8493f-00665c9e48" "proxy-server 3543400" 0.0402 "-" 3543400 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx9f6fc65d18ea476c8493f-00665c9e48" "proxy-server 3543400" 0.0283 "-" 3543400 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/04 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx9f6fc65d18ea476c8493f-00665c9e48 - 0.0607 - - 1717345864.425122738 1717345864.485835314 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx225ea8f135e249848bc2e-00665c9e48" "proxy-server 3543400" 0.0072 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:58491/sdb1/1/a/c2" "tx225ea8f135e249848bc2e-00665c9e48" "container-server 3543400" 0.0021 "-" 3543400 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:33415/sda1/1/a/c2" "tx225ea8f135e249848bc2e-00665c9e48" "container-server 3543400" 0.0092 "-" 3543400 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx225ea8f135e249848bc2e-00665c9e48" "proxy-server 3543400" 0.0302 "-" 3543400 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx225ea8f135e249848bc2e-00665c9e48" "proxy-server 3543400" 0.0477 "-" 3543400 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/04 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx225ea8f135e249848bc2e-00665c9e48 - 0.0665 - - 1717345864.488165617 1717345864.554630518 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items ... ok test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0050 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0160 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0108 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0195 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0069 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0041 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0040 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0074 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0035 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0048 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0042 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0042 "-" 3543406 0 test DEBUG: Unlinked retiring db '/tmp/tmpul_xxgtd/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0034 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0042 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0037 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0173 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0011 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0117 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0089 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0033 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestContainerController.test_GET_invalid_accept test.unit.container.test_server.TestContainerController.test_GET_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_GET_json test.unit.container.test_server.TestContainerController.test_GET_json ... ok test.unit.container.test_server.TestContainerController.test_GET_json_last_modified test.unit.container.test_server.TestContainerController.test_GET_json_last_modified ... ok test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_limit test.unit.container.test_server.TestContainerController.test_GET_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_false test.unit.container.test_server.TestContainerController.test_GET_log_requests_false ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_true test.unit.container.test_server.TestContainerController.test_GET_log_requests_true ... ok test.unit.container.test_server.TestContainerController.test_GET_marker test.unit.container.test_server.TestContainerController.test_GET_marker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode ... ok test INFO: - - - [02/Jun/2024:16:31:04 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0107 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0022 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0044 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmpv8qcd7q3/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0053 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0061 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0014 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0053 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:05 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0015 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0043 "-" 3543390 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0246 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0039 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0006 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0004 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0352 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0087 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0074 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0165 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0126 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0259 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx87fc52c328f64b12afe67-00665c9e45) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes ... ok test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0116 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0121 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0130 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0012 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0053 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0019 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0037 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0106 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0090 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0100 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0121 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0180 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0112 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0159 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0125 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0070 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0034 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0032 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346022.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/8c3/487d1c4493d129889b37ec9e80ed18c3/487d1c4493d129889b37ec9e80ed18c3_1717346022.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346025.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b4a/e62d484238b79671e549c4f98df15b4a/e62d484238b79671e549c4f98df15b4a_1717346025.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346025.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/b4a/e62d484238b79671e549c4f98df15b4a/e62d484238b79671e549c4f98df15b4a_1717346025.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346028.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/187/2c378564d50d40aab80211291c8c3187/2c378564d50d40aab80211291c8c3187_1717346028.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346028.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/187/2c378564d50d40aab80211291c8c3187/2c378564d50d40aab80211291c8c3187_1717346028.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893_1717346031.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346031.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/893/25fce72fe739e5ae0b3f02191753d893/25fce72fe739e5ae0b3f02191753d893_1717346031.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346034.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e6b/1e17050f206eb050635baa94217f8e6b/1e17050f206eb050635baa94217f8e6b_1717346034.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346034.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/e6b/1e17050f206eb050635baa94217f8e6b/1e17050f206eb050635baa94217f8e6b_1717346034.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346037.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/7b3/02d67eb28b0e3c8c21bfd1f69114c7b3/02d67eb28b0e3c8c21bfd1f69114c7b3_1717346037.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346037.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/7b3/02d67eb28b0e3c8c21bfd1f69114c7b3/02d67eb28b0e3c8c21bfd1f69114c7b3_1717346037.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346040.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/254/58275ab922b2d9c974fed445488ff254/58275ab922b2d9c974fed445488ff254_1717346040.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346040.00000-0, db: /tmp/tmpwvllj_c_/sda/containers/0/254/58275ab922b2d9c974fed445488ff254/58275ab922b2d9c974fed445488ff254_1717346040.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345865.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4b4/46f19aaff56ae4ac57fce63d6eed04b4/46f19aaff56ae4ac57fce63d6eed04b4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345865.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4b4/46f19aaff56ae4ac57fce63d6eed04b4/46f19aaff56ae4ac57fce63d6eed04b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345869.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/99c/31380e3c31d5872649889a425a41399c/31380e3c31d5872649889a425a41399c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345869.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/99c/31380e3c31d5872649889a425a41399c/31380e3c31d5872649889a425a41399c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345873.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/5cf/0b6d9c6fe78cb4b21157f70ed26715cf/0b6d9c6fe78cb4b21157f70ed26715cf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345873.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/5cf/0b6d9c6fe78cb4b21157f70ed26715cf/0b6d9c6fe78cb4b21157f70ed26715cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345877.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/529/36551784d07f4a4aef6f7e9c6d938529/36551784d07f4a4aef6f7e9c6d938529.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345877.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/529/36551784d07f4a4aef6f7e9c6d938529/36551784d07f4a4aef6f7e9c6d938529.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345881.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/cf9/7e546adec99e0fa281dfd403f52d1cf9/7e546adec99e0fa281dfd403f52d1cf9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345881.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/cf9/7e546adec99e0fa281dfd403f52d1cf9/7e546adec99e0fa281dfd403f52d1cf9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345885.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/32d/8d4130bf153f46c4c156851c7528532d/8d4130bf153f46c4c156851c7528532d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345885.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/32d/8d4130bf153f46c4c156851c7528532d/8d4130bf153f46c4c156851c7528532d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345889.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/9cc/625074f4da62535c56599869829ba9cc/625074f4da62535c56599869829ba9cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345889.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/9cc/625074f4da62535c56599869829ba9cc/625074f4da62535c56599869829ba9cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345893.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/31a/577112ec93243b6b38c3b6ba1a47331a/577112ec93243b6b38c3b6ba1a47331a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345893.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/31a/577112ec93243b6b38c3b6ba1a47331a/577112ec93243b6b38c3b6ba1a47331a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345897.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/67e/92d45fe72433852df712d40f4947567e/92d45fe72433852df712d40f4947567e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345897.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/67e/92d45fe72433852df712d40f4947567e/92d45fe72433852df712d40f4947567e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345901.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/50f/2dfda13b8b2c063211a229b170b7950f/2dfda13b8b2c063211a229b170b7950f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345901.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/50f/2dfda13b8b2c063211a229b170b7950f/2dfda13b8b2c063211a229b170b7950f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345905.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/0a3/059f7cd8d224c5897d1e7acf09ada0a3/059f7cd8d224c5897d1e7acf09ada0a3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345905.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/0a3/059f7cd8d224c5897d1e7acf09ada0a3/059f7cd8d224c5897d1e7acf09ada0a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345909.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/417/b9eb057ae1e3b48e96e613a12ad94417/b9eb057ae1e3b48e96e613a12ad94417.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345909.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/417/b9eb057ae1e3b48e96e613a12ad94417/b9eb057ae1e3b48e96e613a12ad94417.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345913.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/27c/5bd94edf03635d01a64c72b6003bd27c/5bd94edf03635d01a64c72b6003bd27c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345913.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/27c/5bd94edf03635d01a64c72b6003bd27c/5bd94edf03635d01a64c72b6003bd27c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345921.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/977/28186d1f70ef76a28bc8ace056955977/28186d1f70ef76a28bc8ace056955977.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345921.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/977/28186d1f70ef76a28bc8ace056955977/28186d1f70ef76a28bc8ace056955977.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345929.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/b65/e499cb8202d58e7ad5f94e5e6fe38b65/e499cb8202d58e7ad5f94e5e6fe38b65.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345929.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/b65/e499cb8202d58e7ad5f94e5e6fe38b65/e499cb8202d58e7ad5f94e5e6fe38b65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345933.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/af4/e076bdc1cb8baba7fcbeb39ffa3ddaf4/e076bdc1cb8baba7fcbeb39ffa3ddaf4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345933.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/af4/e076bdc1cb8baba7fcbeb39ffa3ddaf4/e076bdc1cb8baba7fcbeb39ffa3ddaf4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345937.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/8c8/620d98adac4bdd426cc1adf5940c98c8/620d98adac4bdd426cc1adf5940c98c8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345937.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/8c8/620d98adac4bdd426cc1adf5940c98c8/620d98adac4bdd426cc1adf5940c98c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345941.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/ad7/9aebbbd44c6ca8b6a9e71960af3cead7/9aebbbd44c6ca8b6a9e71960af3cead7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345941.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/ad7/9aebbbd44c6ca8b6a9e71960af3cead7/9aebbbd44c6ca8b6a9e71960af3cead7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345945.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/bb3/0e13ba1894fd7686880ce2e061079bb3/0e13ba1894fd7686880ce2e061079bb3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345945.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/bb3/0e13ba1894fd7686880ce2e061079bb3/0e13ba1894fd7686880ce2e061079bb3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345953.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/8a2/4cccedc6e3a351672a95668f6866e8a2/4cccedc6e3a351672a95668f6866e8a2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345953.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/8a2/4cccedc6e3a351672a95668f6866e8a2/4cccedc6e3a351672a95668f6866e8a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345957.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/ac4/2ffb5bec4d928088040ebf4b2ecefac4/2ffb5bec4d928088040ebf4b2ecefac4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345957.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/ac4/2ffb5bec4d928088040ebf4b2ecefac4/2ffb5bec4d928088040ebf4b2ecefac4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345965.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/32c/67c64dda2b60a66458c8b842222eb32c/67c64dda2b60a66458c8b842222eb32c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345965.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/32c/67c64dda2b60a66458c8b842222eb32c/67c64dda2b60a66458c8b842222eb32c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345969.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345969.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345973.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/6cc/e5f758aec496ce42916476fb4ea9e6cc/e5f758aec496ce42916476fb4ea9e6cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345973.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/6cc/e5f758aec496ce42916476fb4ea9e6cc/e5f758aec496ce42916476fb4ea9e6cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345977.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4db/9937054b4ec061f8125fd60bf4dbf4db/9937054b4ec061f8125fd60bf4dbf4db.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345977.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4db/9937054b4ec061f8125fd60bf4dbf4db/9937054b4ec061f8125fd60bf4dbf4db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345981.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4a1/f6a147dfdf96222d3d20fd37815534a1/f6a147dfdf96222d3d20fd37815534a1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345981.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4a1/f6a147dfdf96222d3d20fd37815534a1/f6a147dfdf96222d3d20fd37815534a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345989.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/975/40c65bb6fef1f9af0ce0f9a889d51975/40c65bb6fef1f9af0ce0f9a889d51975.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345989.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/975/40c65bb6fef1f9af0ce0f9a889d51975/40c65bb6fef1f9af0ce0f9a889d51975.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345993.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/720/c672eb5c4fb157ee8b7e6932e26a5720/c672eb5c4fb157ee8b7e6932e26a5720.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345993.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/720/c672eb5c4fb157ee8b7e6932e26a5720/c672eb5c4fb157ee8b7e6932e26a5720.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345997.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/a53/231ffb44cd41be0edc6ed40d4a1dfa53/231ffb44cd41be0edc6ed40d4a1dfa53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345997.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/a53/231ffb44cd41be0edc6ed40d4a1dfa53/231ffb44cd41be0edc6ed40d4a1dfa53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346001.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/837/ba631526b18a59321d0855f186ee7837/ba631526b18a59321d0855f186ee7837.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346001.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/837/ba631526b18a59321d0855f186ee7837/ba631526b18a59321d0855f186ee7837.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346005.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/592/c0d44b0dbbc428d2de62edfbe32ed592/c0d44b0dbbc428d2de62edfbe32ed592.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346005.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/592/c0d44b0dbbc428d2de62edfbe32ed592/c0d44b0dbbc428d2de62edfbe32ed592.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346013.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/f2f/20c9689a0a9aea9e8e025b18b58fbf2f/20c9689a0a9aea9e8e025b18b58fbf2f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346013.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/f2f/20c9689a0a9aea9e8e025b18b58fbf2f/20c9689a0a9aea9e8e025b18b58fbf2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346017.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/058/707f65067a2ad6686f85f9edfd2f8058/707f65067a2ad6686f85f9edfd2f8058.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346017.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/058/707f65067a2ad6686f85f9edfd2f8058/707f65067a2ad6686f85f9edfd2f8058.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346025.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/b4a/e62d484238b79671e549c4f98df15b4a/e62d484238b79671e549c4f98df15b4a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346025.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/b4a/e62d484238b79671e549c4f98df15b4a/e62d484238b79671e549c4f98df15b4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346029.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/c51/014702cf3e6364f1cc17bf7f2d3dac51/014702cf3e6364f1cc17bf7f2d3dac51.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346029.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/c51/014702cf3e6364f1cc17bf7f2d3dac51/014702cf3e6364f1cc17bf7f2d3dac51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346033.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/dd1/84f12721bef5ffa94e25c6863a1badd1/84f12721bef5ffa94e25c6863a1badd1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346033.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/dd1/84f12721bef5ffa94e25c6863a1badd1/84f12721bef5ffa94e25c6863a1badd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346037.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/7b3/02d67eb28b0e3c8c21bfd1f69114c7b3/02d67eb28b0e3c8c21bfd1f69114c7b3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346037.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/7b3/02d67eb28b0e3c8c21bfd1f69114c7b3/02d67eb28b0e3c8c21bfd1f69114c7b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346041.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/685/4ffcdd81d57287b719521b14ca851685/4ffcdd81d57287b719521b14ca851685.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346041.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/685/4ffcdd81d57287b719521b14ca851685/4ffcdd81d57287b719521b14ca851685.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346045.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/094/970bcee06085d9e70038e2bd329fd094/970bcee06085d9e70038e2bd329fd094.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346045.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/094/970bcee06085d9e70038e2bd329fd094/970bcee06085d9e70038e2bd329fd094.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346049.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/fa1/4a860ba03606b566205549d0529fafa1/4a860ba03606b566205549d0529fafa1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346049.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/fa1/4a860ba03606b566205549d0529fafa1/4a860ba03606b566205549d0529fafa1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346053.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/33c/4e2a27e7a23ea2db68b94267153e833c/4e2a27e7a23ea2db68b94267153e833c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346053.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/33c/4e2a27e7a23ea2db68b94267153e833c/4e2a27e7a23ea2db68b94267153e833c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346057.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/414/286f8483218e828516df1fb016ec3414/286f8483218e828516df1fb016ec3414.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346057.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/414/286f8483218e828516df1fb016ec3414/286f8483218e828516df1fb016ec3414.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346061.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/2e1/b79738364dc0d9b4ab38db5a3fdc52e1/b79738364dc0d9b4ab38db5a3fdc52e1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346061.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/2e1/b79738364dc0d9b4ab38db5a3fdc52e1/b79738364dc0d9b4ab38db5a3fdc52e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346065.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/23f/f58cebc8e9f80e0c3d59f7a22a4af23f/f58cebc8e9f80e0c3d59f7a22a4af23f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346065.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/23f/f58cebc8e9f80e0c3d59f7a22a4af23f/f58cebc8e9f80e0c3d59f7a22a4af23f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346069.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/6f3/d75465b42e5dc3b12c47edd8623656f3/d75465b42e5dc3b12c47edd8623656f3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346069.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/6f3/d75465b42e5dc3b12c47edd8623656f3/d75465b42e5dc3b12c47edd8623656f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346073.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/6ea/69993d4b9e11fe7d6e26b0d1b999a6ea/69993d4b9e11fe7d6e26b0d1b999a6ea.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346073.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/6ea/69993d4b9e11fe7d6e26b0d1b999a6ea/69993d4b9e11fe7d6e26b0d1b999a6ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346077.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/7d6/3389b3ee553026d6ca319b0572cdb7d6/3389b3ee553026d6ca319b0572cdb7d6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346077.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/7d6/3389b3ee553026d6ca319b0572cdb7d6/3389b3ee553026d6ca319b0572cdb7d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346081.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/39f/b47b6b9fa58d2c6f4946f170f534739f/b47b6b9fa58d2c6f4946f170f534739f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346081.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/39f/b47b6b9fa58d2c6f4946f170f534739f/b47b6b9fa58d2c6f4946f170f534739f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346085.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/5bd/27af413e145c29aaaedf36d3109815bd/27af413e145c29aaaedf36d3109815bd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346085.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/5bd/27af413e145c29aaaedf36d3109815bd/27af413e145c29aaaedf36d3109815bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346089.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4d4/246c30bc5935967d840225fd570ca4d4/246c30bc5935967d840225fd570ca4d4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346089.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4d4/246c30bc5935967d840225fd570ca4d4/246c30bc5935967d840225fd570ca4d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346093.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/12b/b657dfb4b094b567063e34aad7b4a12b/b657dfb4b094b567063e34aad7b4a12b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346093.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/12b/b657dfb4b094b567063e34aad7b4a12b/b657dfb4b094b567063e34aad7b4a12b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346097.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4dd/d6ee043fcbec6b053b64813e685564dd/d6ee043fcbec6b053b64813e685564dd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346097.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/4dd/d6ee043fcbec6b053b64813e685564dd/d6ee043fcbec6b053b64813e685564dd.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded ... ok test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test INFO: - - - [02/Jun/2024:16:31:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0247 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0005 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:10 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3543390 2 test DEBUG: Unlinked retiring db '/tmp/tmp8rg4r_lm/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0145 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0135 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0151 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0022 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0138 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:11 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0006 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:11 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:11 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0005 "-" 3543390 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:31:11 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0063 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:11 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0005 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0008 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0039 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0007 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0198 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0040 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0171 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0167 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0139 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved ... ok proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfda87b9f556f4441989b5-00665c9e47) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort ... ok test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0031 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0068 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0097 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0116 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0117 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0165 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:21 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0015 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0214 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0171 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0186 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0137 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0091 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0020 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0020 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0116 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0171 "-" 3543406 0 test DEBUG: Unlinked retiring db '/tmp/tmpebfbtzjr/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.1327 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0033 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0138 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0161 "-" 3543406 0 test DEBUG: Unlinked retiring db '/tmp/tmpm8y160oi/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0059 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0133 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0152 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0129 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0164 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0116 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0027 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_GET_non_ascii test.unit.container.test_server.TestContainerController.test_GET_non_ascii ... ok test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestContainerController.test_GET_over_limit test.unit.container.test_server.TestContainerController.test_GET_over_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_path test.unit.container.test_server.TestContainerController.test_GET_path ... ok test.unit.container.test_server.TestContainerController.test_GET_plain test.unit.container.test_server.TestContainerController.test_GET_plain ... ok test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txf81f66bce6f14a109e318-00665c9e53) test INFO: Client disconnected on read of '/a/c' (txn: txf81f66bce6f14a109e318-00665c9e53) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: tx23671284f4654979bd9cf-00665c9e55) test INFO: Client disconnected on read of '/a/c' (txn: tx23671284f4654979bd9cf-00665c9e55) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: tx6702a359434a4e22bbd32-00665c9e58) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: tx6702a359434a4e22bbd32-00665c9e58) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: tx6702a359434a4e22bbd32-00665c9e58) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: tx6702a359434a4e22bbd32-00665c9e58) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version ... ok test INFO: - - - [02/Jun/2024:16:31:11 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0189 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:13 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0023 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0104 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0078 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0129 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0123 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0082 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0086 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543390 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0093 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0123 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0109 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0115 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0128 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:14 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0103 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0002 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0019 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0113 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:14 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0107 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0110 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0108 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:15 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0014 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0118 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0112 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0031 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0031 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0122 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0016 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0002 "-" 3543390 nada test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0002 "-" 3543390 999 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_legacy" 201 - "-" "-" "-" 0.0118 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "HEAD /sda1/p/a/c_legacy" 204 - "-" "-" "-" 0.0017 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0159 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "HEAD /sda1/p/a/c_one" 204 - "-" "-" "-" 0.0017 "-" 3543390 1 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0144 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "HEAD /sda1/p/a/c_two" 204 - "-" "-" "-" 0.0024 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0111 "-" 3543390 3 test INFO: - - - [02/Jun/2024:16:31:15 +0000] "HEAD /sda1/p/a/c_three" 204 - "-" "-" "-" 0.0016 "-" 3543390 3 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.00s) (txn: txfeb149c0a0ce4282bda22-00665c9e13) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0118 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0138 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0031 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0177 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0012 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0010 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0005 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0138 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0015 "-" 3543406 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0156 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0130 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0081 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0075 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0035 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0107 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test.unit.container.test_server.TestContainerController.test_GET_prefix test.unit.container.test_server.TestContainerController.test_GET_prefix ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp98ab8pcx/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp98ab8pcx/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp98ab8pcx/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp98ab8pcx/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:21 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0160 "-" 3543400 - test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok test INFO: - - - [02/Jun/2024:16:31:15 +0000] "PUT /sda1/p/a/c_four" 201 - "-" "-" "-" 0.0119 "-" 3543390 4 test INFO: - - - [02/Jun/2024:16:31:16 +0000] "HEAD /sda1/p/a/c_four" 204 - "-" "-" "-" 0.0030 "-" 3543390 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:17 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:17 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0117 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0119 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0035 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0148 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0108 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0061 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0032 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0111 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0118 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0162 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0041 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0133 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0123 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0205 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0128 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0013 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0107 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0130 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:17 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0082 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0118 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0088 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0218 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0032 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0029 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0128 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0052 "-" 3543390 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types ... ok test.unit.container.test_sharder.TestCleavingContext.test_cursor test.unit.container.test_sharder.TestCleavingContext.test_cursor ... ok test.unit.container.test_sharder.TestCleavingContext.test_delete test.unit.container.test_sharder.TestCleavingContext.test_delete ... ok test.unit.container.test_sharder.TestCleavingContext.test_done test.unit.container.test_sharder.TestCleavingContext.test_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_init test.unit.container.test_sharder.TestCleavingContext.test_init ... ok test.unit.container.test_sharder.TestCleavingContext.test_iter test.unit.container.test_sharder.TestCleavingContext.test_iter ... ok test.unit.container.test_sharder.TestCleavingContext.test_load test.unit.container.test_sharder.TestCleavingContext.test_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_all test.unit.container.test_sharder.TestCleavingContext.test_load_all ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_range_done test.unit.container.test_sharder.TestCleavingContext.test_range_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_reset test.unit.container.test_sharder.TestCleavingContext.test_reset ... ok test.unit.container.test_sharder.TestCleavingContext.test_start test.unit.container.test_sharder.TestCleavingContext.test_start ... ok test.unit.container.test_sharder.TestCleavingContext.test_store test.unit.container.test_sharder.TestCleavingContext.test_store ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346101.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/7d7/6f45006d2ad4502123d61c60ae22e7d7/6f45006d2ad4502123d61c60ae22e7d7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346101.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/7d7/6f45006d2ad4502123d61c60ae22e7d7/6f45006d2ad4502123d61c60ae22e7d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346105.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/cb4/2cd53168c6425d20a26c0be9ff29ecb4/2cd53168c6425d20a26c0be9ff29ecb4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346105.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/cb4/2cd53168c6425d20a26c0be9ff29ecb4/2cd53168c6425d20a26c0be9ff29ecb4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346109.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/952/5570fc45ce32ad71e904ba2564923952/5570fc45ce32ad71e904ba2564923952.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346109.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/952/5570fc45ce32ad71e904ba2564923952/5570fc45ce32ad71e904ba2564923952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346113.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/0dc/0bde4e408e41878be570261b8b4830dc/0bde4e408e41878be570261b8b4830dc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346113.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/0dc/0bde4e408e41878be570261b8b4830dc/0bde4e408e41878be570261b8b4830dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346117.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/b29/b710ccbde6538ecf9436c67e7b009b29/b710ccbde6538ecf9436c67e7b009b29.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346117.00000-0, db: /tmp/tmp_joxasw7/sda/containers/0/b29/b710ccbde6538ecf9436c67e7b009b29/b710ccbde6538ecf9436c67e7b009b29.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345877.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/529/36551784d07f4a4aef6f7e9c6d938529/36551784d07f4a4aef6f7e9c6d938529.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345877.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/529/36551784d07f4a4aef6f7e9c6d938529/36551784d07f4a4aef6f7e9c6d938529.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345880.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/15e/61aaf46c17b1e494fbb883f2cb48215e/61aaf46c17b1e494fbb883f2cb48215e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345880.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/15e/61aaf46c17b1e494fbb883f2cb48215e/61aaf46c17b1e494fbb883f2cb48215e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345883.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a86/9f96247331cfe4b0e61fad020caada86/9f96247331cfe4b0e61fad020caada86.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345883.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a86/9f96247331cfe4b0e61fad020caada86/9f96247331cfe4b0e61fad020caada86.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345886.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/2ed/277e533052e8d17bfea1e9edf9a672ed/277e533052e8d17bfea1e9edf9a672ed.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345886.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/2ed/277e533052e8d17bfea1e9edf9a672ed/277e533052e8d17bfea1e9edf9a672ed.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345889.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/9cc/625074f4da62535c56599869829ba9cc/625074f4da62535c56599869829ba9cc.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345889.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/9cc/625074f4da62535c56599869829ba9cc/625074f4da62535c56599869829ba9cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345892.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/85f/1dd9459a0cdcab7d161fbe7348c6285f/1dd9459a0cdcab7d161fbe7348c6285f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345892.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/85f/1dd9459a0cdcab7d161fbe7348c6285f/1dd9459a0cdcab7d161fbe7348c6285f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345895.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345895.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/031/17f17b97b9b8011bfd6a62697c4a7031/17f17b97b9b8011bfd6a62697c4a7031.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345898.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/e55/c5a6893da312f5919cffa3f3dae8be55/c5a6893da312f5919cffa3f3dae8be55.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345898.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/e55/c5a6893da312f5919cffa3f3dae8be55/c5a6893da312f5919cffa3f3dae8be55.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345901.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/50f/2dfda13b8b2c063211a229b170b7950f/2dfda13b8b2c063211a229b170b7950f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345901.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/50f/2dfda13b8b2c063211a229b170b7950f/2dfda13b8b2c063211a229b170b7950f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345904.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/cd4/a222c7372b2d133af8e22ccf38a71cd4/a222c7372b2d133af8e22ccf38a71cd4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345904.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/cd4/a222c7372b2d133af8e22ccf38a71cd4/a222c7372b2d133af8e22ccf38a71cd4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345907.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345907.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345913.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/27c/5bd94edf03635d01a64c72b6003bd27c/5bd94edf03635d01a64c72b6003bd27c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345913.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/27c/5bd94edf03635d01a64c72b6003bd27c/5bd94edf03635d01a64c72b6003bd27c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345922.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/362/c51b3ea38593efd802b7d34c1b4ac362/c51b3ea38593efd802b7d34c1b4ac362.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345922.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/362/c51b3ea38593efd802b7d34c1b4ac362/c51b3ea38593efd802b7d34c1b4ac362.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345928.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/e2c/7f0a6a311e0b90bcc0faf44286ae8e2c/7f0a6a311e0b90bcc0faf44286ae8e2c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345928.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/e2c/7f0a6a311e0b90bcc0faf44286ae8e2c/7f0a6a311e0b90bcc0faf44286ae8e2c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345937.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/8c8/620d98adac4bdd426cc1adf5940c98c8/620d98adac4bdd426cc1adf5940c98c8.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345937.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/8c8/620d98adac4bdd426cc1adf5940c98c8/620d98adac4bdd426cc1adf5940c98c8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345940.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/622/d60128a68626c160d5792a9a75f6f622/d60128a68626c160d5792a9a75f6f622.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345940.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/622/d60128a68626c160d5792a9a75f6f622/d60128a68626c160d5792a9a75f6f622.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345946.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345946.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345952.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/654/a2e440c8ff3caf61ef956495bd356654/a2e440c8ff3caf61ef956495bd356654.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345952.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/654/a2e440c8ff3caf61ef956495bd356654/a2e440c8ff3caf61ef956495bd356654.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345958.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c54/415e73f2320e2c700eb8695f42618c54/415e73f2320e2c700eb8695f42618c54.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345958.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c54/415e73f2320e2c700eb8695f42618c54/415e73f2320e2c700eb8695f42618c54.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345964.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c67/cd176eaa90f4d4ac9133cc9d19a15c67/cd176eaa90f4d4ac9133cc9d19a15c67.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345964.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c67/cd176eaa90f4d4ac9133cc9d19a15c67/cd176eaa90f4d4ac9133cc9d19a15c67.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345970.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345970.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345973.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/6cc/e5f758aec496ce42916476fb4ea9e6cc/e5f758aec496ce42916476fb4ea9e6cc.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345973.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/6cc/e5f758aec496ce42916476fb4ea9e6cc/e5f758aec496ce42916476fb4ea9e6cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345976.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ef6/379172803f41a0e92f620996a6740ef6/379172803f41a0e92f620996a6740ef6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345976.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ef6/379172803f41a0e92f620996a6740ef6/379172803f41a0e92f620996a6740ef6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345982.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/3cf/f7244f4b19b8781af8dbbc8ecc80c3cf/f7244f4b19b8781af8dbbc8ecc80c3cf.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345982.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/3cf/f7244f4b19b8781af8dbbc8ecc80c3cf/f7244f4b19b8781af8dbbc8ecc80c3cf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345988.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/b97/89ca8b65b5c62704a623432d03fb6b97/89ca8b65b5c62704a623432d03fb6b97.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345988.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/b97/89ca8b65b5c62704a623432d03fb6b97/89ca8b65b5c62704a623432d03fb6b97.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345997.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a53/231ffb44cd41be0edc6ed40d4a1dfa53/231ffb44cd41be0edc6ed40d4a1dfa53.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345997.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a53/231ffb44cd41be0edc6ed40d4a1dfa53/231ffb44cd41be0edc6ed40d4a1dfa53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346000.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/6fe/497c628f729b1bbb60870990f0bad6fe/497c628f729b1bbb60870990f0bad6fe.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346000.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/6fe/497c628f729b1bbb60870990f0bad6fe/497c628f729b1bbb60870990f0bad6fe.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.4:1004/sde (txn: txb7f6722afa3b42afb29be-00665c9e54) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.12:1012/sdm (txn: tx3e724b5778b7480682fa9-00665c9e54) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.3:1003/sdd (txn: tx17c01675620e42998a4a9-00665c9e54) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:21 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0134 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0116 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0146 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx88de693bf57846e0969b1-00665c9e5a" "proxy-server 3543400" 0.0020 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:54867/sda1/1/a/c" "tx88de693bf57846e0969b1-00665c9e5a" "container-server 3543400" 0.0018 "-" 3543400 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:47001/sdb1/1/a/c" "tx88de693bf57846e0969b1-00665c9e5a" "container-server 3543400" 0.0014 "-" 3543400 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx88de693bf57846e0969b1-00665c9e5a" "proxy-server 3543400" 0.0217 "-" 3543400 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx88de693bf57846e0969b1-00665c9e5a" "proxy-server 3543400" 0.0373 "-" 3543400 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx88de693bf57846e0969b1-00665c9e5a - 0.0498 - - 1717345882.464986801 1717345882.514832973 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx789fe512dd254c50b17e4-00665c9e5a" "proxy-server 3543400" 0.0018 "-" 3543400 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:47001/sdb1/1/a1/c1" "tx789fe512dd254c50b17e4-00665c9e5a" "container-server 3543400" 0.0019 "-" 3543400 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:54867/sda1/1/a1/c1" "tx789fe512dd254c50b17e4-00665c9e5a" "container-server 3543400" 0.0016 "-" 3543400 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx789fe512dd254c50b17e4-00665c9e5a" "proxy-server 3543400" 0.0191 "-" 3543400 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx789fe512dd254c50b17e4-00665c9e5a" "proxy-server 3543400" 0.0346 "-" 3543400 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx789fe512dd254c50b17e4-00665c9e5a - 0.0463 - - 1717345882.517049313 1717345882.563308001 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx57bdc77fc0ad4bf9ad2e1-00665c9e5a" "proxy-server 3543400" 0.0125 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:54867/sda1/2/a/c1" "tx57bdc77fc0ad4bf9ad2e1-00665c9e5a" "container-server 3543400" 0.0103 "-" 3543400 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:47001/sdb1/2/a/c1" "tx57bdc77fc0ad4bf9ad2e1-00665c9e5a" "container-server 3543400" 0.0018 "-" 3543400 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx57bdc77fc0ad4bf9ad2e1-00665c9e5a" "proxy-server 3543400" 0.0471 "-" 3543400 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx57bdc77fc0ad4bf9ad2e1-00665c9e5a" "proxy-server 3543400" 0.0326 "-" 3543400 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx57bdc77fc0ad4bf9ad2e1-00665c9e5a - 0.0716 - - 1717345882.565277100 1717345882.636836767 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx141616bfd45845598c2eb-00665c9e5a" "proxy-server 3543400" 0.0108 "-" 3543400 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:47001/sdb1/1/a/c2" "tx141616bfd45845598c2eb-00665c9e5a" "container-server 3543400" 0.0015 "-" 3543400 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:54867/sda1/1/a/c2" "tx141616bfd45845598c2eb-00665c9e5a" "container-server 3543400" 0.0102 "-" 3543400 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx141616bfd45845598c2eb-00665c9e5a" "proxy-server 3543400" 0.0306 "-" 3543400 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx141616bfd45845598c2eb-00665c9e5a" "proxy-server 3543400" 0.0493 "-" 3543400 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx141616bfd45845598c2eb-00665c9e5a - 0.0719 - - 1717345882.639014959 1717345882.710953236 2 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0117 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0022 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0130 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0020 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0089 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0133 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0135 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0043 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:22 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0025 "-" 3543406 0 test DEBUG: Unlinked retiring db '/tmp/tmpmckjlscn/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0061 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0028 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0108 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0066 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0049 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0027 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0124 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0168 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0127 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0051 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0143 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0059 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0124 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0020 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0020 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0019 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestContainerController.test_GET_xml test.unit.container.test_server.TestContainerController.test_GET_xml ... ok test.unit.container.test_server.TestContainerController.test_HEAD test.unit.container.test_server.TestContainerController.test_HEAD ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_HEAD_not_found test.unit.container.test_server.TestContainerController.test_HEAD_not_found ... ok test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestContainerController.test_OPTIONS test.unit.container.test_server.TestContainerController.test_OPTIONS ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata ... ok test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:18 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0206 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0008 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0138 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0038 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0130 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0007 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0127 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0045 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0128 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0124 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0003 "-" 3543390 0 test INFO: - - - [02/Jun/2024:16:31:19 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0113 "-" 3543390 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0111 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0006 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0112 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0008 "-" 3543390 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0199 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0136 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0135 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0179 "-" 3543390 - test INFO: - - - [02/Jun/2024:16:31:20 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0121 "-" 3543390 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0221 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0157 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0153 "-" 3543390 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0115 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0015 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0005 "-" 3543390 2 test INFO: - - - [02/Jun/2024:16:31:20 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0120 "-" 3543390 2 test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration ... ok proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmp_k2qnpd6/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp_k2qnpd6/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmp_k2qnpd6/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp_k2qnpd6/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0214 "-" 3543416 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0138 "-" 3543416 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0160 "-" 3543416 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0276 "-" 3543416 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txab053bd16f864fd188982-00665c9e56" "proxy-server 3543416" 0.0030 "-" 3543416 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:59373/sda1/1/a/c" "txab053bd16f864fd188982-00665c9e56" "container-server 3543416" 0.0045 "-" 3543416 0 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0246 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0108 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0113 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0060 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0072 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0052 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0053 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0068 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0095 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0161 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0034 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0129 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0139 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0245 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0085 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0128 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0242 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0170 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:23 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0180 "-" 3543406 - test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestContainerController.test_POST_insufficient_space test.unit.container.test_server.TestContainerController.test_POST_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_partition test.unit.container.test_server.TestContainerController.test_POST_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT test.unit.container.test_server.TestContainerController.test_PUT ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:18 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:52199/sdb1/1/a/c" "txab053bd16f864fd188982-00665c9e56" "container-server 3543416" 0.0025 "-" 3543416 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txab053bd16f864fd188982-00665c9e56" "proxy-server 3543416" 3.2747 "-" 3543416 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txab053bd16f864fd188982-00665c9e56" "proxy-server 3543416" 3.2948 "-" 3543416 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a/c HTTP/1.0 201 - - t - - - txab053bd16f864fd188982-00665c9e56 - 3.3118 - - 1717345878.861065388 1717345882.172902346 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txc3947a756a4946eea4c26-00665c9e5a" "proxy-server 3543416" 0.0022 "-" 3543416 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:52199/sdb1/1/a1/c1" "txc3947a756a4946eea4c26-00665c9e5a" "container-server 3543416" 0.0021 "-" 3543416 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:59373/sda1/1/a1/c1" "txc3947a756a4946eea4c26-00665c9e5a" "container-server 3543416" 0.0018 "-" 3543416 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txc3947a756a4946eea4c26-00665c9e5a" "proxy-server 3543416" 0.0252 "-" 3543416 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txc3947a756a4946eea4c26-00665c9e5a" "proxy-server 3543416" 0.0443 "-" 3543416 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txc3947a756a4946eea4c26-00665c9e5a - 0.0567 - - 1717345882.175130129 1717345882.231873751 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1b05f302d41e4c3bae1af-00665c9e5a" "proxy-server 3543416" 0.0085 "-" 3543416 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59373/sda1/2/a/c1" "tx1b05f302d41e4c3bae1af-00665c9e5a" "container-server 3543416" 0.0041 "-" 3543416 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:52199/sdb1/2/a/c1" "tx1b05f302d41e4c3bae1af-00665c9e5a" "container-server 3543416" 0.0094 "-" 3543416 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1b05f302d41e4c3bae1af-00665c9e5a" "proxy-server 3543416" 0.0338 "-" 3543416 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1b05f302d41e4c3bae1af-00665c9e5a" "proxy-server 3543416" 0.0507 "-" 3543416 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx1b05f302d41e4c3bae1af-00665c9e5a - 0.0702 - - 1717345882.234007359 1717345882.304212332 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txc06c4be45a1e4ae3ab27f-00665c9e5a" "proxy-server 3543416" 0.0093 "-" 3543416 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:52199/sdb1/1/a/c2" "txc06c4be45a1e4ae3ab27f-00665c9e5a" "container-server 3543416" 0.0018 "-" 3543416 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:59373/sda1/1/a/c2" "txc06c4be45a1e4ae3ab27f-00665c9e5a" "container-server 3543416" 0.0142 "-" 3543416 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txc06c4be45a1e4ae3ab27f-00665c9e5a" "proxy-server 3543416" 0.0360 "-" 3543416 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txc06c4be45a1e4ae3ab27f-00665c9e5a" "proxy-server 3543416" 0.0553 "-" 3543416 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/22 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txc06c4be45a1e4ae3ab27f-00665c9e5a - 0.0753 - - 1717345882.306477785 1717345882.381733179 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test WARNING: Handoff requested (9) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmptqwedg2s/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmptqwedg2s/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmptqwedg2s/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmptqwedg2s/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0125 "-" 3543390 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0100 "-" 3543390 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0087 "-" 3543390 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0111 "-" 3543390 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6f6da33c201141388afdb-00665c9e5e" "proxy-server 3543390" 0.0023 "-" 3543390 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:47497/sdb1/1/a/c" "tx6f6da33c201141388afdb-00665c9e5e" "container-server 3543390" 0.0015 "-" 3543390 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:37191/sda1/1/a/c" "tx6f6da33c201141388afdb-00665c9e5e" "container-server 3543390" 0.0022 "-" 3543390 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6f6da33c201141388afdb-00665c9e5e" "proxy-server 3543390" 0.0197 "-" 3543390 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6f6da33c201141388afdb-00665c9e5e" "proxy-server 3543390" 0.0337 "-" 3543390 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/26 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx6f6da33c201141388afdb-00665c9e5e - 0.0479 - - 1717345886.446325779 1717345886.494234085 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx1462646fcf964efd9d65a-00665c9e5e" "proxy-server 3543390" 0.0018 "-" 3543390 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:47497/sdb1/1/a1/c1" "tx1462646fcf964efd9d65a-00665c9e5e" "container-server 3543390" 0.0014 "-" 3543390 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:37191/sda1/1/a1/c1" "tx1462646fcf964efd9d65a-00665c9e5e" "container-server 3543390" 0.0013 "-" 3543390 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1462646fcf964efd9d65a-00665c9e5e" "proxy-server 3543390" 0.0185 "-" 3543390 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1462646fcf964efd9d65a-00665c9e5e" "proxy-server 3543390" 0.0313 "-" 3543390 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/26 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx1462646fcf964efd9d65a-00665c9e5e - 0.0425 - - 1717345886.496591568 1717345886.539097071 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6c407b894b854297aa30d-00665c9e5e" "proxy-server 3543390" 0.0096 "-" 3543390 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:47497/sdb1/2/a/c1" "tx6c407b894b854297aa30d-00665c9e5e" "container-server 3543390" 0.0070 "-" 3543390 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:37191/sda1/2/a/c1" "tx6c407b894b854297aa30d-00665c9e5e" "container-server 3543390" 0.0016 "-" 3543390 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx6c407b894b854297aa30d-00665c9e5e" "proxy-server 3543390" 0.0390 "-" 3543390 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx6c407b894b854297aa30d-00665c9e5e" "proxy-server 3543390" 0.0265 "-" 3543390 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/26 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx6c407b894b854297aa30d-00665c9e5e - 0.0600 - - 1717345886.541081667 1717345886.601069927 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf826e1a5c9ab46d5ac652-00665c9e5e" "proxy-server 3543390" 0.0058 "-" 3543390 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:47497/sdb1/1/a/c2" "txf826e1a5c9ab46d5ac652-00665c9e5e" "container-server 3543390" 0.0018 "-" 3543390 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:37191/sda1/1/a/c2" "txf826e1a5c9ab46d5ac652-00665c9e5e" "container-server 3543390" 0.0068 "-" 3543390 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf826e1a5c9ab46d5ac652-00665c9e5e" "proxy-server 3543390" 0.0579 "-" 3543390 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:26 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf826e1a5c9ab46d5ac652-00665c9e5e" "proxy-server 3543390" 0.0443 "-" 3543390 2 test.unit.proxy.test_server.TestECObjectController.test_GET_ec test.unit.proxy.test_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/26 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txf826e1a5c9ab46d5ac652-00665c9e5e - 0.0738 - - 1717345886.603072405 1717345886.676897287 2 test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid ... ok test INFO: - - - [02/Jun/2024:16:31:23 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:24 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0222 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:24 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:24 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0076 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:24 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:24 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:24 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0121 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0100 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0117 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0127 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0065 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0121 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0077 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0014 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0131 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0064 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0229 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0162 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0130 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0190 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0046 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0091 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0193 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0177 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3543406 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3543406 - test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0173 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0063 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0034 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0057 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0024 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges ... ok test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0101 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:26 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0050 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0050 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0023 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0022 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0024 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0021 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0019 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0049 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0040 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0038 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0037 "-" 3543406 0 test DEBUG: Unlinked retiring db '/tmp/tmpa8ki4nxl/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0059 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0033 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0048 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0034 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0053 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:27 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0033 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestContainerController.test_PUT_account_update test.unit.container.test_server.TestContainerController.test_PUT_account_update ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0137 "-" 3543406 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3543406 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0182 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0124 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0099 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0084 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0102 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0052 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0131 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0108 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:30 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0225 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0062 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0097 "-" 3543406 - test ERROR: ERROR Account update failed with 127.0.0.1:37157/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0015 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3543406 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0005 "-" 3543406 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0158 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0048 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0105 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543406 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container ... ok test INFO: - - - [02/Jun/2024:16:31:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0101 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0153 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0023 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0021 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0020 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0125 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0059 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:32 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0053 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0135 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0056 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0069 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0011 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0066 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0027 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0020 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0019 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0018 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0020 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0014 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0035 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0022 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0018 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0049 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0012 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0018 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container ... ok test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0025 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0022 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:33 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0025 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0027 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0017 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0018 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test DEBUG: Unlinked retiring db '/tmp/tmpgyw06zan/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0019 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0012 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0205 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0022 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0019 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0171 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0156 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0044 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0015 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0016 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0054 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0044 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0121 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0049 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0165 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0030 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0185 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0072 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0049 "-" 3543406 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0053 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0050 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0127 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0055 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT_utf8 test.unit.container.test_server.TestContainerController.test_PUT_utf8 ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestContainerController.test_UPDATE test.unit.container.test_server.TestContainerController.test_UPDATE ... ok test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestContainerController.test__call__raise_timeout test.unit.container.test_server.TestContainerController.test__call__raise_timeout ... ok test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device ... ok test.unit.container.test_server.TestContainerController.test_acl_container test.unit.container.test_server.TestContainerController.test_acl_container ... ok test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543406 0 test DEBUG: Unlinked retiring db '/tmp/tmp0b0frumo/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:31:35 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0124 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:35 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0139 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:35 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0120 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0022 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0110 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:36 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:36 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:36 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0005 "-" 3543406 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:31:36 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0045 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:31:36 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0004 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0004 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0068 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:36 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0031 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0005 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0185 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0035 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0104 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0159 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0142 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0151 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0022 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0066 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3543406 1 test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestContainerController.test_correct_allowed_method test.unit.container.test_server.TestContainerController.test_correct_allowed_method ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestContainerController.test_creation test.unit.container.test_server.TestContainerController.test_creation ... ok test.unit.container.test_server.TestContainerController.test_delete_auto_create test.unit.container.test_server.TestContainerController.test_delete_auto_create ... ok test.unit.container.test_server.TestContainerController.test_deleted_headers test.unit.container.test_server.TestContainerController.test_deleted_headers ... ok test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestContainerController.test_list_allowed_methods test.unit.container.test_server.TestContainerController.test_list_allowed_methods ... ok test.unit.container.test_server.TestContainerController.test_listing_with_reserved test.unit.container.test_server.TestContainerController.test_listing_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_log_line_format test.unit.container.test_server.TestContainerController.test_log_line_format ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346006.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346006.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346012.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/708/e0212a17aebc68331e7badab59a1c708/e0212a17aebc68331e7badab59a1c708.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346012.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/708/e0212a17aebc68331e7badab59a1c708/e0212a17aebc68331e7badab59a1c708.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346018.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/d60/b7458e039a58a35603e0d761c87e4d60/b7458e039a58a35603e0d761c87e4d60.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346018.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/d60/b7458e039a58a35603e0d761c87e4d60/b7458e039a58a35603e0d761c87e4d60.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346024.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/974/7f371af32892408d278c8d2b4e2ad974/7f371af32892408d278c8d2b4e2ad974.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346024.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/974/7f371af32892408d278c8d2b4e2ad974/7f371af32892408d278c8d2b4e2ad974.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346030.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346030.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346033.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/dd1/84f12721bef5ffa94e25c6863a1badd1/84f12721bef5ffa94e25c6863a1badd1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346033.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/dd1/84f12721bef5ffa94e25c6863a1badd1/84f12721bef5ffa94e25c6863a1badd1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346036.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/271/8f38eee399580620c5ed9c4937aa7271/8f38eee399580620c5ed9c4937aa7271.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346036.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/271/8f38eee399580620c5ed9c4937aa7271/8f38eee399580620c5ed9c4937aa7271.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346042.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/636/3e34021029a00eb6bde27abaad41b636/3e34021029a00eb6bde27abaad41b636.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346042.00000-0, db: /tmp/tmpd622iaxp/sda/containers/0/636/3e34021029a00eb6bde27abaad41b636/3e34021029a00eb6bde27abaad41b636.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345884.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/92a/66e219a1b6e02d4ca1e7cfe50b65192a/66e219a1b6e02d4ca1e7cfe50b65192a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345889.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9cc/625074f4da62535c56599869829ba9cc/625074f4da62535c56599869829ba9cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345894.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/f9c/1df73cb504328c9b033c3e3ad9f66f9c/1df73cb504328c9b033c3e3ad9f66f9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345899.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/882/70111bd24b04c700ef60b5734a529882/70111bd24b04c700ef60b5734a529882.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345904.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/cd4/a222c7372b2d133af8e22ccf38a71cd4/a222c7372b2d133af8e22ccf38a71cd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345909.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/417/b9eb057ae1e3b48e96e613a12ad94417/b9eb057ae1e3b48e96e613a12ad94417.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345924.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/dbc/5eb9fcf1d1efac0a71f8864594814dbc/5eb9fcf1d1efac0a71f8864594814dbc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345929.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b65/e499cb8202d58e7ad5f94e5e6fe38b65/e499cb8202d58e7ad5f94e5e6fe38b65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345939.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d69/82df5f6d5f930f5efcd5c422ac1bcd69/82df5f6d5f930f5efcd5c422ac1bcd69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345944.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7eb/5c7c29ada09e7f3d71c4282e565b37eb/5c7c29ada09e7f3d71c4282e565b37eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345954.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a73/682f3d8315d3c4aa31c9eb70cc5d1a73/682f3d8315d3c4aa31c9eb70cc5d1a73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345959.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/589/718c8292da0acdeb421c88f5fbaa0589/718c8292da0acdeb421c88f5fbaa0589.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345964.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c67/cd176eaa90f4d4ac9133cc9d19a15c67/cd176eaa90f4d4ac9133cc9d19a15c67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345969.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb/b6cc2c43aa50ebab62ed6dbdb85fd9eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345974.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/469/6cc61909e7a11534cf9facccb2cd6469/6cc61909e7a11534cf9facccb2cd6469.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345984.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/8f4/4311c9db62e11eef65dee503d09048f4/4311c9db62e11eef65dee503d09048f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345989.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/975/40c65bb6fef1f9af0ce0f9a889d51975/40c65bb6fef1f9af0ce0f9a889d51975.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345999.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ac8/1ce2a5eebdc14ab3252897f319daeac8/1ce2a5eebdc14ab3252897f319daeac8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346004.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e22/5dae92396724bfb4eee10f10de10de22/5dae92396724bfb4eee10f10de10de22.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346014.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a32/e62cfd993cf033cf47f6d26df631ca32/e62cfd993cf033cf47f6d26df631ca32.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346019.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/60e/9b98addea9ee305bd983149f505cf60e/9b98addea9ee305bd983149f505cf60e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346024.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/974/7f371af32892408d278c8d2b4e2ad974/7f371af32892408d278c8d2b4e2ad974.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346029.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c51/014702cf3e6364f1cc17bf7f2d3dac51/014702cf3e6364f1cc17bf7f2d3dac51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346034.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e6b/1e17050f206eb050635baa94217f8e6b/1e17050f206eb050635baa94217f8e6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346044.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d2f/ca1905f392adc798e70bd9a684deed2f/ca1905f392adc798e70bd9a684deed2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346049.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/fa1/4a860ba03606b566205549d0529fafa1/4a860ba03606b566205549d0529fafa1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346054.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a65/827b1676df1f920fb9a53e5a1cae0a65/827b1676df1f920fb9a53e5a1cae0a65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346059.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/56f/9a8b64160e6ae421965eb79c4a6cc56f/9a8b64160e6ae421965eb79c4a6cc56f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346064.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/3dc/3f52d56d64559e88e28eb55dd2df43dc/3f52d56d64559e88e28eb55dd2df43dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346069.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6f3/d75465b42e5dc3b12c47edd8623656f3/d75465b42e5dc3b12c47edd8623656f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346074.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/53b/217ce7a7d09e39777860c6633c30c53b/217ce7a7d09e39777860c6633c30c53b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346079.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/078/6433e58e0348b3f47247aa2b74b0e078/6433e58e0348b3f47247aa2b74b0e078.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346084.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6d1/9157046c9fe87946853035549dbb46d1/9157046c9fe87946853035549dbb46d1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346089.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/4d4/246c30bc5935967d840225fd570ca4d4/246c30bc5935967d840225fd570ca4d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346094.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/08c/6a2e5f7dcf475cf183f4a9516e0af08c/6a2e5f7dcf475cf183f4a9516e0af08c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346099.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a70/a7c571ebacba5fa57e73a3256a942a70/a7c571ebacba5fa57e73a3256a942a70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346104.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2ec/b74714108d3c09569c58c8d76c7342ec/b74714108d3c09569c58c8d76c7342ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346109.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/952/5570fc45ce32ad71e904ba2564923952/5570fc45ce32ad71e904ba2564923952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346114.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b11/04c52fcff46616dce478b3ff8e7e5b11/04c52fcff46616dce478b3ff8e7e5b11.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346119.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2d7/2128a81fd3c0376e7654966effa7b2d7/2128a81fd3c0376e7654966effa7b2d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346124.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/167/324c357ff9d42ac89e9cb793560e9167/324c357ff9d42ac89e9cb793560e9167.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346129.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/db1/db7683fddb5cf5e5bcb731ae54843db1/db7683fddb5cf5e5bcb731ae54843db1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346134.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/502/32756134775a8993c1c258262cbde502/32756134775a8993c1c258262cbde502.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346139.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/368/3b8b2bde8514da353a6b30929a3a8368/3b8b2bde8514da353a6b30929a3a8368.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346144.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/fb4/7f994b76be14c56697ed8eca9788cfb4/7f994b76be14c56697ed8eca9788cfb4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346149.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/883/bd721bff4f0ca78065850fac18d99883/bd721bff4f0ca78065850fac18d99883.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346154.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/002/346e0e57d1918d2e1822b2019bcb7002/346e0e57d1918d2e1822b2019bcb7002.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346159.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2e0/72f9effd13247b9a8a1bf5b5627fe2e0/72f9effd13247b9a8a1bf5b5627fe2e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346164.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/acb/9ec52493973925b67b6fca4a12667acb/9ec52493973925b67b6fca4a12667acb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346169.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/bd8/42bc7c3a3b03b275f2b739f070fcdbd8/42bc7c3a3b03b275f2b739f070fcdbd8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346174.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ff7/b255d085c3c53ca26177165958e1eff7/b255d085c3c53ca26177165958e1eff7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346179.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/5a8/e8d306be8eace3b9ffde02c606c385a8/e8d306be8eace3b9ffde02c606c385a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346184.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c4f/3d33e391e9c9395fc2b31203a8c40c4f/3d33e391e9c9395fc2b31203a8c40c4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346189.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ea5/012a04332b999578a193ef8d13e03ea5/012a04332b999578a193ef8d13e03ea5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346194.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/227/e62f94e3aab5b55c2f28961381a11227/e62f94e3aab5b55c2f28961381a11227.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346199.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/013/5fc61e3cfcebf5a8ce3b99cc5a58b013/5fc61e3cfcebf5a8ce3b99cc5a58b013.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346204.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/962/9cb103d526d43a66a5a8f048a1198962/9cb103d526d43a66a5a8f048a1198962.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346209.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/672/2c6d76486c1f97276fa47d9c9a44d672/2c6d76486c1f97276fa47d9c9a44d672.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346214.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/797/14b81786bac9a5b4b378afc86aed2797/14b81786bac9a5b4b378afc86aed2797.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346219.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/367/a959f8f9577df2fbf75cd54314923367/a959f8f9577df2fbf75cd54314923367.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346224.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/467/9ff781c5ed8226e3d2242ed7ab2ef467/9ff781c5ed8226e3d2242ed7ab2ef467.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346229.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/89a/d756bd0e65cdb91b393c05c8c57d889a/d756bd0e65cdb91b393c05c8c57d889a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346234.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/864/37a2a969437115a0d1fc0ad5d86ab864/37a2a969437115a0d1fc0ad5d86ab864.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346239.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e6b/1cc7179e299f33ae2a340c8eba37ae6b/1cc7179e299f33ae2a340c8eba37ae6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346244.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/583/e566ccefc66b1dcc5171a3cf8215a583/e566ccefc66b1dcc5171a3cf8215a583.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346249.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b94/c7d39ebb29e6831167390da6527edb94/c7d39ebb29e6831167390da6527edb94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346254.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d59/c02e67e3e106e65d589856186663ed59/c02e67e3e106e65d589856186663ed59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346259.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/581/8a845be8899b80641d35e00cf0e96581/8a845be8899b80641d35e00cf0e96581.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346264.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/af1/525c6711c809b0547e983393d721eaf1/525c6711c809b0547e983393d721eaf1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346269.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d3d/ce14c18ba1bc66f6f161d9e402d7fd3d/ce14c18ba1bc66f6f161d9e402d7fd3d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346274.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/692/20b8a3dba44653e379ef9e7b2e925692/20b8a3dba44653e379ef9e7b2e925692.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346279.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/396/7272307e65b5d3207e58e03cf201f396/7272307e65b5d3207e58e03cf201f396.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346284.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/5b0/391482bb0808615005868af91bed35b0/391482bb0808615005868af91bed35b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346289.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/192/c006d88ae42284f36a5ba73896018192/c006d88ae42284f36a5ba73896018192.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346294.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/412/02df473d450e56b4fc7180e9347eb412/02df473d450e56b4fc7180e9347eb412.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346299.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/546/317e5fb0b11565af8533d73963d00546/317e5fb0b11565af8533d73963d00546.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346304.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/651/df803259ed0bdf6c6b86413887227651/df803259ed0bdf6c6b86413887227651.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346309.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/43e/17c969e22ad9fb4e4970a15b0b1a943e/17c969e22ad9fb4e4970a15b0b1a943e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346314.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/16d/82f1c2b3d47799b02aa83266c2ace16d/82f1c2b3d47799b02aa83266c2ace16d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346319.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/804/07fb83d270b00e323713215e60b07804/07fb83d270b00e323713215e60b07804.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346324.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e00/f9aec1c0cc05e85022504722e29fbe00/f9aec1c0cc05e85022504722e29fbe00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346329.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0eb/748e03e38b5817783fcc5adc9514b0eb/748e03e38b5817783fcc5adc9514b0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346334.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9eb/666078c758a3a7c0d411e438abd529eb/666078c758a3a7c0d411e438abd529eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346339.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7e2/99fc83c8df8f2e28c00f78586a53f7e2/99fc83c8df8f2e28c00f78586a53f7e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346344.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/50b/27fb210b4a7dae23f5d04ddbe7a7550b/27fb210b4a7dae23f5d04ddbe7a7550b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346349.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d76/b21fd4d1f56f43623336de89ab6a3d76/b21fd4d1f56f43623336de89ab6a3d76.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346354.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/925/83216fbb83b0cda56b09d0d97203d925/83216fbb83b0cda56b09d0d97203d925.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346359.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/547/63061137146992e76bb8199dded2f547/63061137146992e76bb8199dded2f547.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346364.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/457/112ddcec7164b61f9226c48b68957457/112ddcec7164b61f9226c48b68957457.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346369.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/19f/8d2af439a57a51976c2e8b54fc97b19f/8d2af439a57a51976c2e8b54fc97b19f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346374.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b0c/6a0fabe4b3328900eeab1ff0c9904b0c/6a0fabe4b3328900eeab1ff0c9904b0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346379.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ceb/e72472d16bcf71d72c22e0b26c450ceb/e72472d16bcf71d72c22e0b26c450ceb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346384.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ccb/010f921613a72dc81bed93a95285fccb/010f921613a72dc81bed93a95285fccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346389.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d63/2a16ebdedaca835956885cfdeca76d63/2a16ebdedaca835956885cfdeca76d63.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346394.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/73d/e904eb5e9cfd954f9aac001f0643f73d/e904eb5e9cfd954f9aac001f0643f73d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346399.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/db5/69687b9b60a4830eb53911f67a84fdb5/69687b9b60a4830eb53911f67a84fdb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346404.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9f3/405fbab0e9ce66cd0a73f7917308c9f3/405fbab0e9ce66cd0a73f7917308c9f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346409.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/f68/609e91029b874e5764bcd6dc53796f68/609e91029b874e5764bcd6dc53796f68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346414.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/947/c80d28a340a6232377229046ccd53947/c80d28a340a6232377229046ccd53947.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346419.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7b4/83cdff253f148422fff5ca4c15f1d7b4/83cdff253f148422fff5ca4c15f1d7b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346424.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/378/6c2b07bcb2e9e0b25f3c8bea02462378/6c2b07bcb2e9e0b25f3c8bea02462378.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346429.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/012/ac016eb0258610b47ad658dcfca6f012/ac016eb0258610b47ad658dcfca6f012.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346434.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a9a/2322c1a1b57bf6c9b889c73c457b1a9a/2322c1a1b57bf6c9b889c73c457b1a9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346439.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d49/841ad5001983132019d54d89faa93d49/841ad5001983132019d54d89faa93d49.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346444.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/cb7/f1ee7718f93965a7ba5df08d75f2bcb7/f1ee7718f93965a7ba5df08d75f2bcb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346449.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b20/2d4a8281d8499e65d05224aac2b62b20/2d4a8281d8499e65d05224aac2b62b20.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346454.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/55c/19597d94f29b94b63a5b34bc8451555c/19597d94f29b94b63a5b34bc8451555c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346459.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0a9/75296332ce45eb9d13aa86716322b0a9/75296332ce45eb9d13aa86716322b0a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346464.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2b2/799ea392bd529e4d53a635807aad12b2/799ea392bd529e4d53a635807aad12b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346469.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/949/683b1a52b6ad7242a319309dcb310949/683b1a52b6ad7242a319309dcb310949.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346474.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7d7/34dbae561b47aa7379b4523a7f5167d7/34dbae561b47aa7379b4523a7f5167d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346479.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/407/9e49b6c940c3174ad635337f4a0d0407/9e49b6c940c3174ad635337f4a0d0407.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346484.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/217/2311a74e791bb24c77a14dadf9636217/2311a74e791bb24c77a14dadf9636217.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346489.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/881/fbca0193656dcf55e0502bed47f07881/fbca0193656dcf55e0502bed47f07881.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346494.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d7a/2187cf4bec89dcfbd29aa3ef01d4cd7a/2187cf4bec89dcfbd29aa3ef01d4cd7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346499.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/5b5/528b7083001c42642853b775cacf25b5/528b7083001c42642853b775cacf25b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346504.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6d2/ac5d3f02c1041d21a841b92228b826d2/ac5d3f02c1041d21a841b92228b826d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346509.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/582/c554a928f073e9b98827212eb36cc582/c554a928f073e9b98827212eb36cc582.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346514.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/fae/3c483b5f692139fdd01e3d0ea08d8fae/3c483b5f692139fdd01e3d0ea08d8fae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346519.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9bd/e5402c7e1548cb3c369d6a2f1816b9bd/e5402c7e1548cb3c369d6a2f1816b9bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346524.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c59/baf6af78af602cd4f360fefd15774c59/baf6af78af602cd4f360fefd15774c59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346529.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/91c/4bc4708abefa999410e3b5fd751ba91c/4bc4708abefa999410e3b5fd751ba91c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346534.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2c1/e7a1682ca207bd38c60af64803a422c1/e7a1682ca207bd38c60af64803a422c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346539.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7ed/41ec06bd01c779cbd7118437be4d77ed/41ec06bd01c779cbd7118437be4d77ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346544.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/cc3/79eef62c81e8c417b91c40e4e4cf4cc3/79eef62c81e8c417b91c40e4e4cf4cc3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346549.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/2c0/8b9284141f12e46f87f4184c25dcf2c0/8b9284141f12e46f87f4184c25dcf2c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346554.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/12b/9bb3da0fec0b7a7d024ce10748dde12b/9bb3da0fec0b7a7d024ce10748dde12b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346559.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/3f2/8cd286e864131bdaaebc8001c98473f2/8cd286e864131bdaaebc8001c98473f2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346564.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d5a/9ae70ab335f7dbcc5ec56e30ec44bd5a/9ae70ab335f7dbcc5ec56e30ec44bd5a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346569.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/003/4567a19e507591d943664b969053a003/4567a19e507591d943664b969053a003.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346574.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/efd/740df114b3703f4b8fddc15d8baeeefd/740df114b3703f4b8fddc15d8baeeefd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346579.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/461/52eecb5a21a312c63d361d1171c7a461/52eecb5a21a312c63d361d1171c7a461.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346584.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9ad/8e6c7ded34396f72b91f0045da40d9ad/8e6c7ded34396f72b91f0045da40d9ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346589.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b77/612079a85dee173bbd599aca61015b77/612079a85dee173bbd599aca61015b77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346594.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/4cd/780e90710e8ba184cfe4734155a964cd/780e90710e8ba184cfe4734155a964cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346599.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/8e8/770b2104c5d1e92ca7ffd6823e3508e8/770b2104c5d1e92ca7ffd6823e3508e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346604.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/213/f9fcb53c793072af32eb334345069213/f9fcb53c793072af32eb334345069213.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346609.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/287/941c0e207c960dbd9b04329972c66287/941c0e207c960dbd9b04329972c66287.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346614.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/987/f9c23850017ac8881690b032b2865987/f9c23850017ac8881690b032b2865987.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346619.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/4e8/d4e33dbe358d1ab0ddc0b5e7e0c574e8/d4e33dbe358d1ab0ddc0b5e7e0c574e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346624.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7a3/dddd38d439b29f42c149653e958bc7a3/dddd38d439b29f42c149653e958bc7a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346629.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/940/0000159b0387c3b7106e2897e8992940/0000159b0387c3b7106e2897e8992940.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346634.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/07b/423326c11c9a17405d4f7f9e5985707b/423326c11c9a17405d4f7f9e5985707b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346639.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/1de/aa85a8f683a35bc2ece861103982c1de/aa85a8f683a35bc2ece861103982c1de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346644.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/380/30fe46d502dca9a1f34b0d3e6dcba380/30fe46d502dca9a1f34b0d3e6dcba380.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346649.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/410/b22fc1ff74d5d455fd66a5cd8ba77410/b22fc1ff74d5d455fd66a5cd8ba77410.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346654.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/173/da0e11644432c597bc3ee60e6ca6e173/da0e11644432c597bc3ee60e6ca6e173.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346659.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/1e6/a767283c1cb1490f12d446a6943711e6/a767283c1cb1490f12d446a6943711e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346664.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/36d/efb8e3ce8efd2d281e16bce4e0e4836d/efb8e3ce8efd2d281e16bce4e0e4836d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346669.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/3dd/5663fd172b1fd767efa3481daee2f3dd/5663fd172b1fd767efa3481daee2f3dd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346674.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/35e/56a25bfc0e53d56c058b40caf77a535e/56a25bfc0e53d56c058b40caf77a535e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346679.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/409/a0d20fe252a008a606ed52b46e7a5409/a0d20fe252a008a606ed52b46e7a5409.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346684.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c4b/517171f252d08c1c9fb6a9dd88f2bc4b/517171f252d08c1c9fb6a9dd88f2bc4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346689.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0cd/0d25f53176fc70d0048acb81e5f530cd/0d25f53176fc70d0048acb81e5f530cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346694.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/001/c6f1e7f7a67365566067c45beda8e001/c6f1e7f7a67365566067c45beda8e001.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346699.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/98b/197a4648d3baa1e88b6b245e0c9c198b/197a4648d3baa1e88b6b245e0c9c198b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346704.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a0c/027742e868db059b70c18720250f6a0c/027742e868db059b70c18720250f6a0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346709.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d07/04ad17e5c3b4aa8e6e8adccd25d19d07/04ad17e5c3b4aa8e6e8adccd25d19d07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346714.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/be3/5cee21064623ca2394ca60007472dbe3/5cee21064623ca2394ca60007472dbe3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346719.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/f89/eb502f2fac0e72aaf3cf40333b207f89/eb502f2fac0e72aaf3cf40333b207f89.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346724.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/74b/c35d7d36ad892264612bc78c536a974b/c35d7d36ad892264612bc78c536a974b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346729.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/13d/f6f4b130d55ae852ce09698af313b13d/f6f4b130d55ae852ce09698af313b13d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346734.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/925/613ff077fcfb9807049097b4acda8925/613ff077fcfb9807049097b4acda8925.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346739.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/5ed/07b4de0f9002c31951d2a006f4d1d5ed/07b4de0f9002c31951d2a006f4d1d5ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346744.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e08/5df782d85f60f91d19e4b468a2e7ae08/5df782d85f60f91d19e4b468a2e7ae08.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346749.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9c3/777531632617e0b018b49190dbb079c3/777531632617e0b018b49190dbb079c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346754.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c72/d4f0d5ef497e9dd8c1792a80d1e1dc72/d4f0d5ef497e9dd8c1792a80d1e1dc72.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346759.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/15a/da07386545c33c5a000ed75d015cf15a/da07386545c33c5a000ed75d015cf15a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346764.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a7e/5bb995f5d1afd810e3e28c9468263a7e/5bb995f5d1afd810e3e28c9468263a7e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346769.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/30c/568fcbd7483411091e17541cd9b3f30c/568fcbd7483411091e17541cd9b3f30c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346774.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/109/ecce68027d633e29c12b599ad50d2109/ecce68027d633e29c12b599ad50d2109.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346779.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e74/343614eef3cf64aff54e5a2add4efe74/343614eef3cf64aff54e5a2add4efe74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346784.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/511/ea9326d9c91753595a319c7c899f5511/ea9326d9c91753595a319c7c899f5511.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346789.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/4d4/e141247637b6758c2b9c6df3a87744d4/e141247637b6758c2b9c6df3a87744d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346794.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/403/e61d54dd27800e578e52d4d3d9669403/e61d54dd27800e578e52d4d3d9669403.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346799.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d23/41d7c2c7f635e60d3ceabf629e907d23/41d7c2c7f635e60d3ceabf629e907d23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346804.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a96/69c6356efbdfd7980d1c1197e0e6fa96/69c6356efbdfd7980d1c1197e0e6fa96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346809.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/09b/3d3231568a42b3262dc90e06ad36209b/3d3231568a42b3262dc90e06ad36209b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346814.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ed7/c62b8da553f61fc683007acd19577ed7/c62b8da553f61fc683007acd19577ed7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346819.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/904/51a781328b80ec70e6299b6aa2c4e904/51a781328b80ec70e6299b6aa2c4e904.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346824.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ed4/bde45470a7088d629eb6d8679b456ed4/bde45470a7088d629eb6d8679b456ed4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346829.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/87f/079bcfc3149c5ac14b4857768841b87f/079bcfc3149c5ac14b4857768841b87f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346834.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/913/97b35cb7e71a7d38591472a0462c0913/97b35cb7e71a7d38591472a0462c0913.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346839.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a13/363e1c59293ee28ce16f308002263a13/363e1c59293ee28ce16f308002263a13.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346844.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ed7/e80266a6d8ffe1b63ffcdfe801a06ed7/e80266a6d8ffe1b63ffcdfe801a06ed7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346849.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/926/bce5fb8917533a1a2b7d31412205e926/bce5fb8917533a1a2b7d31412205e926.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346854.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ab9/bc8bc95feccb284149506142482b4ab9/bc8bc95feccb284149506142482b4ab9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346859.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/85d/15b696863bda11c70b3565922b22185d/15b696863bda11c70b3565922b22185d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346864.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/324/4dd3ed8c455b01850ab7033b2be6e324/4dd3ed8c455b01850ab7033b2be6e324.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346869.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/4dc/38fc857dc75e5b364ce1387e820b94dc/38fc857dc75e5b364ce1387e820b94dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346874.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/893/c09eb4126a4d3da3d9e07a2ff0751893/c09eb4126a4d3da3d9e07a2ff0751893.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346879.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/281/7819ac3024b31cffccb890ea597ee281/7819ac3024b31cffccb890ea597ee281.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346884.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/633/c43acfbec9e3e59cc9eccd5bc1e16633/c43acfbec9e3e59cc9eccd5bc1e16633.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346889.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b84/f99ff80c7d237b11cf41e90c2201cb84/f99ff80c7d237b11cf41e90c2201cb84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346894.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/84d/f83621523ec96e0eb0cd3d8c4cc5784d/f83621523ec96e0eb0cd3d8c4cc5784d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346899.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c2a/d3cacbae70ad6e44600d7cd58ee7dc2a/d3cacbae70ad6e44600d7cd58ee7dc2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346904.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/72b/cbfadb9524ca1884639d65cdc219372b/cbfadb9524ca1884639d65cdc219372b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346909.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9b6/abf27e7640cb516f57de9ceb886de9b6/abf27e7640cb516f57de9ceb886de9b6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346914.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7e9/0cd9d5b8f228901b51bb0f6402bf97e9/0cd9d5b8f228901b51bb0f6402bf97e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346919.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/69d/e5256fe299c596a4d47e40fd802e269d/e5256fe299c596a4d47e40fd802e269d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346924.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a61/e65aad627e5afb4ad3733875f0eeda61/e65aad627e5afb4ad3733875f0eeda61.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346929.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0ef/c679b1c733ff0ab13993db737bd770ef/c679b1c733ff0ab13993db737bd770ef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346934.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/0d7/a570bfb083098c294c8c7c672ba340d7/a570bfb083098c294c8c7c672ba340d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346939.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/1c5/1be1c69ae395cc22c179ed6c12ff51c5/1be1c69ae395cc22c179ed6c12ff51c5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346944.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6f9/f2209291f66b87c38d0bbb26d7fb16f9/f2209291f66b87c38d0bbb26d7fb16f9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346949.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/837/b2a8776a4f9676ce7b9a1a6f700da837/b2a8776a4f9676ce7b9a1a6f700da837.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346954.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/506/dc1e24694fe28c24595368898037b506/dc1e24694fe28c24595368898037b506.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346959.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6d5/18af65c9acffb8fce38f0672c01716d5/18af65c9acffb8fce38f0672c01716d5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346964.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/a24/cf099972f7afb7b91f49576020d27a24/cf099972f7afb7b91f49576020d27a24.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346969.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/087/4931ae612cc49081ad6bd5bb53927087/4931ae612cc49081ad6bd5bb53927087.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346974.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ee2/d5d9527ba915fd49b7d479deff6ecee2/d5d9527ba915fd49b7d479deff6ecee2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346979.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/8ff/7853bf51cdb633b85401af3ba31528ff/7853bf51cdb633b85401af3ba31528ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346984.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b16/60ad175392ed3498abe729e9625e4b16/60ad175392ed3498abe729e9625e4b16.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346989.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/fd4/96c4cd3691fd0983ae797a1a5f5c1fd4/96c4cd3691fd0983ae797a1a5f5c1fd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346994.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6c4/69dab5ba187c9d11dff0a115415966c4/69dab5ba187c9d11dff0a115415966c4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346999.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/924/8e690ae57d8c8ae63f2ad599e158d924/8e690ae57d8c8ae63f2ad599e158d924.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347004.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6be/d3c47849c5e457df6278b84b447f16be/d3c47849c5e457df6278b84b447f16be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347009.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/7ca/cda6b840d5fba0bf0d795d6f62f9d7ca/cda6b840d5fba0bf0d795d6f62f9d7ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347014.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/dcc/21aa45675702cae4a677b64609508dcc/21aa45675702cae4a677b64609508dcc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347019.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/11a/2440ed188bd9f3e4a5d4acb6a900111a/2440ed188bd9f3e4a5d4acb6a900111a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347024.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/91f/a3b426bf69a101a2f120a9172f7ab91f/a3b426bf69a101a2f120a9172f7ab91f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347029.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/92b/0ec6d50539a279bfd1e8337c3cb1592b/0ec6d50539a279bfd1e8337c3cb1592b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347034.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/473/9925e9bd213f6d4ad75e7bb28c223473/9925e9bd213f6d4ad75e7bb28c223473.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347039.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/254/a5b7062b6aefa600679554ecde19b254/a5b7062b6aefa600679554ecde19b254.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347044.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/ec2/d416f15176dfec00c7c9933c01818ec2/d416f15176dfec00c7c9933c01818ec2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347049.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/95e/81c1713b2dc949b48a13b4d818b1995e/81c1713b2dc949b48a13b4d818b1995e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347054.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/416/455c4fc39b96ad4cade47ba2e357c416/455c4fc39b96ad4cade47ba2e357c416.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347059.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/595/a95d970b0ed848c36b37bdd784ede595/a95d970b0ed848c36b37bdd784ede595.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347064.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/955/3bc08d25fb69dad5434534547b711955/3bc08d25fb69dad5434534547b711955.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347069.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/c5b/0afb716b99bfec4978918b86828f6c5b/0afb716b99bfec4978918b86828f6c5b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347074.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/feb/dae82bc1338eca8047f8f1725be88feb/dae82bc1338eca8047f8f1725be88feb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347079.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/b32/3e2fb09e5a2e3df3c84560b35486fb32/3e2fb09e5a2e3df3c84560b35486fb32.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347084.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/6b0/80ecc63a05a391939e5ecdba392fb6b0/80ecc63a05a391939e5ecdba392fb6b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347089.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/231/9c4475ef7da6f8aedd584e0610726231/9c4475ef7da6f8aedd584e0610726231.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347094.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/465/4b202fd66f86f3596570cbea18d82465/4b202fd66f86f3596570cbea18d82465.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347099.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/e3f/00fc6019b16a8bf5b586e4220054ae3f/00fc6019b16a8bf5b586e4220054ae3f.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0165 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0071 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0122 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0018 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0110 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0123 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0438 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0711 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0141 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0125 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0014 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0167 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0012 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0109 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0031 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0157 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0002 "-" 3543406 nada test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0002 "-" 3543406 999 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0130 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/c_nulo" 204 - "-" "-" "-" 0.0016 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0122 "-" 3543406 1 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/c_unu" 204 - "-" "-" "-" 0.0016 "-" 3543406 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0144 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0172 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0033 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0148 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test.unit.container.test_server.TestContainerController.test_markers_with_reserved test.unit.container.test_server.TestContainerController.test_markers_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_not_allowed_method test.unit.container.test_server.TestContainerController.test_not_allowed_method ... ok test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps ... ok test.unit.container.test_server.TestContainerController.test_object_update_with_offset test.unit.container.test_server.TestContainerController.test_object_update_with_offset ... ok test.unit.container.test_server.TestContainerController.test_params_format test.unit.container.test_server.TestContainerController.test_params_format ... ok test.unit.container.test_server.TestContainerController.test_params_utf8 test.unit.container.test_server.TestContainerController.test_params_utf8 ... ok test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file ... ok test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0129 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0032 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0067 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0094 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0133 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0011 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0118 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0047 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0096 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0108 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0157 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:37 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0193 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0116 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:37 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0013 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0149 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0152 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0134 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0068 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0177 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0089 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0203 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0031 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0029 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0004 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0147 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0057 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0119 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0012 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0115 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0032 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0132 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0007 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0125 "-" 3543406 0 test.unit.container.test_server.TestContainerController.test_prefix_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_put_auto_create test.unit.container.test_server.TestContainerController.test_put_auto_create ... ok test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestContainerController.test_serv_reserv test.unit.container.test_server.TestContainerController.test_serv_reserv ... ok test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestContainerController.test_through_call test.unit.container.test_server.TestContainerController.test_through_call ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path test.unit.container.test_server.TestContainerController.test_through_call_invalid_path ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestContainerController.test_weird_content_types test.unit.container.test_server.TestContainerController.test_weird_content_types ... ok test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE test.unit.obj.test_server.TestObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET test.unit.obj.test_server.TestObjectController.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_but_expired test.unit.obj.test_server.TestObjectController.test_GET_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match test.unit.obj.test_server.TestObjectController.test_GET_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.obj.test_server.TestObjectController.test_GET_if_none_match test.unit.obj.test_server.TestObjectController.test_GET_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine test.unit.obj.test_server.TestObjectController.test_GET_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD test.unit.obj.test_server.TestObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_OPTIONS test.unit.obj.test_server.TestObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_but_expired test.unit.obj.test_server.TestObjectController.test_POST_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_connection test.unit.obj.test_server.TestObjectController.test_POST_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_invalid_path test.unit.obj.test_server.TestObjectController.test_POST_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_not_exist test.unit.obj.test_server.TestObjectController.test_POST_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_system_metadata test.unit.obj.test_server.TestObjectController.test_POST_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_update_meta test.unit.obj.test_server.TestObjectController.test_POST_update_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_commits_data test.unit.obj.test_server.TestObjectController.test_PUT_commits_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_common test.unit.obj.test_server.TestObjectController.test_PUT_common ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_connection test.unit.obj.test_server.TestObjectController.test_PUT_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_etag test.unit.obj.test_server.TestObjectController.test_PUT_no_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite test.unit.obj.test_server.TestObjectController.test_PUT_overwrite ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_slow test.unit.obj.test_server.TestObjectController.test_PUT_slow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_works test.unit.obj.test_server.TestObjectController.test_REPLICATE_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test__call__returns_500 test.unit.obj.test_server.TestObjectController.test__call__returns_500 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_allowed_headers test.unit.obj.test_server.TestObjectController.test_allowed_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_http_connect test.unit.obj.test_server.TestObjectController.test_async_update_http_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_bad_sinces test.unit.obj.test_server.TestObjectController.test_bad_sinces ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_method test.unit.obj.test_server.TestObjectController.test_call_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_request test.unit.obj.test_server.TestObjectController.test_call_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_name_collision test.unit.obj.test_server.TestObjectController.test_call_name_collision ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_not_found test.unit.obj.test_server.TestObjectController.test_call_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_put test.unit.obj.test_server.TestObjectController.test_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_async test.unit.obj.test_server.TestObjectController.test_container_update_async ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_bad_args test.unit.obj.test_server.TestObjectController.test_container_update_bad_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_success test.unit.obj.test_server.TestObjectController.test_container_update_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_content_encoding test.unit.obj.test_server.TestObjectController.test_content_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_correct_allowed_method test.unit.obj.test_server.TestObjectController.test_correct_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_cap test.unit.obj.test_server.TestObjectController.test_delete_at_cap ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_negative test.unit.obj.test_server.TestObjectController.test_delete_at_negative ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_backend_replication test.unit.obj.test_server.TestObjectController.test_delete_backend_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_device_unavailable test.unit.obj.test_server.TestObjectController.test_device_unavailable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_dynamic_datadir test.unit.obj.test_server.TestObjectController.test_dynamic_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_init test.unit.obj.test_server.TestObjectController.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_list_allowed_methods test.unit.obj.test_server.TestObjectController.test_list_allowed_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_log_line_format test.unit.obj.test_server.TestObjectController.test_log_line_format ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_object_name_length test.unit.obj.test_server.TestObjectController.test_max_object_name_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_upload_time test.unit.obj.test_server.TestObjectController.test_max_upload_time ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_allowed_method test.unit.obj.test_server.TestObjectController.test_not_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_update_with_offset test.unit.obj.test_server.TestObjectController.test_object_update_with_offset ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_serv_reserv test.unit.obj.test_server.TestObjectController.test_serv_reserv ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_short_body test.unit.obj.test_server.TestObjectController.test_short_body ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim ... ok test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0033 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0121 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0150 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0003 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0128 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0175 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0006 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0009 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0184 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0246 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0168 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0144 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0150 "-" 3543406 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0281 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0148 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3543406 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3543406 0 test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0010 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0011 "-" 3543406 - test INFO: - - - [02/Jun/2024:16:31:38 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0108 "-" 3543406 0 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: tx6c78c2f4eea84affa85a6-00665c9e6c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6c4bea63695444ffa39ab-00665c9e6c) test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347104.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/709/46ada3d38d1b5700cb8b462353b49709/46ada3d38d1b5700cb8b462353b49709.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347109.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/825/63bf95f274bba71ac69579c67f6ea825/63bf95f274bba71ac69579c67f6ea825.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347114.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/08f/8309caf6bda4cca4e37ed7eb666d808f/8309caf6bda4cca4e37ed7eb666d808f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347119.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/913/030b7dfd628027e3781ad76bf812a913/030b7dfd628027e3781ad76bf812a913.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347124.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/028/243a7e4c651c9a998b5bd7a4fa785028/243a7e4c651c9a998b5bd7a4fa785028.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347129.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/533/7283f7414810f63a6f42b931404a7533/7283f7414810f63a6f42b931404a7533.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347134.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/1d6/3a758f18ec6ef249621c87fa57f491d6/3a758f18ec6ef249621c87fa57f491d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347139.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/d7b/9aacaa5bc1170d16aaf92e9913295d7b/9aacaa5bc1170d16aaf92e9913295d7b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347144.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/9f5/d63dbd9dd994944ce0e026d382d1e9f5/d63dbd9dd994944ce0e026d382d1e9f5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347149.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/04b/51db650d98a5bde40efa37ef81c0204b/51db650d98a5bde40efa37ef81c0204b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347154.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/85d/ad9ecb4095b340274fb6e3d1369d585d/ad9ecb4095b340274fb6e3d1369d585d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347159.00000-0, db: /tmp/tmp2w209gah/sda/containers/0/481/0d6ea938cc5061abbb9eeeae7e13b481/0d6ea938cc5061abbb9eeeae7e13b481.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/e55/c5a6893da312f5919cffa3f3dae8be55/c5a6893da312f5919cffa3f3dae8be55.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345898.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/e55/c5a6893da312f5919cffa3f3dae8be55/c5a6893da312f5919cffa3f3dae8be55_1717345898.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/410/b831e1521e42362aaf60eafad1dce410/b831e1521e42362aaf60eafad1dce410.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345902.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/410/b831e1521e42362aaf60eafad1dce410/b831e1521e42362aaf60eafad1dce410_1717345902.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/4d3/76b441cf2b0607691c2cb984068864d3/76b441cf2b0607691c2cb984068864d3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345906.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/4d3/76b441cf2b0607691c2cb984068864d3/76b441cf2b0607691c2cb984068864d3_1717345906.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f_1717345910.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421_1717345914.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/744/5228a20266373daa2c036003e3975744/5228a20266373daa2c036003e3975744.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345918.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/744/5228a20266373daa2c036003e3975744/5228a20266373daa2c036003e3975744_1717345918.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/362/c51b3ea38593efd802b7d34c1b4ac362/c51b3ea38593efd802b7d34c1b4ac362.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345922.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/362/c51b3ea38593efd802b7d34c1b4ac362/c51b3ea38593efd802b7d34c1b4ac362_1717345922.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/1ab/bb86b1d8f3e41d16afaa2b1b5ba911ab/bb86b1d8f3e41d16afaa2b1b5ba911ab.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345926.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/1ab/bb86b1d8f3e41d16afaa2b1b5ba911ab/bb86b1d8f3e41d16afaa2b1b5ba911ab_1717345926.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/dcb/9ce78ad7519f77b225ce82fec74badcb/9ce78ad7519f77b225ce82fec74badcb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345930.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/dcb/9ce78ad7519f77b225ce82fec74badcb/9ce78ad7519f77b225ce82fec74badcb_1717345930.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1_1717345934.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/b2d/9cc05f1466be45fbde1e64f1b8fb9b2d/9cc05f1466be45fbde1e64f1b8fb9b2d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345938.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/b2d/9cc05f1466be45fbde1e64f1b8fb9b2d/9cc05f1466be45fbde1e64f1b8fb9b2d_1717345938.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/7cc/b63fa752ff9a489a89dbace6c7ece7cc/b63fa752ff9a489a89dbace6c7ece7cc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345942.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/7cc/b63fa752ff9a489a89dbace6c7ece7cc/b63fa752ff9a489a89dbace6c7ece7cc_1717345942.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345946.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e_1717345946.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/b69/b7866328e9ff41c34dfaa43b12317b69/b7866328e9ff41c34dfaa43b12317b69.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345950.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/b69/b7866328e9ff41c34dfaa43b12317b69/b7866328e9ff41c34dfaa43b12317b69_1717345950.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a73/682f3d8315d3c4aa31c9eb70cc5d1a73/682f3d8315d3c4aa31c9eb70cc5d1a73.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345954.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a73/682f3d8315d3c4aa31c9eb70cc5d1a73/682f3d8315d3c4aa31c9eb70cc5d1a73_1717345954.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/c54/415e73f2320e2c700eb8695f42618c54/415e73f2320e2c700eb8695f42618c54.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345958.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/c54/415e73f2320e2c700eb8695f42618c54/415e73f2320e2c700eb8695f42618c54_1717345958.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ed3/7016679a2b3f47cf3c66666d5f6fbed3/7016679a2b3f47cf3c66666d5f6fbed3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345962.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ed3/7016679a2b3f47cf3c66666d5f6fbed3/7016679a2b3f47cf3c66666d5f6fbed3_1717345962.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/0ed/16a1ae8285942162f69d710c5a57d0ed/16a1ae8285942162f69d710c5a57d0ed.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345966.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/0ed/16a1ae8285942162f69d710c5a57d0ed/16a1ae8285942162f69d710c5a57d0ed_1717345966.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345970.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff_1717345970.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/469/6cc61909e7a11534cf9facccb2cd6469/6cc61909e7a11534cf9facccb2cd6469.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345974.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/469/6cc61909e7a11534cf9facccb2cd6469/6cc61909e7a11534cf9facccb2cd6469_1717345974.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/0d0/88664c6307a3b1506a7c730a5abe90d0/88664c6307a3b1506a7c730a5abe90d0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345978.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/0d0/88664c6307a3b1506a7c730a5abe90d0/88664c6307a3b1506a7c730a5abe90d0_1717345978.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/3cf/f7244f4b19b8781af8dbbc8ecc80c3cf/f7244f4b19b8781af8dbbc8ecc80c3cf.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345982.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/3cf/f7244f4b19b8781af8dbbc8ecc80c3cf/f7244f4b19b8781af8dbbc8ecc80c3cf_1717345982.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a86/cee9b0a96eb2f4124cf61e01ea590a86/cee9b0a96eb2f4124cf61e01ea590a86.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345986.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a86/cee9b0a96eb2f4124cf61e01ea590a86/cee9b0a96eb2f4124cf61e01ea590a86_1717345986.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/fe6/fb2e86a1afea266cd93a4b1c0dc36fe6/fb2e86a1afea266cd93a4b1c0dc36fe6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345990.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/fe6/fb2e86a1afea266cd93a4b1c0dc36fe6/fb2e86a1afea266cd93a4b1c0dc36fe6_1717345990.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a_1717345994.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/513/41db166df269eddb0664009737651513/41db166df269eddb0664009737651513.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345998.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/513/41db166df269eddb0664009737651513/41db166df269eddb0664009737651513_1717345998.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/d94/f80bb0aca97a231dcdd46c9cea69cd94/f80bb0aca97a231dcdd46c9cea69cd94.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346002.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/d94/f80bb0aca97a231dcdd46c9cea69cd94/f80bb0aca97a231dcdd46c9cea69cd94_1717346002.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346006.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1_1717346006.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/fc6/1398ecba34e0d89c184c83f06fa4cfc6/1398ecba34e0d89c184c83f06fa4cfc6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346010.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/fc6/1398ecba34e0d89c184c83f06fa4cfc6/1398ecba34e0d89c184c83f06fa4cfc6_1717346010.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a32/e62cfd993cf033cf47f6d26df631ca32/e62cfd993cf033cf47f6d26df631ca32.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346014.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a32/e62cfd993cf033cf47f6d26df631ca32/e62cfd993cf033cf47f6d26df631ca32_1717346014.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/d60/b7458e039a58a35603e0d761c87e4d60/b7458e039a58a35603e0d761c87e4d60.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346018.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/d60/b7458e039a58a35603e0d761c87e4d60/b7458e039a58a35603e0d761c87e4d60_1717346018.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/8c3/487d1c4493d129889b37ec9e80ed18c3/487d1c4493d129889b37ec9e80ed18c3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346022.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/8c3/487d1c4493d129889b37ec9e80ed18c3/487d1c4493d129889b37ec9e80ed18c3_1717346022.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/d65/61bbd97b4675786fb5dc5d4a248b5d65/61bbd97b4675786fb5dc5d4a248b5d65.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346026.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/d65/61bbd97b4675786fb5dc5d4a248b5d65/61bbd97b4675786fb5dc5d4a248b5d65_1717346026.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346030.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27_1717346030.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/e6b/1e17050f206eb050635baa94217f8e6b/1e17050f206eb050635baa94217f8e6b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346034.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/e6b/1e17050f206eb050635baa94217f8e6b/1e17050f206eb050635baa94217f8e6b_1717346034.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/027/13a861a3dc41fa59cb81511580862027/13a861a3dc41fa59cb81511580862027.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346038.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/027/13a861a3dc41fa59cb81511580862027/13a861a3dc41fa59cb81511580862027_1717346038.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/636/3e34021029a00eb6bde27abaad41b636/3e34021029a00eb6bde27abaad41b636.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346042.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/636/3e34021029a00eb6bde27abaad41b636/3e34021029a00eb6bde27abaad41b636_1717346042.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/bc9/464bccfc4534158b53a5055e3ed2bbc9/464bccfc4534158b53a5055e3ed2bbc9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346046.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/bc9/464bccfc4534158b53a5055e3ed2bbc9/464bccfc4534158b53a5055e3ed2bbc9_1717346046.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/808/ee2bfada3eb7a1f85192f40d7761d808/ee2bfada3eb7a1f85192f40d7761d808.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346050.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/808/ee2bfada3eb7a1f85192f40d7761d808/ee2bfada3eb7a1f85192f40d7761d808_1717346050.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a65/827b1676df1f920fb9a53e5a1cae0a65/827b1676df1f920fb9a53e5a1cae0a65.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346054.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a65/827b1676df1f920fb9a53e5a1cae0a65/827b1676df1f920fb9a53e5a1cae0a65_1717346054.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/be9/62752d4ba2bb31412f5696797cc68be9/62752d4ba2bb31412f5696797cc68be9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346058.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/be9/62752d4ba2bb31412f5696797cc68be9/62752d4ba2bb31412f5696797cc68be9_1717346058.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/447/29d0238b1284aba340bcb63ba83e7447/29d0238b1284aba340bcb63ba83e7447.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346062.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/447/29d0238b1284aba340bcb63ba83e7447/29d0238b1284aba340bcb63ba83e7447_1717346062.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/a06/360bb3da57fc722726b9bd5af568ca06/360bb3da57fc722726b9bd5af568ca06.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346066.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/a06/360bb3da57fc722726b9bd5af568ca06/360bb3da57fc722726b9bd5af568ca06_1717346066.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/964/ad49306915196ae164a2ce9596ae5964/ad49306915196ae164a2ce9596ae5964.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346070.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/964/ad49306915196ae164a2ce9596ae5964/ad49306915196ae164a2ce9596ae5964_1717346070.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/53b/217ce7a7d09e39777860c6633c30c53b/217ce7a7d09e39777860c6633c30c53b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346074.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/53b/217ce7a7d09e39777860c6633c30c53b/217ce7a7d09e39777860c6633c30c53b_1717346074.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/824/c4afd444a06ffa070776ebd64ccb4824/c4afd444a06ffa070776ebd64ccb4824.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346078.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/824/c4afd444a06ffa070776ebd64ccb4824/c4afd444a06ffa070776ebd64ccb4824_1717346078.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/aa1/9702b5957a6f857007cd46bda4d77aa1/9702b5957a6f857007cd46bda4d77aa1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346082.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/aa1/9702b5957a6f857007cd46bda4d77aa1/9702b5957a6f857007cd46bda4d77aa1_1717346082.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/39e/a4116ac5ad831759a84e45854ccab39e/a4116ac5ad831759a84e45854ccab39e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346086.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/39e/a4116ac5ad831759a84e45854ccab39e/a4116ac5ad831759a84e45854ccab39e_1717346086.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/2d6/2babc59ee60f61da2b53859777c9d2d6/2babc59ee60f61da2b53859777c9d2d6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346090.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/2d6/2babc59ee60f61da2b53859777c9d2d6/2babc59ee60f61da2b53859777c9d2d6_1717346090.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/08c/6a2e5f7dcf475cf183f4a9516e0af08c/6a2e5f7dcf475cf183f4a9516e0af08c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346094.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/08c/6a2e5f7dcf475cf183f4a9516e0af08c/6a2e5f7dcf475cf183f4a9516e0af08c_1717346094.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/91f/d29cc2097689e561a8da29321e36d91f/d29cc2097689e561a8da29321e36d91f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346098.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/91f/d29cc2097689e561a8da29321e36d91f/d29cc2097689e561a8da29321e36d91f_1717346098.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/b85/69a5481116999cd9335ae97f8c265b85/69a5481116999cd9335ae97f8c265b85.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346102.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/b85/69a5481116999cd9335ae97f8c265b85/69a5481116999cd9335ae97f8c265b85_1717346102.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/50f/b1bd653a5e34d2410fddc16d8237750f/b1bd653a5e34d2410fddc16d8237750f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346106.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/50f/b1bd653a5e34d2410fddc16d8237750f/b1bd653a5e34d2410fddc16d8237750f_1717346106.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/52c/2f8e06f657ff80348fb8b63ce18b952c/2f8e06f657ff80348fb8b63ce18b952c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346110.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/52c/2f8e06f657ff80348fb8b63ce18b952c/2f8e06f657ff80348fb8b63ce18b952c_1717346110.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/b11/04c52fcff46616dce478b3ff8e7e5b11/04c52fcff46616dce478b3ff8e7e5b11.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346114.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/b11/04c52fcff46616dce478b3ff8e7e5b11/04c52fcff46616dce478b3ff8e7e5b11_1717346114.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/172/24bb66a37f99c710d5bbf0a7fc39a172/24bb66a37f99c710d5bbf0a7fc39a172.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346118.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/172/24bb66a37f99c710d5bbf0a7fc39a172/24bb66a37f99c710d5bbf0a7fc39a172_1717346118.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ed3/2c38ab17d7acca48e6ec090e0e856ed3/2c38ab17d7acca48e6ec090e0e856ed3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346122.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ed3/2c38ab17d7acca48e6ec090e0e856ed3/2c38ab17d7acca48e6ec090e0e856ed3_1717346122.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/769/2187a1fe3961d0458f4abbd003ca2769/2187a1fe3961d0458f4abbd003ca2769.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346126.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/769/2187a1fe3961d0458f4abbd003ca2769/2187a1fe3961d0458f4abbd003ca2769_1717346126.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/94f/18c0253baf08d423e0ff1fe1d7b7b94f/18c0253baf08d423e0ff1fe1d7b7b94f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346130.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/94f/18c0253baf08d423e0ff1fe1d7b7b94f/18c0253baf08d423e0ff1fe1d7b7b94f_1717346130.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/502/32756134775a8993c1c258262cbde502/32756134775a8993c1c258262cbde502.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346134.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/502/32756134775a8993c1c258262cbde502/32756134775a8993c1c258262cbde502_1717346134.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ee8/0df7551e6366d1617fc5b3184d15bee8/0df7551e6366d1617fc5b3184d15bee8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346138.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ee8/0df7551e6366d1617fc5b3184d15bee8/0df7551e6366d1617fc5b3184d15bee8_1717346138.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ada/eda0f8d04faec9e2232d6e03bfbfeada/eda0f8d04faec9e2232d6e03bfbfeada.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346142.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ada/eda0f8d04faec9e2232d6e03bfbfeada/eda0f8d04faec9e2232d6e03bfbfeada_1717346142.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/8c1/1a78e058c420d84bb0d1c1f79c9f68c1/1a78e058c420d84bb0d1c1f79c9f68c1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346146.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/8c1/1a78e058c420d84bb0d1c1f79c9f68c1/1a78e058c420d84bb0d1c1f79c9f68c1_1717346146.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/966/f38081baf352eef21faa24dc3430d966/f38081baf352eef21faa24dc3430d966.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346150.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/966/f38081baf352eef21faa24dc3430d966/f38081baf352eef21faa24dc3430d966_1717346150.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/002/346e0e57d1918d2e1822b2019bcb7002/346e0e57d1918d2e1822b2019bcb7002.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346154.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/002/346e0e57d1918d2e1822b2019bcb7002/346e0e57d1918d2e1822b2019bcb7002_1717346154.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/f73/00ff22dc4f7c6fd64a978384fbcf2f73/00ff22dc4f7c6fd64a978384fbcf2f73.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346158.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/f73/00ff22dc4f7c6fd64a978384fbcf2f73/00ff22dc4f7c6fd64a978384fbcf2f73_1717346158.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/f14/410f6215dce5888dbfa4d52556e3ef14/410f6215dce5888dbfa4d52556e3ef14.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346162.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/f14/410f6215dce5888dbfa4d52556e3ef14/410f6215dce5888dbfa4d52556e3ef14_1717346162.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/bd1/600e394b20bdcdedbdc68b4f08907bd1/600e394b20bdcdedbdc68b4f08907bd1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346166.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/bd1/600e394b20bdcdedbdc68b4f08907bd1/600e394b20bdcdedbdc68b4f08907bd1_1717346166.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/1b4/3d34f444098dfb7fc5ba03a75e7b11b4/3d34f444098dfb7fc5ba03a75e7b11b4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346170.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/1b4/3d34f444098dfb7fc5ba03a75e7b11b4/3d34f444098dfb7fc5ba03a75e7b11b4_1717346170.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ff7/b255d085c3c53ca26177165958e1eff7/b255d085c3c53ca26177165958e1eff7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346174.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ff7/b255d085c3c53ca26177165958e1eff7/b255d085c3c53ca26177165958e1eff7_1717346174.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/67e/570826ad7bf22ff09e3fe4ab03ee467e/570826ad7bf22ff09e3fe4ab03ee467e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346178.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/67e/570826ad7bf22ff09e3fe4ab03ee467e/570826ad7bf22ff09e3fe4ab03ee467e_1717346178.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/2ac/c04192285846c0c76535d1738aedd2ac/c04192285846c0c76535d1738aedd2ac.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346182.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/2ac/c04192285846c0c76535d1738aedd2ac/c04192285846c0c76535d1738aedd2ac_1717346182.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/e2a/84e3cd4133b87fb2996bba0177f99e2a/84e3cd4133b87fb2996bba0177f99e2a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346186.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/e2a/84e3cd4133b87fb2996bba0177f99e2a/84e3cd4133b87fb2996bba0177f99e2a_1717346186.00000.db test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6c4bea63695444ffa39ab-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6c4bea63695444ffa39ab-00665c9e6c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: txc434274740a543b5b99a1-00665c9e6c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: txff7aca5ef84e46f9961da-00665c9e6c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: txffe8f51cc7704ff09cc7e-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: tx24e19497e59b466890a11-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx14650178fae8450aba43c-00665c9e6c) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx1743e0289b744b20bc690-00665c9e6c) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx1743e0289b744b20bc690-00665c9e6c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: tx4869aaf8fbce480a9c8ae-00665c9e6c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx238481adb7a54ab0a0293-00665c9e6c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: txcdbcbad928d946c48b610-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: txfbf0eb2f773a4c2f98133-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx90327f1c5f76439590d6f-00665c9e6c) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txf58aa03f0203459e94095-00665c9e6c) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txf58aa03f0203459e94095-00665c9e6c) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: txd88872a5f04243878e7a0-00665c9e6c) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txd695d8b96bdd407f80ce4-00665c9e6c) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx86b620b2152e456a8fdea-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: txc5def95d7bfa4036ba47b-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx32546eb350614394a6403-00665c9e6c) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txfcc89c7b4d394cdd82f67-00665c9e6c) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txfcc89c7b4d394cdd82f67-00665c9e6c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx5bba09af4bb24ef28b451-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: txf7a3664690f6473e8d6a5-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: tx2f541cc218724e62b850c-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: tx2f541cc218724e62b850c-00665c9e6c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST ... ok test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (201, 404, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (201, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: Container GET returning 503 for [] (txn: txf5963e9e725849d981542-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx131a912d81674189b4861-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx131a912d81674189b4861-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx131a912d81674189b4861-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: tx131a912d81674189b4861-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx131a912d81674189b4861-00665c9e6c) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx131a912d81674189b4861-00665c9e6c) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx131a912d81674189b4861-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: tx7dea0ffbb11749ba9f3bd-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: tx2a684097b8ec4e47bda44-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6b52698082ff4e339b92a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6b52698082ff4e339b92a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6b52698082ff4e339b92a-00665c9e6c) test ERROR: Container GET returning 503 for [] (txn: tx6b52698082ff4e339b92a-00665c9e6c) test INFO: Client disconnected on read of '/a/c' (txn: tx6b52698082ff4e339b92a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: Container GET returning 503 for [] (txn: tx58da4fd78d6d4697b694a-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txea90b3cb52fe4d239c264-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txea90b3cb52fe4d239c264-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txea90b3cb52fe4d239c264-00665c9e6c) test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txea90b3cb52fe4d239c264-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txea90b3cb52fe4d239c264-00665c9e6c) test ERROR: Container GET returning 503 for [] (txn: txea90b3cb52fe4d239c264-00665c9e6c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx69c467d3c3704f2ab5d7a-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx69c467d3c3704f2ab5d7a-00665c9e6d) test ERROR: Container GET returning 503 for [] (txn: tx69c467d3c3704f2ab5d7a-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx0468f4ae8e874ca5a8195-00665c9e6d) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txb757e3b8743649cfa6c0e-00665c9e6d) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txb757e3b8743649cfa6c0e-00665c9e6d) test ERROR: Container GET returning 503 for [500, 500] (txn: txb757e3b8743649cfa6c0e-00665c9e6d) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx95809e8b86a94defb596c-00665c9e6d) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx5c9fc1c7d10d428c83941-00665c9e6d) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx5c9fc1c7d10d428c83941-00665c9e6d) test INFO: Client disconnected on read of '/a/c' (txn: tx5c9fc1c7d10d428c83941-00665c9e6d) test ERROR: Container GET returning 503 for [] (txn: tx7757a647ebc7449999fca-00665c9e6d) test INFO: Client disconnected on read of '/a/c' (txn: tx7757a647ebc7449999fca-00665c9e6d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx705760cc09b544b89050a-00665c9e6d) test INFO: Client disconnected on read of '/a/c' (txn: tx705760cc09b544b89050a-00665c9e6d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3dc0f7488ac948348b3e3-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3dc0f7488ac948348b3e3-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3dc0f7488ac948348b3e3-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3dc0f7488ac948348b3e3-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3dc0f7488ac948348b3e3-00665c9e6d) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: tx3dc0f7488ac948348b3e3-00665c9e6d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5baa5211c9af4d26a3ae4-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5baa5211c9af4d26a3ae4-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5baa5211c9af4d26a3ae4-00665c9e6d) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5baa5211c9af4d26a3ae4-00665c9e6d) test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5baa5211c9af4d26a3ae4-00665c9e6d) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: tx5baa5211c9af4d26a3ae4-00665c9e6d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/664/7bb972b7aa8f6b89ebe77bce1da1d664/7bb972b7aa8f6b89ebe77bce1da1d664.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346190.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/664/7bb972b7aa8f6b89ebe77bce1da1d664/7bb972b7aa8f6b89ebe77bce1da1d664_1717346190.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/227/e62f94e3aab5b55c2f28961381a11227/e62f94e3aab5b55c2f28961381a11227.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346194.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/227/e62f94e3aab5b55c2f28961381a11227/e62f94e3aab5b55c2f28961381a11227_1717346194.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/970/907bf26bcb73702514b77feec9478970/907bf26bcb73702514b77feec9478970.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346198.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/970/907bf26bcb73702514b77feec9478970/907bf26bcb73702514b77feec9478970_1717346198.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/249/3edbb7295ca373f0f103f746aa0b5249/3edbb7295ca373f0f103f746aa0b5249.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346202.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/249/3edbb7295ca373f0f103f746aa0b5249/3edbb7295ca373f0f103f746aa0b5249_1717346202.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/d2a/106d56997456c3dd3f975a987d81fd2a/106d56997456c3dd3f975a987d81fd2a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346206.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/d2a/106d56997456c3dd3f975a987d81fd2a/106d56997456c3dd3f975a987d81fd2a_1717346206.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/702/47002958df5e7c1c0bc3887477304702/47002958df5e7c1c0bc3887477304702.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346210.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/702/47002958df5e7c1c0bc3887477304702/47002958df5e7c1c0bc3887477304702_1717346210.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/797/14b81786bac9a5b4b378afc86aed2797/14b81786bac9a5b4b378afc86aed2797.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346214.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/797/14b81786bac9a5b4b378afc86aed2797/14b81786bac9a5b4b378afc86aed2797_1717346214.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/1b9/899afe8d4a38bb57ef5ba19b9effc1b9/899afe8d4a38bb57ef5ba19b9effc1b9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346218.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/1b9/899afe8d4a38bb57ef5ba19b9effc1b9/899afe8d4a38bb57ef5ba19b9effc1b9_1717346218.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/dbb/a48132b14744f12ec0fe57ffaec8edbb/a48132b14744f12ec0fe57ffaec8edbb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346222.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/dbb/a48132b14744f12ec0fe57ffaec8edbb/a48132b14744f12ec0fe57ffaec8edbb_1717346222.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/6f0/93d43dc8e3588b15d457097319f806f0/93d43dc8e3588b15d457097319f806f0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346226.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/6f0/93d43dc8e3588b15d457097319f806f0/93d43dc8e3588b15d457097319f806f0_1717346226.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/94e/5e9ba0d5fb502ea4f09b37cd076b594e/5e9ba0d5fb502ea4f09b37cd076b594e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346230.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/94e/5e9ba0d5fb502ea4f09b37cd076b594e/5e9ba0d5fb502ea4f09b37cd076b594e_1717346230.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/864/37a2a969437115a0d1fc0ad5d86ab864/37a2a969437115a0d1fc0ad5d86ab864.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346234.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/864/37a2a969437115a0d1fc0ad5d86ab864/37a2a969437115a0d1fc0ad5d86ab864_1717346234.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/aeb/06cafd63a227881de38fec3255054aeb/06cafd63a227881de38fec3255054aeb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346238.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/aeb/06cafd63a227881de38fec3255054aeb/06cafd63a227881de38fec3255054aeb_1717346238.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/b6d/b8d3befbcbe9cca33eb5ac5f27016b6d/b8d3befbcbe9cca33eb5ac5f27016b6d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346242.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/b6d/b8d3befbcbe9cca33eb5ac5f27016b6d/b8d3befbcbe9cca33eb5ac5f27016b6d_1717346242.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/cbf/7f1d65cd683541cc459900d81d656cbf/7f1d65cd683541cc459900d81d656cbf.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346246.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/cbf/7f1d65cd683541cc459900d81d656cbf/7f1d65cd683541cc459900d81d656cbf_1717346246.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/20c/91b038233a330dcf4f1522200f3db20c/91b038233a330dcf4f1522200f3db20c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346250.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/20c/91b038233a330dcf4f1522200f3db20c/91b038233a330dcf4f1522200f3db20c_1717346250.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/d59/c02e67e3e106e65d589856186663ed59/c02e67e3e106e65d589856186663ed59.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346254.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/d59/c02e67e3e106e65d589856186663ed59/c02e67e3e106e65d589856186663ed59_1717346254.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/2e4/3efb559739f8f9ba8fe4a60af50bf2e4/3efb559739f8f9ba8fe4a60af50bf2e4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346258.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/2e4/3efb559739f8f9ba8fe4a60af50bf2e4/3efb559739f8f9ba8fe4a60af50bf2e4_1717346258.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/08d/656a067398ff9d15689ad08e6d22008d/656a067398ff9d15689ad08e6d22008d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346262.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/08d/656a067398ff9d15689ad08e6d22008d/656a067398ff9d15689ad08e6d22008d_1717346262.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/83e/c2fafcbae74c4e6d71c8ab55948fd83e/c2fafcbae74c4e6d71c8ab55948fd83e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346266.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/83e/c2fafcbae74c4e6d71c8ab55948fd83e/c2fafcbae74c4e6d71c8ab55948fd83e_1717346266.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/f51/b7c39c43afdf43f85956fe0fffe15f51/b7c39c43afdf43f85956fe0fffe15f51.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346270.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/f51/b7c39c43afdf43f85956fe0fffe15f51/b7c39c43afdf43f85956fe0fffe15f51_1717346270.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/692/20b8a3dba44653e379ef9e7b2e925692/20b8a3dba44653e379ef9e7b2e925692.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346274.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/692/20b8a3dba44653e379ef9e7b2e925692/20b8a3dba44653e379ef9e7b2e925692_1717346274.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/606/67ef67cde8b80d18015e5e6d72cc7606/67ef67cde8b80d18015e5e6d72cc7606.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346278.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/606/67ef67cde8b80d18015e5e6d72cc7606/67ef67cde8b80d18015e5e6d72cc7606_1717346278.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/e32/b806a55c0535afab72048de8e9716e32/b806a55c0535afab72048de8e9716e32.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346282.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/e32/b806a55c0535afab72048de8e9716e32/b806a55c0535afab72048de8e9716e32_1717346282.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/f0e/26c0facd6eabc8bd101888f90b4d9f0e/26c0facd6eabc8bd101888f90b4d9f0e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346286.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/f0e/26c0facd6eabc8bd101888f90b4d9f0e/26c0facd6eabc8bd101888f90b4d9f0e_1717346286.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/8fd/1a78061b3d4578dc9775fc95cb3e78fd/1a78061b3d4578dc9775fc95cb3e78fd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346290.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/8fd/1a78061b3d4578dc9775fc95cb3e78fd/1a78061b3d4578dc9775fc95cb3e78fd_1717346290.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/412/02df473d450e56b4fc7180e9347eb412/02df473d450e56b4fc7180e9347eb412.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346294.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/412/02df473d450e56b4fc7180e9347eb412/02df473d450e56b4fc7180e9347eb412_1717346294.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/fb8/3ce4be17123507777643f57d327b5fb8/3ce4be17123507777643f57d327b5fb8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346298.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/fb8/3ce4be17123507777643f57d327b5fb8/3ce4be17123507777643f57d327b5fb8_1717346298.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/9b1/6279cebec47f7b36a716fb7e837439b1/6279cebec47f7b36a716fb7e837439b1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346302.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/9b1/6279cebec47f7b36a716fb7e837439b1/6279cebec47f7b36a716fb7e837439b1_1717346302.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/056/79e6ece54fe99aee2cdc814f77acb056/79e6ece54fe99aee2cdc814f77acb056.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346306.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/056/79e6ece54fe99aee2cdc814f77acb056/79e6ece54fe99aee2cdc814f77acb056_1717346306.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/665/c337fa4b276e71ddf99d5aae7675a665/c337fa4b276e71ddf99d5aae7675a665.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346310.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/665/c337fa4b276e71ddf99d5aae7675a665/c337fa4b276e71ddf99d5aae7675a665_1717346310.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/16d/82f1c2b3d47799b02aa83266c2ace16d/82f1c2b3d47799b02aa83266c2ace16d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346314.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/16d/82f1c2b3d47799b02aa83266c2ace16d/82f1c2b3d47799b02aa83266c2ace16d_1717346314.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/ca0/8f4b43c1d4d491ef1daa853908976ca0/8f4b43c1d4d491ef1daa853908976ca0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346318.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/ca0/8f4b43c1d4d491ef1daa853908976ca0/8f4b43c1d4d491ef1daa853908976ca0_1717346318.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/3f9/af99f20e416e968f34d436247d86b3f9/af99f20e416e968f34d436247d86b3f9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346322.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/3f9/af99f20e416e968f34d436247d86b3f9/af99f20e416e968f34d436247d86b3f9_1717346322.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/2ba/76247e7ed045249e780718660bd0e2ba/76247e7ed045249e780718660bd0e2ba.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346326.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/2ba/76247e7ed045249e780718660bd0e2ba/76247e7ed045249e780718660bd0e2ba_1717346326.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/432/67bbd46eb70c3baacddf990dff41c432/67bbd46eb70c3baacddf990dff41c432.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346330.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/432/67bbd46eb70c3baacddf990dff41c432/67bbd46eb70c3baacddf990dff41c432_1717346330.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/9eb/666078c758a3a7c0d411e438abd529eb/666078c758a3a7c0d411e438abd529eb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346334.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/9eb/666078c758a3a7c0d411e438abd529eb/666078c758a3a7c0d411e438abd529eb_1717346334.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/f68/1345e36394b3aa4a55064c0c0af23f68/1345e36394b3aa4a55064c0c0af23f68.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346338.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/f68/1345e36394b3aa4a55064c0c0af23f68/1345e36394b3aa4a55064c0c0af23f68_1717346338.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/c17/0e6796afa6f1379ed34a452ed6e2ec17/0e6796afa6f1379ed34a452ed6e2ec17.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346342.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/c17/0e6796afa6f1379ed34a452ed6e2ec17/0e6796afa6f1379ed34a452ed6e2ec17_1717346342.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/10d/7cca4a284896061c8a16f239e20a410d/7cca4a284896061c8a16f239e20a410d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346346.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/10d/7cca4a284896061c8a16f239e20a410d/7cca4a284896061c8a16f239e20a410d_1717346346.00000.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 ... ok sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/3e8/fb00ad2a5c768c6f36dbbf64a94e03e8/fb00ad2a5c768c6f36dbbf64a94e03e8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346350.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/3e8/fb00ad2a5c768c6f36dbbf64a94e03e8/fb00ad2a5c768c6f36dbbf64a94e03e8_1717346350.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/925/83216fbb83b0cda56b09d0d97203d925/83216fbb83b0cda56b09d0d97203d925.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346354.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/925/83216fbb83b0cda56b09d0d97203d925/83216fbb83b0cda56b09d0d97203d925_1717346354.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/159/110b2129c5d2677c21cad81423341159/110b2129c5d2677c21cad81423341159.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346358.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/159/110b2129c5d2677c21cad81423341159/110b2129c5d2677c21cad81423341159_1717346358.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/7b6/bdd2fbb58b0301a77606fd5693c1f7b6/bdd2fbb58b0301a77606fd5693c1f7b6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346362.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/7b6/bdd2fbb58b0301a77606fd5693c1f7b6/bdd2fbb58b0301a77606fd5693c1f7b6_1717346362.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/5c7/3b4bd8cb2c5d304b8deef63e73f475c7/3b4bd8cb2c5d304b8deef63e73f475c7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346366.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/5c7/3b4bd8cb2c5d304b8deef63e73f475c7/3b4bd8cb2c5d304b8deef63e73f475c7_1717346366.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/28c/7455cccd7e89b7b673f6e3a86f2b628c/7455cccd7e89b7b673f6e3a86f2b628c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346370.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/28c/7455cccd7e89b7b673f6e3a86f2b628c/7455cccd7e89b7b673f6e3a86f2b628c_1717346370.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/b0c/6a0fabe4b3328900eeab1ff0c9904b0c/6a0fabe4b3328900eeab1ff0c9904b0c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346374.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/b0c/6a0fabe4b3328900eeab1ff0c9904b0c/6a0fabe4b3328900eeab1ff0c9904b0c_1717346374.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/cee/c29fe90330fcfad2cabe43c51b470cee/c29fe90330fcfad2cabe43c51b470cee.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346378.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/cee/c29fe90330fcfad2cabe43c51b470cee/c29fe90330fcfad2cabe43c51b470cee_1717346378.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/17d/3d91e9bd5182be5d626ed0de35a6d17d/3d91e9bd5182be5d626ed0de35a6d17d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346382.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/17d/3d91e9bd5182be5d626ed0de35a6d17d/3d91e9bd5182be5d626ed0de35a6d17d_1717346382.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/289/530cb418b56977d647009b51c91cb289/530cb418b56977d647009b51c91cb289.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346386.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/289/530cb418b56977d647009b51c91cb289/530cb418b56977d647009b51c91cb289_1717346386.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/1be/a02b8014a5c6e0407678fa2d72fce1be/a02b8014a5c6e0407678fa2d72fce1be.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346390.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/1be/a02b8014a5c6e0407678fa2d72fce1be/a02b8014a5c6e0407678fa2d72fce1be_1717346390.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/73d/e904eb5e9cfd954f9aac001f0643f73d/e904eb5e9cfd954f9aac001f0643f73d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346394.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/73d/e904eb5e9cfd954f9aac001f0643f73d/e904eb5e9cfd954f9aac001f0643f73d_1717346394.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/8f2/ee6b716d2972ffbf2a96b717584f48f2/ee6b716d2972ffbf2a96b717584f48f2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346398.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/8f2/ee6b716d2972ffbf2a96b717584f48f2/ee6b716d2972ffbf2a96b717584f48f2_1717346398.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/4c3/b41df006b43bf29b6416af3ab0c224c3/b41df006b43bf29b6416af3ab0c224c3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346402.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/4c3/b41df006b43bf29b6416af3ab0c224c3/b41df006b43bf29b6416af3ab0c224c3_1717346402.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp9cz79kme/sda/containers/0/2ea/798534ad35629b1723ccea6dc11312ea/798534ad35629b1723ccea6dc11312ea.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346406.00000-0, db: /tmp/tmp9cz79kme/sda/containers/0/2ea/798534ad35629b1723ccea6dc11312ea/798534ad35629b1723ccea6dc11312ea_1717346406.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345907.00000-0, db: /tmp/tmpzbnjaqu_/sda/containers/0/226/8145a26873826017b40f0ae4fbbc0226/8145a26873826017b40f0ae4fbbc0226.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpzbnjaqu_/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345915.00000-0, db: /tmp/tmpzbnjaqu_/sda/containers/0/d41/740cb5d220733796133a1347f1910d41/740cb5d220733796133a1347f1910d41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpzbnjaqu_/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmprxpc0y09/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmprxpc0y09/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmprxpc0y09/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmprxpc0y09/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345912.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/1bb/1322d2f31c3849007194e95bed8a71bb/1322d2f31c3849007194e95bed8a71bb.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345912.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/1bb/1322d2f31c3849007194e95bed8a71bb/1322d2f31c3849007194e95bed8a71bb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345912.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/1bb/1322d2f31c3849007194e95bed8a71bb/1322d2f31c3849007194e95bed8a71bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345920.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/deb/12ecad9a67c2a93914ed09a180d18deb/12ecad9a67c2a93914ed09a180d18deb.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345920.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/deb/12ecad9a67c2a93914ed09a180d18deb/12ecad9a67c2a93914ed09a180d18deb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345920.00000-0, db: /tmp/tmp6a70_x15/sda/containers/0/deb/12ecad9a67c2a93914ed09a180d18deb/12ecad9a67c2a93914ed09a180d18deb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmpcv496h_2/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345911.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/fe0/24bc79dddc1ff24f83ed2491c0d5ffe0/24bc79dddc1ff24f83ed2491c0d5ffe0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345917.00000-0, db: /tmp/tmpufq8kjz0/sda/containers/0/2e3/d5d9a91077fe0a4620b0a5ce77f982e3/d5d9a91077fe0a4620b0a5ce77f982e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345908.00000-0, db: /tmp/tmpa9xjy558/sda/containers/0/39a/cf7cc2c4de2e780b407e90c33bbcc39a/cf7cc2c4de2e780b407e90c33bbcc39a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmpa9xjy558/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345912.00000-0, db: /tmp/tmpa9xjy558/sda/containers/0/1bb/1322d2f31c3849007194e95bed8a71bb/1322d2f31c3849007194e95bed8a71bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345914.00000-0, db: /tmp/tmpa9xjy558/sda/containers/0/421/a1e6f849f40814830dc89aa909ca7421/a1e6f849f40814830dc89aa909ca7421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345910.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c7f/aed1a2dcb0ae2271d04422c047eabc7f/aed1a2dcb0ae2271d04422c047eabc7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345913.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/27c/5bd94edf03635d01a64c72b6003bd27c/5bd94edf03635d01a64c72b6003bd27c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345913.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/27c/5bd94edf03635d01a64c72b6003bd27c/5bd94edf03635d01a64c72b6003bd27c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345916.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ca5/e640742f16d32613acf86fe88bb2fca5/e640742f16d32613acf86fe88bb2fca5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345919.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/0a9/c416db506af47319f4513348412460a9/c416db506af47319f4513348412460a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345922.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/362/c51b3ea38593efd802b7d34c1b4ac362/c51b3ea38593efd802b7d34c1b4ac362.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345922.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/362/c51b3ea38593efd802b7d34c1b4ac362/c51b3ea38593efd802b7d34c1b4ac362.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345925.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3/2d69e6b96ed080b0dcfbdb76fa12a2f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345928.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/e2c/7f0a6a311e0b90bcc0faf44286ae8e2c/7f0a6a311e0b90bcc0faf44286ae8e2c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345928.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/e2c/7f0a6a311e0b90bcc0faf44286ae8e2c/7f0a6a311e0b90bcc0faf44286ae8e2c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345931.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/f47/3efc1e1e6838789c861ddf2e93c96f47/3efc1e1e6838789c861ddf2e93c96f47.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345934.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/8a1/3d50c812399be5d9559869bd734d68a1/3d50c812399be5d9559869bd734d68a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345937.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/8c8/620d98adac4bdd426cc1adf5940c98c8/620d98adac4bdd426cc1adf5940c98c8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345937.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/8c8/620d98adac4bdd426cc1adf5940c98c8/620d98adac4bdd426cc1adf5940c98c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345940.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/622/d60128a68626c160d5792a9a75f6f622/d60128a68626c160d5792a9a75f6f622.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345940.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/622/d60128a68626c160d5792a9a75f6f622/d60128a68626c160d5792a9a75f6f622.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345943.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/8cc/0651a4eaccab13f1aa20b1c2679528cc/0651a4eaccab13f1aa20b1c2679528cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345946.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345946.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a6e/6d58ed6c56f265dfde35d34680657a6e/6d58ed6c56f265dfde35d34680657a6e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345949.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/0eb/b187d4a5ed535e2646359a6f6f2fb0eb/b187d4a5ed535e2646359a6f6f2fb0eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345952.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/654/a2e440c8ff3caf61ef956495bd356654/a2e440c8ff3caf61ef956495bd356654.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345952.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/654/a2e440c8ff3caf61ef956495bd356654/a2e440c8ff3caf61ef956495bd356654.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345955.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/029/ece6354d2142a2c7aa08101c35521029/ece6354d2142a2c7aa08101c35521029.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345958.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c54/415e73f2320e2c700eb8695f42618c54/415e73f2320e2c700eb8695f42618c54.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345958.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c54/415e73f2320e2c700eb8695f42618c54/415e73f2320e2c700eb8695f42618c54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345961.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/839/81eadae9ee9a406c85bdd7e907bca839/81eadae9ee9a406c85bdd7e907bca839.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345964.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c67/cd176eaa90f4d4ac9133cc9d19a15c67/cd176eaa90f4d4ac9133cc9d19a15c67.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345964.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c67/cd176eaa90f4d4ac9133cc9d19a15c67/cd176eaa90f4d4ac9133cc9d19a15c67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345967.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/bd2/775679db67b325352066cff0a73edbd2/775679db67b325352066cff0a73edbd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345970.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345970.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/5ff/9ee55e2f11f39252422714ed28c645ff/9ee55e2f11f39252422714ed28c645ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345973.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/6cc/e5f758aec496ce42916476fb4ea9e6cc/e5f758aec496ce42916476fb4ea9e6cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345973.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/6cc/e5f758aec496ce42916476fb4ea9e6cc/e5f758aec496ce42916476fb4ea9e6cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345976.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ef6/379172803f41a0e92f620996a6740ef6/379172803f41a0e92f620996a6740ef6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345976.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ef6/379172803f41a0e92f620996a6740ef6/379172803f41a0e92f620996a6740ef6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345979.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c9b/e48cf167e75188a60d7d5318dc44cc9b/e48cf167e75188a60d7d5318dc44cc9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345982.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/3cf/f7244f4b19b8781af8dbbc8ecc80c3cf/f7244f4b19b8781af8dbbc8ecc80c3cf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345982.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/3cf/f7244f4b19b8781af8dbbc8ecc80c3cf/f7244f4b19b8781af8dbbc8ecc80c3cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345985.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c1d/0df858d5632d7474a3205cfec3520c1d/0df858d5632d7474a3205cfec3520c1d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345988.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/b97/89ca8b65b5c62704a623432d03fb6b97/89ca8b65b5c62704a623432d03fb6b97.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345988.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/b97/89ca8b65b5c62704a623432d03fb6b97/89ca8b65b5c62704a623432d03fb6b97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345991.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/d29/656dc90165850f3945509ca86f6b0d29/656dc90165850f3945509ca86f6b0d29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345994.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/56a/460638c1df643be637bd78e1d9eee56a/460638c1df643be637bd78e1d9eee56a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345997.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a53/231ffb44cd41be0edc6ed40d4a1dfa53/231ffb44cd41be0edc6ed40d4a1dfa53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717345997.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a53/231ffb44cd41be0edc6ed40d4a1dfa53/231ffb44cd41be0edc6ed40d4a1dfa53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346000.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/6fe/497c628f729b1bbb60870990f0bad6fe/497c628f729b1bbb60870990f0bad6fe.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346000.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/6fe/497c628f729b1bbb60870990f0bad6fe/497c628f729b1bbb60870990f0bad6fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346003.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/4b0/5d526ea8292511279f07a55b014e34b0/5d526ea8292511279f07a55b014e34b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346006.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346006.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ad1/efa101e5de563f07cb09ebb8b7576ad1/efa101e5de563f07cb09ebb8b7576ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346009.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/ebc/46e50e0cde4a0730605c66688d288ebc/46e50e0cde4a0730605c66688d288ebc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346012.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/708/e0212a17aebc68331e7badab59a1c708/e0212a17aebc68331e7badab59a1c708.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346012.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/708/e0212a17aebc68331e7badab59a1c708/e0212a17aebc68331e7badab59a1c708.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346015.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/488/889c43d49a669fa0b958ca901d615488/889c43d49a669fa0b958ca901d615488.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346018.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/d60/b7458e039a58a35603e0d761c87e4d60/b7458e039a58a35603e0d761c87e4d60.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346018.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/d60/b7458e039a58a35603e0d761c87e4d60/b7458e039a58a35603e0d761c87e4d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346021.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/563/0698a1949c100b1940f09567349d8563/0698a1949c100b1940f09567349d8563.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346024.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/974/7f371af32892408d278c8d2b4e2ad974/7f371af32892408d278c8d2b4e2ad974.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346024.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/974/7f371af32892408d278c8d2b4e2ad974/7f371af32892408d278c8d2b4e2ad974.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346027.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/364/9738d39eaa71d198b094798be3017364/9738d39eaa71d198b094798be3017364.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346030.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346030.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a27/5d9324c94d6fa893f0e2c13d631dea27/5d9324c94d6fa893f0e2c13d631dea27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346033.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/dd1/84f12721bef5ffa94e25c6863a1badd1/84f12721bef5ffa94e25c6863a1badd1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346033.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/dd1/84f12721bef5ffa94e25c6863a1badd1/84f12721bef5ffa94e25c6863a1badd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346036.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/271/8f38eee399580620c5ed9c4937aa7271/8f38eee399580620c5ed9c4937aa7271.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346036.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/271/8f38eee399580620c5ed9c4937aa7271/8f38eee399580620c5ed9c4937aa7271.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346039.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/2a9/59082f505045acc3f8a98e85ef7a22a9/59082f505045acc3f8a98e85ef7a22a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346042.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/636/3e34021029a00eb6bde27abaad41b636/3e34021029a00eb6bde27abaad41b636.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346042.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/636/3e34021029a00eb6bde27abaad41b636/3e34021029a00eb6bde27abaad41b636.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346045.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/094/970bcee06085d9e70038e2bd329fd094/970bcee06085d9e70038e2bd329fd094.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346045.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/094/970bcee06085d9e70038e2bd329fd094/970bcee06085d9e70038e2bd329fd094.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346048.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/16c/1881cd80f33d1cdf1660aa958a79916c/1881cd80f33d1cdf1660aa958a79916c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346048.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/16c/1881cd80f33d1cdf1660aa958a79916c/1881cd80f33d1cdf1660aa958a79916c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346051.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346051.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/630/5523d69032ca63c7db0a616b91112630/5523d69032ca63c7db0a616b91112630.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346054.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a65/827b1676df1f920fb9a53e5a1cae0a65/827b1676df1f920fb9a53e5a1cae0a65.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346054.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a65/827b1676df1f920fb9a53e5a1cae0a65/827b1676df1f920fb9a53e5a1cae0a65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346057.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/414/286f8483218e828516df1fb016ec3414/286f8483218e828516df1fb016ec3414.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346057.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/414/286f8483218e828516df1fb016ec3414/286f8483218e828516df1fb016ec3414.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_check_node test.unit.container.test_sharder.TestSharder.test_check_node ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346060.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/505/7886bb9d108c06b6395eca78f6ea1505/7886bb9d108c06b6395eca78f6ea1505.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346060.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/505/7886bb9d108c06b6395eca78f6ea1505/7886bb9d108c06b6395eca78f6ea1505.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346063.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c7a/87b0db4a3a68a6cad2f717ee23cacc7a/87b0db4a3a68a6cad2f717ee23cacc7a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346063.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c7a/87b0db4a3a68a6cad2f717ee23cacc7a/87b0db4a3a68a6cad2f717ee23cacc7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346066.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a06/360bb3da57fc722726b9bd5af568ca06/360bb3da57fc722726b9bd5af568ca06.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346066.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/a06/360bb3da57fc722726b9bd5af568ca06/360bb3da57fc722726b9bd5af568ca06.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346069.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/6f3/d75465b42e5dc3b12c47edd8623656f3/d75465b42e5dc3b12c47edd8623656f3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346069.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/6f3/d75465b42e5dc3b12c47edd8623656f3/d75465b42e5dc3b12c47edd8623656f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346072.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c0b/cc0d374aa0843d286cf60ea8abb45c0b/cc0d374aa0843d286cf60ea8abb45c0b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346072.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/c0b/cc0d374aa0843d286cf60ea8abb45c0b/cc0d374aa0843d286cf60ea8abb45c0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346075.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/256/8c28cad3e3414d4c0358c374c3b65256/8c28cad3e3414d4c0358c374c3b65256.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346075.00000-0, db: /tmp/tmpmo6inx3u/sda/containers/0/256/8c28cad3e3414d4c0358c374c3b65256/8c28cad3e3414d4c0358c374c3b65256.db sharder-test WARNING: Skipping d100 as it is not mounted sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002911090850830078s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001480579376220703s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_0 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpxwgddk7l/sda/containers/0/b85/9d7a328814f4e99b3c0e6a0597883b85/9d7a328814f4e99b3c0e6a0597883b85.db, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication ... ok sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Cleaved ShardRange in 0.008s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002853870391845703s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014448165893554688s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpxwgddk7l/sda/containers/0/069/5a8214a697fa9e61b996df73488a7069/5a8214a697fa9e61b996df73488a7069.db, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: misplaced object source bounds [('', 'obj004')], path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001876354217529297s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00012230873107910156s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000213623046875s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002760887145996094s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001442432403564453s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345673.56112, (5, 5) as of 1717345907.35735, cleaved as of 1717345673.56112> in 0.026s, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpxwgddk7l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.02192.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002593994140625s, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00016570091247558594s, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test INFO: Cleaved ShardRange in 0.032s, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002455711364746094s, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00018310546875s, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717345673.56112, (1, 6144) as of 1717345907.60937, cleaved as of 1717345673.56112> in 0.031s, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpbn0fhmny/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.42427.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002899169921875s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014519691467285156s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaved ShardRange in 0.034s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: got 6 rows (deleted=False) in 0.0002911090850830078s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015163421630859375s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges ... ok test.unit.container.test_sharder.TestSharder.test_cleave_repeated test.unit.container.test_sharder.TestSharder.test_cleave_repeated ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names ... ok sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345673.56112, (6, 6) as of 1717345907.89368, cleaved as of 1717345673.56112> in 0.025s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '7e74820f-9e3d-4cb2-a8ab-dc614efa6ee5-sda', 'cursor': '', 'max_row': 12, 'cleave_to_row': 10, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Completed cleaving, DB remaining in sharding state, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaving from row 10 into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025272369384765625s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015473365783691406s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaved ShardRange in 0.022s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaving from row 10 into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345673.56112, (6, 6) as of 1717345907.89368, cleaved as of 1717345673.56112>, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000255584716796875s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014495849609375s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345673.56112, (6, 6) as of 1717345907.89368, cleaved as of 1717345673.56112> in 0.016s, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpsi8vg45a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345907.69184.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync ... ok sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002636909484863281s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015807151794433594s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaved ShardRange in 0.024s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, found as of 1717345673.56112>, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00027298927307128906s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015354156494140625s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmptig87olg/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - shard db already in sync, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717345673.56112, (3, 6) as of 1717345908.42888, cleaved as of 1717345673.56112> in 0.009s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025153160095214844s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014781951904296875s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717345673.56112, (1, 100) as of 1717345908.49519, cleaved as of 1717345673.56112> in 0.03s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025844573974609375s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001614093780517578s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_root test.unit.container.test_sharder.TestSharder.test_cleave_root ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff ... ok sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717345673.56112, (1, 1000) as of 1717345908.58495, cleaved as of 1717345673.56112> in 0.028s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717345673.56112, (2, 15) as of 1717345908.62721, active as of 1717345673.56112>, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002574920654296875s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016450881958007812s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717345673.56112, (2, 15) as of 1717345908.62721, active as of 1717345673.56112> in 0.022s, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmptig87olg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.11792.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002598762512207031s, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013828277587890625s, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test INFO: Cleaved ShardRange in 0.019s, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpq6cuccgp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.73419.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023055076599121094s, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001327991485595703s, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states ... ok sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023436546325683594s, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001373291015625s, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'d' to 'x' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - zero objects found, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'x' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00028705596923828125s, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001392364501953125s, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'x' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - zero objects found, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpabntwedd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345908.89195.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002636909484863281s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018596649169921875s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010776519775390625s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001513957977294922s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaved ShardRange in 0.032s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, found as of 1717345673.56112>, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002675056457519531s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001270771026611328s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001399517059326172s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00011181831359863281s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016260147094726562s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file ... ok sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpx_udidfl/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - shard db already in sync, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717345673.56112, (3, 6) as of 1717345909.49781, cleaved as of 1717345673.56112> in 0.011s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002601146697998047s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010609626770019531s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014543533325195312s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717345673.56112, (1, 100) as of 1717345909.56500, cleaved as of 1717345673.56112> in 0.026s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025844573974609375s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.393692016601562e-05s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017309188842773438s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.821487426757812e-05s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717345673.56112, (1, 1000) as of 1717345909.65904, cleaved as of 1717345673.56112> in 0.039s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717345673.56112, (2, 15) as of 1717345909.70853, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002644062042236328s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00011014938354492188s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001475811004638672s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one ... ok test ERROR: Problem finding shard upper in '/tmp/tmp1xup0ya7/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpdn4hwpqe/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpdn4hwpqe/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change ... ok sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717345673.56112, (2, 15) as of 1717345909.70853, active as of 1717345673.56112> in 0.021s, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmpx_udidfl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.16826.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002906322479248047s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001697540283203125s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Cleaved ShardRange in 0.026s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717345673.56112, (500000, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00026702880859375s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018715858459472656s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_x_1 for ShardRange, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Cleaved ShardRange<'d' to 'x' as of 1717345673.56112, (2, 103) as of 1717345910.01099, cleaved as of 1717345673.56112> in 0.027s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: Continuing to cleave (2 done, 1 todo), path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'d' to MaxBound as of 1717345910.05648, (500000, 0) as of 1717345910.05648, created as of 1717345910.05648>, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002675056457519531s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017714500427246094s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__2 for ShardRange, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test INFO: Cleaved ShardRange<'d' to MaxBound as of 1717345910.05648, (3, 1103) as of 1717345910.12008, cleaved as of 1717345910.05648> in 0.032s, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp6kwes5v7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345909.81892.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024890899658203125s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00011563301086425781s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017309188842773438s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/c_here_0 in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpqdd1vmkl/sda/containers/0/e19/9e88c3bdfd03a830d1fc09d35d671e19/9e88c3bdfd03a830d1fc09d35d671e19.db, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015115737915039062s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00015163421630859375s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 8.678436279296875e-05s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00010085105895996094s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_0 for ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002779960632324219s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017642974853515625s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_there_0 for ShardRange<'here' to 'where' as of 1717345910.15942, (0, 0) as of 1717345910.15942, sharding as of 1717345910.15942>, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717345673.56112, (3, 6) as of 1717345910.38696, cleaved as of 1717345673.56112> in 0.03s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002453327178955078s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 9.560585021972656e-05s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001666545867919922s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000171661376953125s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013256072998046875s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_1 for ShardRange<'there' to 'where' as of 1717345673.56112, (2, 15) as of 1717345910.42055, cleaved as of 1717345673.56112>, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002617835998535156s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000179290771484375s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_where_1 for ShardRange<'here' to 'where' as of 1717345910.15942, (0, 0) as of 1717345910.15942, sharding as of 1717345910.15942>, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db test.unit.container.test_sharder.TestSharder.test_cleave_shard test.unit.container.test_sharder.TestSharder.test_cleave_shard ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors ... ok sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717345673.56112, (2, 15) as of 1717345910.42055, cleaved as of 1717345673.56112> in 0.019s, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpqdd1vmkl/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345910.15944.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpy9b772i1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022220611572265625s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012922286987304688s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001308917999267578s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_1_ for ShardRange<'h' to MaxBound as of 1717345910.71594, (1000, 11111) as of 1717345910.71594, created as of 1717345910.71594>, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003044605255126953s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001659393310546875s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_1_ for ShardRange<'h' to 'w' as of 1717345933.00000, (0, 0) as of 1717345922.00000, shrunk as of 1717345922.00000>, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717345910.71594, (1000, 11111) as of 1717345910.71594, created as of 1717345910.71594> in 0.028s, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_1, db: /tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717345923.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022029876708984375s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013065338134765625s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013637542724609375s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013208389282226562s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_2_ for ShardRange<'h' to MaxBound as of 1717345910.89891, (1000, 11111) as of 1717345910.89891, cleaved as of 1717345910.89891>, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00026869773864746094s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001571178436279297s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted ... ok sharder-test INFO: Replicating new shard container .shards_a/acceptor_2_ for ShardRange<'h' to 'w' as of 1717345960.00000, (0, 0) as of 1717345949.00000, shrunk as of 1717345949.00000>, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717345910.89891, (1000, 11111) as of 1717345910.89891, cleaved as of 1717345910.89891> in 0.029s, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_2, db: /tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717345950.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002846717834472656s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016021728515625s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013446807861328125s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013208389282226562s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_3_ for ShardRange<'h' to MaxBound as of 1717345911.17688, (1000, 11111) as of 1717345911.17688, active as of 1717345911.17688>, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002875328063964844s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001499652862548828s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_3_ for ShardRange<'h' to 'w' as of 1717345987.00000, (0, 0) as of 1717345976.00000, shrunk as of 1717345976.00000>, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717345911.17688, (1000, 11111) as of 1717345911.17688, active as of 1717345911.17688> in 0.03s, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_3, db: /tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717345977.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00027298927307128906s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012731552124023438s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012922286987304688s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001289844512939453s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_k for ShardRange<'d' to 'k' as of 1717345911.35774, (1000, 11111) as of 1717345911.35774, created as of 1717345911.35774>, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025844573974609375s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017309188842773438s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_k for ShardRange<'h' to 'w' as of 1717346003.00000, (0, 0) as of 1717346003.00000, shrinking as of 1717346003.00000>, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345911.35774, (1000, 11111) as of 1717345911.35774, created as of 1717345911.35774> in 0.043s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs ... ok sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_ for ShardRange<'k' to MaxBound as of 1717345911.35781, (1000, 11111) as of 1717345911.35781, created as of 1717345911.35781>, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00026702880859375s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013780593872070312s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_ for ShardRange<'h' to 'w' as of 1717346003.00000, (0, 0) as of 1717346003.00000, shrinking as of 1717346003.00000>, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717345911.35781, (1000, 11111) as of 1717345911.35781, created as of 1717345911.35781> in 0.029s, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_4, db: /tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717346004.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025653839111328125s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013136863708496094s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012969970703125s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013327598571777344s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_k for ShardRange<'d' to 'k' as of 1717345911.63284, (1000, 11111) as of 1717345911.63284, cleaved as of 1717345911.63284>, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002593994140625s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001513957977294922s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_k for ShardRange<'h' to 'w' as of 1717346036.00000, (0, 0) as of 1717346036.00000, shrinking as of 1717346036.00000>, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345911.63284, (1000, 11111) as of 1717345911.63284, cleaved as of 1717345911.63284> in 0.019s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_ for ShardRange<'k' to MaxBound as of 1717345911.63290, (1000, 11111) as of 1717345911.63290, cleaved as of 1717345911.63290>, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00026869773864746094s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014543533325195312s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_ for ShardRange<'h' to 'w' as of 1717346036.00000, (0, 0) as of 1717346036.00000, shrinking as of 1717346036.00000>, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717345911.63290, (1000, 11111) as of 1717345911.63290, cleaved as of 1717345911.63290> in 0.017s, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_5, db: /tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717346037.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002193450927734375s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001289844512939453s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012922286987304688s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required ... ok sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001251697540283203s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_k for ShardRange<'d' to 'k' as of 1717345911.90073, (1000, 11111) as of 1717345911.90073, active as of 1717345911.90073>, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000263214111328125s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014829635620117188s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_k for ShardRange<'h' to 'w' as of 1717346069.00000, (0, 0) as of 1717346069.00000, shrinking as of 1717346069.00000>, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345911.90073, (1000, 11111) as of 1717345911.90073, active as of 1717345911.90073> in 0.018s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_ for ShardRange<'k' to MaxBound as of 1717345911.90078, (1000, 11111) as of 1717345911.90078, active as of 1717345911.90078>, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002560615539550781s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000164031982421875s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_ for ShardRange<'h' to 'w' as of 1717346069.00000, (0, 0) as of 1717346069.00000, shrinking as of 1717346069.00000>, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717345911.90078, (1000, 11111) as of 1717345911.90078, active as of 1717345911.90078> in 0.019s, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_6, db: /tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717346070.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002777576446533203s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013256072998046875s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013208389282226562s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014257431030273438s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_k for ShardRange<'d' to 'k' as of 1717345912.14304, (1000, 11111) as of 1717345912.14304, cleaved as of 1717345912.14304>, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003008842468261719s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001633167266845703s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_k for ShardRange<'h' to 'w' as of 1717346102.00000, (0, 0) as of 1717346102.00000, shrinking as of 1717346102.00000>, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345912.14304, (1000, 11111) as of 1717345912.14304, cleaved as of 1717345912.14304> in 0.02s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_t for ShardRange<'k' to 't' as of 1717345912.14310, (1000, 11111) as of 1717345912.14310, cleaved as of 1717345912.14310>, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002524852752685547s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001652240753173828s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta ... ok sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_t for ShardRange<'h' to 'w' as of 1717346102.00000, (0, 0) as of 1717346102.00000, shrinking as of 1717346102.00000>, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345912.14310, (1000, 11111) as of 1717345912.14310, cleaved as of 1717345912.14310> in 0.022s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_ for ShardRange<'t' to MaxBound as of 1717345912.14313, (1000, 11111) as of 1717345912.14313, cleaved as of 1717345912.14313>, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024390220642089844s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016069412231445312s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_ for ShardRange<'h' to 'w' as of 1717346102.00000, (0, 0) as of 1717346102.00000, shrinking as of 1717346102.00000>, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717345912.14313, (1000, 11111) as of 1717345912.14313, cleaved as of 1717345912.14313> in 0.021s, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_7, db: /tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717346103.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023174285888671875s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018525123596191406s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013327598571777344s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001323223114013672s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_k for ShardRange<'d' to 'k' as of 1717345912.43172, (1000, 11111) as of 1717345912.43172, created as of 1717345912.43172>, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002720355987548828s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001518726348876953s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_k for ShardRange<'h' to 'w' as of 1717346141.00000, (0, 0) as of 1717346141.00000, shrinking as of 1717346141.00000>, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345912.43172, (1000, 11111) as of 1717345912.43172, created as of 1717345912.43172> in 0.018s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_t for ShardRange<'k' to 't' as of 1717345912.43178, (1000, 11111) as of 1717345912.43178, created as of 1717345912.43178>, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002486705780029297s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001513957977294922s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_t for ShardRange<'h' to 'w' as of 1717346141.00000, (0, 0) as of 1717346141.00000, shrinking as of 1717346141.00000>, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345912.43178, (1000, 11111) as of 1717345912.43178, created as of 1717345912.43178> in 0.021s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_ for ShardRange<'t' to MaxBound as of 1717345912.43180, (1000, 11111) as of 1717345912.43180, created as of 1717345912.43180>, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002474784851074219s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015616416931152344s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_close_error test.unit.obj.test_diskfile.TestECDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit test.unit.obj.test_diskfile.TestECDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_delete test.unit.obj.test_diskfile.TestECDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments test.unit.obj.test_diskfile.TestECDiskFile.test_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults ... ok sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_ for ShardRange<'h' to 'w' as of 1717346141.00000, (0, 0) as of 1717346141.00000, shrinking as of 1717346141.00000>, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717345912.43180, (1000, 11111) as of 1717345912.43180, created as of 1717345912.43180> in 0.022s, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_8, db: /tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717346142.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002124309539794922s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012755393981933594s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012969970703125s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_k for ShardRange<'d' to 'k' as of 1717345912.76947, (1000, 11111) as of 1717345912.76947, active as of 1717345912.76947>, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002505779266357422s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001456737518310547s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_k for ShardRange<'h' to 'w' as of 1717346180.00000, (0, 0) as of 1717346180.00000, shrinking as of 1717346180.00000>, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345912.76947, (1000, 11111) as of 1717345912.76947, active as of 1717345912.76947> in 0.017s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_t for ShardRange<'k' to 't' as of 1717345912.76953, (1000, 11111) as of 1717345912.76953, active as of 1717345912.76953>, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024199485778808594s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015163421630859375s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_t for ShardRange<'h' to 'w' as of 1717346180.00000, (0, 0) as of 1717346180.00000, shrinking as of 1717346180.00000>, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345912.76953, (1000, 11111) as of 1717345912.76953, active as of 1717345912.76953> in 0.02s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_ for ShardRange<'t' to MaxBound as of 1717345912.76955, (1000, 11111) as of 1717345912.76955, active as of 1717345912.76955>, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024437904357910156s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013875961303710938s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_ for ShardRange<'h' to 'w' as of 1717346180.00000, (0, 0) as of 1717346180.00000, shrinking as of 1717346180.00000>, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717345912.76955, (1000, 11111) as of 1717345912.76955, active as of 1717345912.76955> in 0.022s, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_9, db: /tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717346181.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpwq8i1g85/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021839141845703125s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013136863708496094s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012946128845214844s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_10_k for ShardRange<'d' to 'k' as of 1717345913.06359, (1000, 11111) as of 1717345913.06359, created as of 1717345913.06359>, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024366378784179688s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014519691467285156s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_10_k for ShardRange<'h' to 'w' as of 1717346219.00000, (0, 0) as of 1717346219.00000, shrinking as of 1717346219.00000>, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345913.06359, (1000, 11111) as of 1717345913.06359, created as of 1717345913.06359> in 0.017s, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'd972519f-baea-4f21-9bd4-0f57e0c3ce14-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 0}, path: .shards_a/donor_10, db: /tmp/tmpwq8i1g85/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717346220.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002570152282714844s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001285076141357422s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012969970703125s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_k for ShardRange<'d' to 'k' as of 1717345913.23198, (1000, 11111) as of 1717345913.23198, cleaved as of 1717345913.23198>, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002551078796386719s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001468658447265625s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_k for ShardRange<'h' to 'w' as of 1717346245.00000, (0, 0) as of 1717346245.00000, shrinking as of 1717346245.00000>, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345913.23198, (1000, 11111) as of 1717345913.23198, cleaved as of 1717345913.23198> in 0.019s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_t for ShardRange<'k' to 't' as of 1717345913.23202, (1000, 11111) as of 1717345913.23202, cleaved as of 1717345913.23202>, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002551078796386719s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015807151794433594s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_t for ShardRange<'h' to 'w' as of 1717346245.00000, (0, 0) as of 1717346245.00000, shrinking as of 1717346245.00000>, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717345913.23202, (1000, 11111) as of 1717345913.23202, cleaved as of 1717345913.23202> in 0.02s, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '63de5a6e-633c-4d00-81b3-353c1b7e6d04-sda', 'cursor': 't', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: .shards_a/donor_11, db: /tmp/tmpwq8i1g85/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717346246.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021791458129882812s, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016760826110839844s, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014519691467285156s, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012826919555664062s, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e6bac801-832b-4745-84a2-d6ea5e86153f-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 1}, path: .shards_a/donor_12, db: /tmp/tmpwq8i1g85/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717346278.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000225067138671875s, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012803077697753906s, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013017654418945312s, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012826919555664062s, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0e04c3c4-460c-41cb-b24d-f8428900ce58-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 2}, path: .shards_a/donor_13, db: /tmp/tmpwq8i1g85/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717346298.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002377033233642578s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001270771026611328s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001316070556640625s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013065338134765625s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking ... ok test ERROR: Failed to unlink '/tmp/tmpbqpf75kb/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpbqpf75kb/containers/part/suffix/hash/container_1717345911.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpbqpf75kb/containers/part/suffix/hash/container_1717345911.00000.db': ['/tmp/tmpbqpf75kb/containers/part/suffix/hash/container.db', '/tmp/tmpbqpf75kb/containers/part/suffix/hash/container_1717345912.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpbqpf75kb/containers/part/suffix/hash/container_1717345912.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp3mj7upgj/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp3mj7upgj/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp3mj7upgj/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp3mj7upgj/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0120 "-" 3543414 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0113 "-" 3543414 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0116 "-" 3543414 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0090 "-" 3543414 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf4e7fa0879b44c9e95c7a-00665c9e79" "proxy-server 3543414" 0.0017 "-" 3543414 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:50451/sdb1/1/a/c" "txf4e7fa0879b44c9e95c7a-00665c9e79" "container-server 3543414" 0.0015 "-" 3543414 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:52739/sda1/1/a/c" "txf4e7fa0879b44c9e95c7a-00665c9e79" "container-server 3543414" 0.0014 "-" 3543414 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf4e7fa0879b44c9e95c7a-00665c9e79" "proxy-server 3543414" 0.0298 "-" 3543414 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf4e7fa0879b44c9e95c7a-00665c9e79" "proxy-server 3543414" 0.0192 "-" 3543414 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/53 PUT /v1/a/c HTTP/1.0 201 - - t - - - txf4e7fa0879b44c9e95c7a-00665c9e79 - 0.0432 - - 1717345913.386833906 1717345913.430045366 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txbebcdc74554446c7a758d-00665c9e79" "proxy-server 3543414" 0.0017 "-" 3543414 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:52739/sda1/1/a1/c1" "txbebcdc74554446c7a758d-00665c9e79" "container-server 3543414" 0.0021 "-" 3543414 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:50451/sdb1/1/a1/c1" "txbebcdc74554446c7a758d-00665c9e79" "container-server 3543414" 0.0013 "-" 3543414 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbebcdc74554446c7a758d-00665c9e79" "proxy-server 3543414" 0.0209 "-" 3543414 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbebcdc74554446c7a758d-00665c9e79" "proxy-server 3543414" 0.0358 "-" 3543414 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/53 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txbebcdc74554446c7a758d-00665c9e79 - 0.0463 - - 1717345913.431997538 1717345913.478268385 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf81793e9cd7444d9b7a50-00665c9e79" "proxy-server 3543414" 0.0078 "-" 3543414 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:50451/sdb1/2/a/c1" "txf81793e9cd7444d9b7a50-00665c9e79" "container-server 3543414" 0.0016 "-" 3543414 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:52739/sda1/2/a/c1" "txf81793e9cd7444d9b7a50-00665c9e79" "container-server 3543414" 0.0066 "-" 3543414 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txf81793e9cd7444d9b7a50-00665c9e79" "proxy-server 3543414" 0.0278 "-" 3543414 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txf81793e9cd7444d9b7a50-00665c9e79" "proxy-server 3543414" 0.0421 "-" 3543414 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/53 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txf81793e9cd7444d9b7a50-00665c9e79 - 0.0597 - - 1717345913.480206251 1717345913.539949417 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txfca34fd27a824de9be746-00665c9e79" "proxy-server 3543414" 0.0092 "-" 3543414 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:52739/sda1/1/a/c2" "txfca34fd27a824de9be746-00665c9e79" "container-server 3543414" 0.0063 "-" 3543414 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:50451/sdb1/1/a/c2" "txfca34fd27a824de9be746-00665c9e79" "container-server 3543414" 0.0014 "-" 3543414 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txfca34fd27a824de9be746-00665c9e79" "proxy-server 3543414" 0.0351 "-" 3543414 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:31:53 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txfca34fd27a824de9be746-00665c9e79" "proxy-server 3543414" 0.0235 "-" 3543414 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/31/53 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txfca34fd27a824de9be746-00665c9e79 - 0.0550 - - 1717345913.541882515 1717345913.596866131 2 test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range ... ok sharder-test INFO: Cleaving from row None into .shards_a/acceptor_14_k for ShardRange<'d' to 'k' as of 1717345913.83677, (1000, 11111) as of 1717345913.83677, cleaved as of 1717345913.83677>, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026035308837890625s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001766681671142578s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_14_k for ShardRange<'h' to 'w' as of 1717346317.00000, (0, 0) as of 1717346317.00000, shrinking as of 1717346317.00000>, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717345913.83677, (1000, 11111) as of 1717345913.83677, cleaved as of 1717345913.83677> in 0.019s, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test INFO: Stopped cleave at gap: 'k' - 't', path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '384cb47b-48f4-4cc4-a11c-240155223e78-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 1}, path: .shards_a/donor_14, db: /tmp/tmpwq8i1g85/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717346318.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002307891845703125s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015807151794433594s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0021338462829589844s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013399124145507812s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test INFO: Cleaving from row None into .shards/other for ShardRange<'here' to 'where' as of 1717345916.00000, (0, 0) as of 1717345916.00000, active as of 1717345916.00000>, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002560615539550781s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001747608184814453s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards/other for ShardRange<'here' to 'there' as of 1717345914.10974, (0, 0) as of 1717345914.00000, shrunk as of 1717345914.00000>, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test INFO: Cleaved ShardRange<'here' to 'where' as of 1717345916.00000, (0, 0) as of 1717345916.00000, active as of 1717345916.00000> in 0.025s, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpd9ograk3/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022649765014648438s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012612342834472656s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013828277587890625s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001285076141357422s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test INFO: Cleaving from row None into .shards_a/shard_d for ShardRange<'here' to MaxBound as of 1717345917.00000, (0, 0) as of 1717345917.00000, active as of 1717345917.00000>, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002574920654296875s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017189979553222656s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/shard_d for ShardRange<'here' to 'there' as of 1717345915.00000, (0, 0) as of 1717345915.00000, sharding as of 1717345915.00000>, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test INFO: Cleaved ShardRange<'here' to MaxBound as of 1717345917.00000, (0, 0) as of 1717345917.00000, active as of 1717345917.00000> in 0.021s, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpmi0awvz_/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022292137145996094s, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001289844512939453s, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001285076141357422s, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000133514404296875s, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002777576446533203s, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001342296600341797s, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpt5nlimsz/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025272369384765625s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013709068298339844s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001342296600341797s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013208389282226562s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025534629821777344s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015616416931152344s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container a/c for ShardRange<'here' to 'there' as of 1717345914.57485, (0, 0) as of 1717345915.00000, shrunk as of 1717345915.00000>, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test INFO: Cleaved ShardRange in 0.019s, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp0be7_oz1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345916.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaving from row None into .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002617835998535156s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015044212341308594s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaved ShardRange in 0.031s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_2 for ShardRange<'b' to 'd' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002627372741699219s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001633167266845703s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_d_2 for ShardRange, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaved ShardRange<'b' to 'd' as of 1717345673.56112, (2, 3) as of 1717345914.75710, cleaved as of 1717345673.56112> in 0.02s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Continuing to cleave (2 done, 2 todo), path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaving from row None into .shards_a/c_f_3 for ShardRange<'d' to 'f' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00027370452880859375s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015044212341308594s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_f_3 for ShardRange, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaved ShardRange<'d' to 'f' as of 1717345673.56112, (2, 103) as of 1717345914.82330, cleaved as of 1717345673.56112> in 0.02s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Stopped cleave at unready ShardRange<'f' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, found as of 1717345673.56112>, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'f' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345914.84314>, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002448558807373047s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001647472381591797s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test INFO: Cleaved ShardRange<'f' to MaxBound as of 1717345673.56112, (1, 1000) as of 1717345914.89600, cleaved as of 1717345914.84314> in 0.024s, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpzd9v3_ej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.60309.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics ... ok sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002579689025878906s, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00016188621520996094s, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test INFO: Cleaved ShardRange in 0.027s, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024628639221191406s, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00018477439880371094s, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717345673.56112, (1, 18432) as of 1717345915.10534, cleaved as of 1717345673.56112> in 0.034s, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpsbd19q16/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345914.93710.db REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, sync, (5, '8fba474fb85b9b785d6fd66651255666', '17a550cd-e741-4e6f-8744-3186078be2a7-sdb', '1717345915.16246', '0000000000.00000', '0', '') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_items, ([{'ROWID': 1, 'name': 'obj002', 'created_at': '1717345917.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'obj003', 'created_at': '1717345918.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'obj004', 'created_at': '1717345919.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'obj005', 'created_at': '1717345920.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'obj006', 'created_at': '1717345921.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '17a550cd-e741-4e6f-8744-3186078be2a7-sdb') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_syncs, ([],) sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002808570861816406s, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001418590545654297s, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaved ShardRange in 0.025s, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002582073211669922s, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014162063598632812s, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345673.56112, (3, 3) as of 1717345915.38764, cleaved as of 1717345673.56112> in 0.04s, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpcar0j9v9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, sync, (7, '00000000000000000000000000000000', 'ee417faf-753b-4848-9025-1f0659020cf2-', '1717345915.14595', '0000000000.00000', '0', '{"X-Container-Sysmeta-Shard-Context-2f0e1f6f-d929-487e-bf38-5426e6673826-sda": ["{\\"ref\\": \\"2f0e1f6f-d929-487e-bf38-5426e6673826-sda\\", \\"cursor\\": \\"\\", \\"max_row\\": 7, \\"cleave_to_row\\": 7, \\"last_cleave_to_row\\": null, \\"cleaving_done\\": true, \\"misplaced_done\\": true, \\"ranges_done\\": 2, \\"ranges_todo\\": 0}", "1717345915.40202"]}') REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, merge_shard_ranges, ([{'name': '.shards_a/c_obj004_0', 'timestamp': '1717345673.56112', 'lower': '', 'upper': 'obj004', 'object_count': 4, 'bytes_used': 4, 'meta_timestamp': '1717345915.31755', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345673.56112', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/c__1', 'timestamp': '1717345673.56112', 'lower': 'obj004', 'upper': '', 'object_count': 3, 'bytes_used': 3, 'meta_timestamp': '1717345915.38764', 'deleted': 0, 'state': 40, 'state_timestamp': '1717345673.56112', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717345915.21261', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717345915.40955', 'deleted': 0, 'state': 70, 'state_timestamp': '1717345915.21269', 'epoch': '1717345915.21269', 'reported': 0, 'tombstones': -1}], 'ee417faf-753b-4848-9025-1f0659020cf2-') sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002448558807373047s, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001423358917236328s, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaved ShardRange in 0.016s, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717345673.56112, (3, 3) as of 1717345915.38764, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024175643920898438s, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013780593872070312s, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.dbtest.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_root test.unit.container.test_sharder.TestSharder.test_complete_sharding_root ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response ... ok sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717345673.56112, (3, 3) as of 1717345915.38764, active as of 1717345673.56112> in 0.017s, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpcar0j9v9/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.21269.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmpgvd6dj2k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e3f3ffbe-fc1e-4e4f-a09f-d07a320e7704-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpnpzoxmiq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e3f3ffbe-fc1e-4e4f-a09f-d07a320e7704-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpnpzoxmiq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e3f3ffbe-fc1e-4e4f-a09f-d07a320e7704-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpnpzoxmiq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e3f3ffbe-fc1e-4e4f-a09f-d07a320e7704-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpnpzoxmiq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'a8d43686-9d42-42a7-af9f-ec1887262ec3-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpnpzoxmiq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '5e94fbb9-fe0c-4050-9632-9a79fc34ef3d-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpv125iz2e/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '5e94fbb9-fe0c-4050-9632-9a79fc34ef3d-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpv125iz2e/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '5e94fbb9-fe0c-4050-9632-9a79fc34ef3d-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpv125iz2e/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '5e94fbb9-fe0c-4050-9632-9a79fc34ef3d-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpv125iz2e/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'd3b2ba7e-dc07-44d0-ba56-c9967a750968-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpv125iz2e/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345915.00000.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: TypeError('the JSON object must be str, bytes or bytearray, not dict'), path: a/c, db: /tmp/tmp293eqgjq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: JSONDecodeError('Expecting value: line 1 column 1 (char 0)'), path: a/c, db: /tmp/tmp293eqgjq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: ValueError('not a list'), path: a/c, db: /tmp/tmp293eqgjq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('name'), path: a/c, db: /tmp/tmp293eqgjq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('timestamp'), path: a/c, db: /tmp/tmp293eqgjq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('object_count'), path: a/c, db: /tmp/tmp293eqgjq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type None, path: a/c, db: /tmp/tmp0hgpm3kq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'object', path: a/c, db: /tmp/tmp0hgpm3kq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'disco', path: a/c, db: /tmp/tmp0hgpm3kq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get shard ranges from a/c: Unexpected response: 404, path: a/c, db: /tmp/tmpz7_612ff/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpxbkb3tqv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root ... ok sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmpxbkb3tqv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Found 2 compactible sequences of length(s) [2, 2], path: a/c, db: /tmp/tmpxbkb3tqv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpxbkb3tqv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange<'c' to 'e' as of 1717345916.22100, (100000, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpxbkb3tqv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpl0rw7jw0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpl0rw7jw0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpl0rw7jw0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpxvf8v0q3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpubzzj27m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345916.00000.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpb5e3ym82/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpb5e3ym82/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpb5e3ym82/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: a/c, db: /tmp/tmpb5e3ym82/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpb5e3ym82/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpb5e3ym82/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmplw2_7tl3/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmplw2_7tl3/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmplw2_7tl3/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: .shards_a/c_, db: /tmp/tmplw2_7tl3/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmplw2_7tl3/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmplw2_7tl3/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp83juf2a4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp83juf2a4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp83juf2a4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmp83juf2a4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmpydz0p_hn/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmpydz0p_hn/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size ... ok test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part ... ok test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate ... ok sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmpydz0p_hn/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmpydz0p_hn/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp8m380rvi/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp8m380rvi/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp8m380rvi/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp8m380rvi/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpc_hp2ga8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpc_hp2ga8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpc_hp2ga8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpc_hp2ga8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp2rgje_l9/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp2rgje_l9/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp2rgje_l9/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats test.unit.container.test_sharder.TestSharder.test_increment_stats ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd ... ok test.unit.container.test_sharder.TestSharder.test_init test.unit.container.test_sharder.TestSharder.test_init ... ok test.unit.container.test_sharder.TestSharder.test_init_deprecated_options test.unit.container.test_sharder.TestSharder.test_init_deprecated_options ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name ... ok test.unit.container.test_sharder.TestSharder.test_log_broker test.unit.container.test_sharder.TestSharder.test_log_broker ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception test.unit.container.test_sharder.TestSharder.test_log_broker_exception ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_levels test.unit.container.test_sharder.TestSharder.test_log_broker_levels ... ok sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:31:57 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test WARNING: Repeat cleaving required, context: {'ref': '6321ec8a-e632-4532-a5df-826353b70bef-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpi65u2zv_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345917.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T16:31:57.000000. DB state: sharding, own_shard_range state: sharding, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 2, 'active': 0}, path: a/c, db: /tmp/tmpi65u2zv_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345917.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T16:31:59.000000. DB state: sharding, own_shard_range state: shrinking, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 0, 'active': 1}, path: .shards_a/shard_c, db: /tmp/tmprtni7w5c/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345919.00000.db sharder-test WARNING: shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test DEBUG: bonjour mes amis, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: bonjour mes amis, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: bonjour mes amis, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: bonjour mes amis, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpfarkywkx/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: exception: test, path: a/c, db: /tmp/tmppbv_i178/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 448, in test_log_broker_exception raise ValueError('test') ValueError: test sharder-test DEBUG: bonjour mes amis, path: , db: /tmp/tmpff6p_48a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: bonjour mes amis, path: a/c, db: sharder-test INFO: bonjour mes amis, path: , db: /tmp/tmpff6p_48a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: bonjour mes amis, path: a/c, db: sharder-test WARNING: bonjour mes amis, path: , db: /tmp/tmpff6p_48a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: bonjour mes amis, path: a/c, db: sharder-test ERROR: bonjour mes amis, path: , db: /tmp/tmpff6p_48a/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: bonjour mes amis, path: a/c, db: sharder-test DEBUG: Unlinked retiring db '/tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002455711364746094s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002300739288330078s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022530555725097656s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025391578674316406s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002586841583251953s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001819133758544922s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001728534698486328s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001671314239501953s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed ... ok sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017833709716796875s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002484321594238281s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002334117889404297s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345918.35605, (0, 0) as of 1717345918.35605, active as of 1717345918.35605>, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345918.35608, (0, 0) as of 1717345918.35608, active as of 1717345918.35608>, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/there-where in state active, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345918.35610, (0, 0) as of 1717345918.35610, active as of 1717345918.35610>, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/where-yonder in state active, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018453598022460938s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016736984252929688s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016832351684570312s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016927719116210938s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017118453979492188s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020122528076171875s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017952919006347656s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016689300537109375s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017976760864257812s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018548965454101562s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018143653869628906s, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345918.35608, (0, 0) as of 1717345918.35608, active as of 1717345918.35608>, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345918.35610, (0, 0) as of 1717345918.35610, active as of 1717345918.35610>, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpbsazaa3l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.35991.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002384185791015625s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002930164337158203s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022673606872558594s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024580955505371094s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024700164794921875s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00025272369384765625s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019097328186035156s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001819133758544922s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017547607421875s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018548965454101562s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017499923706054688s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001842975616455078s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001780986785888672s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001666545867919922s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001919269561767578s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018453598022460938s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002129077911376953s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001983642578125s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018024444580078125s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017905235290527344s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016617774963378906s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001704692840576172s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019931793212890625s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001819133758544922s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001773834228515625s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016617774963378906s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016546249389648438s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019431114196777344s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018095970153808594s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017380714416503906s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001671314239501953s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001690387725830078s, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpoqzn_in6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000255584716796875s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023651123046875s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022649765014648438s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002760887145996094s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00028824806213378906s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017952919006347656s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018024444580078125s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019502639770507812s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018167495727539062s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002601146697998047s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023221969604492188s, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345918.96018, (0, 0) as of 1717345918.96018, active as of 1717345918.96018>, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345918.96021, (0, 0) as of 1717345918.96021, active as of 1717345918.96021>, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345918.96023, (0, 0) as of 1717345918.96023, active as of 1717345918.96023>, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 1 required (not removing objects), shard db: /tmp/tmp_h5_yzr5/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp_h5_yzr5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345918.96430.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002548694610595703s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023317337036132812s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024390220642089844s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002665519714355469s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002579689025878906s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019216537475585938s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018095970153808594s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016880035400390625s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017690658569335938s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024819374084472656s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022864341735839844s, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345919.12909, (0, 0) as of 1717345919.12909, active as of 1717345919.12909>, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345919.12911, (0, 0) as of 1717345919.12911, active as of 1717345919.12911>, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/there-where in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpdw9pbm6s/sda/containers/0/170/533f29a8ddb31e55151765e09ccbb170/533f29a8ddb31e55151765e09ccbb170.db, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345919.12913, (0, 0) as of 1717345919.12913, active as of 1717345919.12913>, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpdw9pbm6s/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.13275.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002491474151611328s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022912025451660156s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002200603485107422s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002498626708984375s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002799034118652344s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000209808349609375s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018024444580078125s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016880035400390625s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas ... ok sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017976760864257812s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002734661102294922s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00025582313537597656s, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345919.29804, (0, 0) as of 1717345919.29804, active as of 1717345919.29804>, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/here-there in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmp2yzejvl5/sda/containers/0/ed5/9b40bf3cdc484061806b8d0a545d2ed5/9b40bf3cdc484061806b8d0a545d2ed5.db, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345919.29806, (0, 0) as of 1717345919.29806, active as of 1717345919.29806>, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345919.29808, (0, 0) as of 1717345919.29808, active as of 1717345919.29808>, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 2 required (not removing objects), shard db: /tmp/tmp2yzejvl5/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp2yzejvl5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.30196.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002586841583251953s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022745132446289062s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002295970916748047s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025391578674316406s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002925395965576172s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001842975616455078s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001838207244873047s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001685619354248047s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001800060272216797s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003056526184082031s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002510547637939453s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345919.47286, (0, 0) as of 1717345919.47286, active as of 1717345919.47286>, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345919.47288, (0, 0) as of 1717345919.47288, active as of 1717345919.47288>, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345919.47290, (0, 0) as of 1717345919.47290, active as of 1717345919.47290>, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001881122589111328s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001659393310546875s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001666545867919922s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017881393432617188s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00023794174194335938s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017952919006347656s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016689300537109375s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016641616821289062s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017142295837402344s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017023086547851562s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017023086547851562s, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345919.47286, (0, 0) as of 1717345919.47286, active as of 1717345919.47286>, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpnoq61_v4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.47670.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024127960205078125s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001494884490966797s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020384788513183594s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016760826110839844s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017452239990234375s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002522468566894531s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024080276489257812s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'yonder')], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020456314086914062s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00019240379333496094s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018835067749023438s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017404556274414062s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001697540283203125s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017189979553222656s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001838207244873047s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001819133758544922s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017118453979492188s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018286705017089844s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022649765014648438s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002148151397705078s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001914501190185547s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023698806762695312s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024390220642089844s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000316619873046875s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002307891845703125s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016641616821289062s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016689300537109375s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016570091247558594s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016641616821289062s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025200843811035156s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00030493736267089844s, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'yonder' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Placed 3 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmpuwqub5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345919.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026226043701171875s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024819374084472656s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022673606872558594s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002319812774658203s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002295970916748047s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002391338348388672s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018310546875s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00025177001953125s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Placed 1 misplaced objects (2 unplaced), path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020194053649902344s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00021195411682128906s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001881122589111328s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001804828643798828s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001773834228515625s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019979476928710938s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018787384033203125s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018405914306640625s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00020551681518554688s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018787384033203125s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019621849060058594s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018930435180664062s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding ... ok sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020313262939453125s, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Placed 5 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp3zzgn8t1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717345921.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025200843811035156s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014495849609375s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014328956604003906s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014328956604003906s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018739700317382812s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00018286705017089844s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000247955322265625s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018477439880371094s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002014636993408203s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00019860267639160156s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001914501190185547s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001614093780517578s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001876354217529297s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017309188842773438s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018978118896484375s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001728534698486328s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019288063049316406s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018596649169921875s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001614093780517578s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018739700317382812s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017309188842773438s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017595291137695312s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018477439880371094s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001881122589111328s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002155303955078125s, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpmfv9lda3/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002410411834716797s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014710426330566406s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014257431030273438s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001533031463623047s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019550323486328125s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017452239990234375s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002818107604980469s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001742839813232422s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016546249389648438s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000186920166015625s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017404556274414062s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016236305236816406s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017523765563964844s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016188621520996094s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one ... ok sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002086162567138672s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016427040100097656s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001773834228515625s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020742416381835938s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023818016052246094s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002079010009765625s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019669532775878906s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019121170043945312s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001666545867919922s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020575523376464844s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000164031982421875s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002009868621826172s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001761913299560547s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002574920654296875s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016760826110839844s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018715858459472656s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017833709716796875s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001780986785888672s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016641616821289062s, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpd2vshzfk/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00026607513427734375s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014472007751464844s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014400482177734375s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014352798461914062s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020265579223632812s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001838207244873047s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002899169921875s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001659393310546875s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025773048400878906s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000225067138671875s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017642974853515625s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001690387725830078s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001633167266845703s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001735687255859375s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001857280731201172s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002052783966064453s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002307891845703125s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001647472381591797s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001723766326904297s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001628398895263672s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001938343048095703s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001800060272216797s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019860267639160156s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019741058349609375s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021314620971679688s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020074844360351562s, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, active as of 1717345673.56112>, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp3xis4gxa/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002703666687011719s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002810955047607422s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023365020751953125s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025463104248046875s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002582073211669922s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018477439880371094s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017881393432617188s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016808509826660156s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018095970153808594s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002522468566894531s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024080276489257812s, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717345921.00181, (0, 0) as of 1717345921.00181, active as of 1717345921.00181>, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717345921.00183, (0, 0) as of 1717345921.00183, active as of 1717345921.00183>, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717345921.00185, (0, 0) as of 1717345921.00185, active as of 1717345921.00185>, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpbnrwt7zd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.00592.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpegwn1sem/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpegwn1sem/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpegwn1sem/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpegwn1sem/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp2z2_5vgx/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp2z2_5vgx/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp2z2_5vgx/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmp2z2_5vgx/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpzow43fp_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmpzow43fp_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpzow43fp_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmpzow43fp_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard ... ok sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpnr9nmtkr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpnr9nmtkr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpnr9nmtkr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpnr9nmtkr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:2 success:2 failure:0 skipped:1 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test ERROR: Unhandled exception while processing: kapow!, path: a/c1, db: /tmp/tmp_mnbt4nf/sdb/containers/0/ash/c1hash/c1hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2516, in _one_shard_cycle self._process_broker(broker, node, part) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1245, in _execute_mock_call result = effect(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 886, in mock_processing raise Exception('kapow!') Exception: kapow! sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:2 failure:1 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: Unlinked retiring db '/tmp/tmp_mnbt4nf/sda/containers/0/ash/c0hash/c0hash.db' sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 test.unit.container.test_sharder.TestSharder.test_one_shard_cycle test.unit.container.test_sharder.TestSharder.test_one_shard_cycle ... ok test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers ... ok sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Found no containers directories sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.05s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 0) sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.01s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdc) sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters ... ok test.unit.container.test_sharder.TestSharder.test_periodic_warning test.unit.container.test_sharder.TestSharder.test_periodic_warning ... ok sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.00s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdb, sdc) sharder-test INFO: Since Sun Jun 2 04:32:01 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:01 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmpvvtktr7r/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmpvvtktr7r/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmpvvtktr7r/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmpvvtktr7r/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard ... ok test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp41mg4znb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345921.90518.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmps5nuiaei/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps5nuiaei/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps5nuiaei/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps5nuiaei/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded (deleted), path: a/c, db: /tmp/tmps5nuiaei/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrinking' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharding' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharded' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrunk' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpg43jt7gz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others ... ok sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpfehv4do1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002510547637939453s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013184547424316406s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021910667419433594s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012969970703125s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash60False/hash60False_1717345922.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022268295288085938s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001316070556640625s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002636909484863281s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001361370086669922s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash60True/hash60True_1717345941.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023221969604492188s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001361370086669922s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022554397583007812s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013184547424316406s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges ... ok sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/lse/hash50False/hash50False_1717345960.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025343894958496094s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013327598571777344s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022411346435546875s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001327991485595703s, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - zero objects found, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmp4vmqhd0q/sda/containers/0/rue/hash50True/hash50True_1717345976.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h60/hash60/hash60_1717345922.96250.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin ... ok sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h50/hash50/hash50_1717345923.02285.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmps2fbw3oo/sda/containers/0/h70/hash70/hash70_1717345923.08740.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:01 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 5.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 3604.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 13:00:10 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 30.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test ERROR: Unhandled exception while dumping progress: Test over, path: a/c2, db: /tmp/tmpfcm_int2/sda/containers/0/ash/c2hash/c2hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over test.unit.container.test_sharder.TestSharder.test_run_forever test.unit.container.test_sharder.TestSharder.test_run_forever ... ok sharder-test ERROR: Exception in sharder: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2553, in run_forever self._one_shard_cycle(devices_to_shard=Everything(), File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2530, in _one_shard_cycle self._periodic_report_stats() File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1143, in _periodic_report_stats if (time.time() - self.reported) >= self.stats_interval: ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error test.unit.container.test_sharder.TestSharder.test_send_shard_ranges test.unit.container.test_sharder.TestSharder.test_send_shard_ranges ... ok sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpr9u81f6u/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003178119659423828s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001456737518310547s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj002_0 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmp6amxac9o/sda/containers/0/8fc/857fe81dc763e7cff7e7cd987d6c28fc/857fe81dc763e7cff7e7cd987d6c28fc.db, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaved ShardRange in 0.012s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002613067626953125s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014472007751464844s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmp6amxac9o/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017023086547851562s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001475811004638672s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112> - shard db already in sync, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmp6amxac9o/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010561943054199219s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=True) in 9.512901306152344e-05s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112> - shard db already in sync, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 0 required, shard db: /tmp/tmp6amxac9o/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010228157043457031s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=True) in 9.369850158691406e-05s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112> - shard db already in sync, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaved ShardRange<'obj002' to 'obj004' as of 1717345673.56112, (0, 0) as of 1717345673.56112, cleaved as of 1717345673.56112> in 0.003s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj006_2 for ShardRange<'obj004' to 'obj006' as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025343894958496094s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014472007751464844s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj006_2 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaved ShardRange<'obj004' to 'obj006' as of 1717345673.56112, (2, 2) as of 1717345923.70738, cleaved as of 1717345673.56112> in 0.023s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000110626220703125s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=True) in 9.679794311523438e-05s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002703666687011719s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001404285430908203s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__3 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmp6amxac9o/sda/containers/0/683/f7a6bb6d9367b7ae7b6b30a833289683/f7a6bb6d9367b7ae7b6b30a833289683.db, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures ... ok test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts ... ok test.unit.container.test_sharder.TestSharder.test_sharding_enabled test.unit.container.test_sharder.TestSharder.test_sharding_enabled ... ok sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00011038780212402344s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001304149627685547s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112>, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaving ShardRange<'obj006' to MaxBound as of 1717345673.56112, (0, 0) as of 1717345673.56112, created as of 1717345673.56112> - shard db already in sync, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test INFO: Cleaved ShardRange<'obj006' to MaxBound as of 1717345673.56112, (3, 3) as of 1717345923.83888, cleaved as of 1717345673.56112> in 0.012s, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp6amxac9o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.35638.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e75817a2-a86f-49bf-9ba9-388caf96ea69-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpa7yaibtn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e75817a2-a86f-49bf-9ba9-388caf96ea69-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpa7yaibtn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e75817a2-a86f-49bf-9ba9-388caf96ea69-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpa7yaibtn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e75817a2-a86f-49bf-9ba9-388caf96ea69-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpa7yaibtn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3e14dad2-f1f4-4e67-90a8-6cd9729714ff-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpa7yaibtn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717345923.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpt3voti8q/sda/containers/0/ash/c2hash/c2hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmpt3voti8q/sda/containers/0/ash/c1hash/c1hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmpt3voti8q/sda/containers/0/ash/c3hash/c3hash.db' sharder-test INFO: Since Sun Jun 2 04:32:04 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_shard - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported ... ok sharder-test INFO: Since Sun Jun 2 04:32:04 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:32:04 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpga3umv0z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpazsuc8cd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpazsuc8cd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpazsuc8cd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpazsuc8cd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpazsuc8cd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpazsuc8cd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 3, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=3, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 4, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range ... ok test.unit.container.test_sharder.TestSharder.test_update_stat test.unit.container.test_sharder.TestSharder.test_update_stat ... ok test.unit.container.test_sharder.TestSharder.test_yield_objects test.unit.container.test_sharder.TestSharder.test_yield_objects ... ok sharder-test DEBUG: updated root objs=4, tombstones=4, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 5, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=5, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 6, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=2, tombstones=6, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 7, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=1, tombstones=7, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 8, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=8, path: a/c, db: /tmp/tmpo07kvowb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=False) in 0.0003616809844970703s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=True) in 0.0003421306610107422s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=False) in 0.0002968311309814453s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00027942657470703125s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.0002372264862060547s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00026917457580566406s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.000286102294921875s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.0002713203430175781s, path: a/c, db: /tmp/tmpz50xqj1c/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024580955505371094s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002932548522949219s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002334117889404297s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.0002601146697998047s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002307891845703125s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.00019550323486328125s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001709461212158203s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00019431114196777344s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001761913299560547s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002856254577636719s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017070770263671875s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00025725364685058594s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017023086547851562s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020742416381835938s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001881122589111328s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00018167495727539062s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002014636993408203s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.0001957416534423828s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017642974853515625s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00018095970153808594s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017976760864257812s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001780986785888672s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001838207244873047s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00017952919006347656s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00020170211791992188s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.0001995563507080078s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range ... ok test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects ... ok test.unit.container.test_sync.TestContainerSync.test_FileLikeIter test.unit.container.test_sync.TestContainerSync.test_FileLikeIter ... ok test.unit.container.test_sync.TestContainerSync.test_container_first_loop test.unit.container.test_sync.TestContainerSync.test_container_first_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_report test.unit.container.test_sync.TestContainerSync.test_container_report ... ok test.unit.container.test_sync.TestContainerSync.test_container_second_loop test.unit.container.test_sync.TestContainerSync.test_container_second_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_stop_at test.unit.container.test_sync.TestContainerSync.test_container_stop_at ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete ... ok sharder-test DEBUG: got 3 rows (deleted=False) in 0.00018715858459472656s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00018739700317382812s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017762184143066406s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00018334388732910156s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0003960132598876953s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00020122528076171875s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00022649765014648438s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0001876354217529297s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.0001919269561767578s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017571449279785156s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=False) in 0.0002758502960205078s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00020599365234375s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.0003762245178222656s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00020813941955566406s, path: a/c, db: /tmp/tmprqnrlhhx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023293495178222656s, path: a/c, db: /tmp/tmp580cytwv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001347064971923828s, path: a/c, db: /tmp/tmp580cytwv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717345925.105955, time window end: 1717345925.105955, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 5, sync_point2: None, total_rows: 1 test-container-sync ERROR: ERROR Syncing : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 388, in container_sync if not self.container_sync_row( ^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: TestContainerSync.test_container_report..fake_container_sync_row() missing 1 required positional argument: 'realm_key' test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717345925.1085823, time window end: 1717345925.1085944, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync ERROR: ERROR Syncing {'ROWID': 1, 'name': 'o'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 534, in container_sync_row row['created_at']) ~~~^^^^^^^^^^^^^^ KeyError: 'created_at' test-container-sync INFO: Container sync report: a/c, time window start: 1717345925.108779, time window end: 1717345925.109301, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync INFO: Container sync report: a/c, time window start: 1717345925.1094651, time window end: 1717345925.1095395, puts: 0, posts: 0, deletes: 1, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms ... ok test.unit.container.test_sync.TestContainerSync.test_run_forever test.unit.container.test_sync.TestContainerSync.test_run_forever ... ok test.unit.container.test_sync.TestContainerSync.test_run_once test.unit.container.test_sync.TestContainerSync.test_run_once ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status ... ok test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test DEBUG: Skipping the update of recently changed /tmp/tmp460gej8k/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmp460gej8k/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmp460gej8k/sdq/objects/auditor_status_ALL.jsontest.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle ... ok test.unit.obj.test_server.TestZeroCopy.test_GET test.unit.obj.test_server.TestZeroCopy.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_GET_big test.unit.obj.test_server.TestZeroCopy.test_GET_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_quarantine test.unit.obj.test_server.TestZeroCopy.test_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache ... ok test DEBUG: Skipping the update of recently changed /tmp/tmp460gej8k/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping: /tmp/tmpzs2ahgvh/garbage is not a directory test DEBUG: Skipping: /tmp/tmpzs2ahgvh/garbage is not mounted test DEBUG: Skipping: /tmp/tmpp500bsmo/sdq is not mounted test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx55e1cef834994cc5ac32d-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbbbdf95d916849d0b7a26-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx843528986b5f4bb8a90e1-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx88c9a9bcbde54fb6a24f7-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx511f99d6384149b294172-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txb46ad89f3ae947b99f002-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3082d0c8346d4a4c9fdb9-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7b91add965de41cfb3135-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txf6fa1232139d4fbcbb307-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx53cfe649fe7547cfafb8f-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx0a18c19e7b1c43e599b7c-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx5936e2e1b5ef4119bb3d5-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txdba43ff1a7e84139a55f1-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx21c55ccb19fb4f3083e3a-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa27601374fa44f71993d1-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd6cb56d22ae144fc95906-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx328e475c7bcf4827a38ce-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx58a6e24165f34e19b12f6-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx1e377229895a470a9ea5e-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txafdb1cecba76438c8774b-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa8159fd3c2b44d33963af-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx53a0c35a684044dd9b909-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx07cc4e28286b4a6fae795-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txd849c63acf784b8f8c2cf-00665c9e85) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txd2df730f4f3c48f482935-00665c9e85) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: txe42002bf962d4d218ead7-00665c9e85) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txf620b7d8d9aa4e52aebbe-00665c9e85) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx8bfa0afde030407faf922-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx7fb984d506454429ae8c9-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: txbdd848346ee54386a7226-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: txba7e1717bb7d48d1b490b-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx71c6a0243bf54157afb5a-00665c9e85) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: tx3f719ba05ac34915ad981-00665c9e85) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbe91dd6915094e4b8c0e4-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx20696f4cf250419d8bc98-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf9746e60b9954352998fd-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txa9c19310552442e6aee3c-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb29cb3dc4fda4669bab24-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb55d30f2d6444dc4bd73a-00665c9e85) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: txf4775edfe5b04e40acdf9-00665c9e85) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf4775edfe5b04e40acdf9-00665c9e85) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx968d396f83cb485faa761-00665c9e85) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx968d396f83cb485faa761-00665c9e85) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx4d6bb9a8c58a454e91520-00665c9e86) proxy-server ERROR: Object GET returning 503 for [] (txn: tx4d6bb9a8c58a454e91520-00665c9e86) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfbcf70ba00da4a6595199-00665c9e86) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3c3815bc92e048fea262d-00665c9e86) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx489761bae7cd4bf4b3f15-00665c9e86) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config ... ok test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmp7uvn1b07/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp7uvn1b07/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmp7uvn1b07/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp7uvn1b07/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0159 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0116 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0107 "-" 3543394 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0110 "-" 3543394 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe56b162a58dd4b24b5cec-00665c9e87" "proxy-server 3543394" 0.0018 "-" 3543394 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:47811/sdb1/1/a/c" "txe56b162a58dd4b24b5cec-00665c9e87" "container-server 3543394" 0.0015 "-" 3543394 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:35831/sda1/1/a/c" "txe56b162a58dd4b24b5cec-00665c9e87" "container-server 3543394" 0.0014 "-" 3543394 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txe56b162a58dd4b24b5cec-00665c9e87" "proxy-server 3543394" 0.0188 "-" 3543394 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txe56b162a58dd4b24b5cec-00665c9e87" "proxy-server 3543394" 0.0326 "-" 3543394 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/07 PUT /v1/a/c HTTP/1.0 201 - - t - - - txe56b162a58dd4b24b5cec-00665c9e87 - 0.0437 - - 1717345927.266469717 1717345927.310129166 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx28819d55b04142d39239c-00665c9e87" "proxy-server 3543394" 0.0017 "-" 3543394 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:47811/sdb1/1/a1/c1" "tx28819d55b04142d39239c-00665c9e87" "container-server 3543394" 0.0016 "-" 3543394 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:35831/sda1/1/a1/c1" "tx28819d55b04142d39239c-00665c9e87" "container-server 3543394" 0.0013 "-" 3543394 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx28819d55b04142d39239c-00665c9e87" "proxy-server 3543394" 0.0185 "-" 3543394 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx28819d55b04142d39239c-00665c9e87" "proxy-server 3543394" 0.0310 "-" 3543394 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/07 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx28819d55b04142d39239c-00665c9e87 - 0.0412 - - 1717345927.311986446 1717345927.353162289 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx61ad21acdcac4e62b8d65-00665c9e87" "proxy-server 3543394" 0.0141 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:35831/sda1/2/a/c1" "tx61ad21acdcac4e62b8d65-00665c9e87" "container-server 3543394" 0.0084 "-" 3543394 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:47811/sdb1/2/a/c1" "tx61ad21acdcac4e62b8d65-00665c9e87" "container-server 3543394" 0.0014 "-" 3543394 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx61ad21acdcac4e62b8d65-00665c9e87" "proxy-server 3543394" 0.0449 "-" 3543394 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx61ad21acdcac4e62b8d65-00665c9e87" "proxy-server 3543394" 0.0327 "-" 3543394 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/07 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx61ad21acdcac4e62b8d65-00665c9e87 - 0.0698 - - 1717345927.355082750 1717345927.424880266 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx677e944064c54e42aa0d8-00665c9e87" "proxy-server 3543394" 0.0121 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:35831/sda1/1/a/c2" "tx677e944064c54e42aa0d8-00665c9e87" "container-server 3543394" 0.0214 "-" 3543394 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:47811/sdb1/1/a/c2" "tx677e944064c54e42aa0d8-00665c9e87" "container-server 3543394" 0.0015 "-" 3543394 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx677e944064c54e42aa0d8-00665c9e87" "proxy-server 3543394" 0.0939 "-" 3543394 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:07 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx677e944064c54e42aa0d8-00665c9e87" "proxy-server 3543394" 0.0827 "-" 3543394 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/07 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx677e944064c54e42aa0d8-00665c9e87 - 0.1173 - - 1717345927.427069902 1717345927.544339418 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test.unit.proxy.test_mem_server.TestContainerController.test_DELETE test.unit.proxy.test_mem_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx2cf5de1c3822435bb4dba-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx2c04f3fe626445b9a5da3-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: tx8478d148e9064d19aea59-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx8478d148e9064d19aea59-00665c9e87) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx8478d148e9064d19aea59-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx6612c61080fe4305b1035-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx6612c61080fe4305b1035-00665c9e87) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx6612c61080fe4305b1035-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx4df7733f81924c86bf9de-00665c9e87) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx4df7733f81924c86bf9de-00665c9e87) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txa102ba6e28f7444580f75-00665c9e87) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txa102ba6e28f7444580f75-00665c9e87) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx58feddce75014ee69a268-00665c9e87) test INFO: Client disconnected on read of '/a/c' (txn: tx58feddce75014ee69a268-00665c9e87) test INFO: Client disconnected on read of '/a/c' (txn: tx2d32954aa9a14c5bbd9cb-00665c9e87) test INFO: Client disconnected on read of '/a/c' (txn: tx2d32954aa9a14c5bbd9cb-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx4da595a8b2d6464e9e5c5-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx06ce9b2de2e748f199e51-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx0aa466398ecd4802b76e3-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx0aa466398ecd4802b76e3-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx0aa466398ecd4802b76e3-00665c9e87) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx0aa466398ecd4802b76e3-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx4efcc67c65fd49358bad5-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx4efcc67c65fd49358bad5-00665c9e87) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx4efcc67c65fd49358bad5-00665c9e87) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx4efcc67c65fd49358bad5-00665c9e87) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: txf54d447033114a9dbc82e-00665c9e87) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: txf54d447033114a9dbc82e-00665c9e87) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: txf54d447033114a9dbc82e-00665c9e87) test ERROR: Account HEAD returning 503 for [] (txn: txf54d447033114a9dbc82e-00665c9e87) test.unit.proxy.test_mem_server.TestContainerController.test_POST test.unit.proxy.test_mem_server.TestContainerController.test_POST ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.1:2001/sdb (txn: txad6089b12b3744c5ac329-00665c9e87) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx3584184148aa4ed998661-00665c9e87) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx3584184148aa4ed998661-00665c9e87) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx26ae6f924262486f88f5a-00665c9e87) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6c25fd9f7ffe4fc2b0ccd-00665c9e87) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3eb697bb391e439aafe84-00665c9e87) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb66c50cb4a6341d8bdf6c-00665c9e87) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx680e4e236ffe4964a72a8-00665c9e87) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx00bffe208fb8417aad450-00665c9e87) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx00bffe208fb8417aad450-00665c9e87) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7eaf75283154426b8db59-00665c9e87) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx7eaf75283154426b8db59-00665c9e87) test WARNING: Could not autocreate account '/account' (txn: tx7eaf75283154426b8db59-00665c9e87) test INFO: autocreate account '/account' (txn: txa559e18afce74af0bb9d2-00665c9e87) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test ERROR: Account HEAD returning 503 for [503, 503] (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test WARNING: Could not autocreate account '/account' (txn: txc5139d0cf4ad4c41b776e-00665c9e88) test INFO: autocreate account '/account' (txn: txdf994fcdd2554648bee8f-00665c9e88) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx5d3d8c94374241fba6ea5-00665c9e88) test.unit.proxy.test_mem_server.TestContainerController.test_PUT test.unit.proxy.test_mem_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl ... ok test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx5d3d8c94374241fba6ea5-00665c9e88) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx5d3d8c94374241fba6ea5-00665c9e88) test INFO: autocreate account '/account' (txn: txf196a5b0bf5849b0a9c0f-00665c9e88) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txb328b87ba81b489a9dadd-00665c9e88) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: txb328b87ba81b489a9dadd-00665c9e88) test INFO: autocreate account '/account' (txn: txb328b87ba81b489a9dadd-00665c9e88) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx7081fe2dfbcb4424948e5-00665c9e88) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx7081fe2dfbcb4424948e5-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: tx1e354c1661574115aea5a-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: tx63636b558e914966aee51-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca0ed534cf9f4327b772f-00665c9e88) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ff5a1dff20b488fb2ce7-00665c9e88) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx58e3f475b02c4d2eb2852-00665c9e88) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx25451f2bcdca463e9828a-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txcc5ca561acf24599a2427-00665c9e88) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txd9b718f51fd94cc3b5fa6-00665c9e88) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx69393ccee17942bab5e43-00665c9e88) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx69393ccee17942bab5e43-00665c9e88) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx69393ccee17942bab5e43-00665c9e88) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txe8319cc96a614fcab2afa-00665c9e88) test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txe8319cc96a614fcab2afa-00665c9e88) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txe8319cc96a614fcab2afa-00665c9e88) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx348c5471afe64777a8a20-00665c9e88) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx348c5471afe64777a8a20-00665c9e88) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx348c5471afe64777a8a20-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx348c5471afe64777a8a20-00665c9e88) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx40d59fc634cf48049328a-00665c9e88) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx40d59fc634cf48049328a-00665c9e88) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx40d59fc634cf48049328a-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx40d59fc634cf48049328a-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: txf1e1e984e77347818cd2d-00665c9e88) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txca649ffdd11341fe900a2-00665c9e88) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txca649ffdd11341fe900a2-00665c9e88) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca649ffdd11341fe900a2-00665c9e88) test ERROR: Account HEAD returning 503 for [] (txn: txca649ffdd11341fe900a2-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx0f528c46376d44cc8260e-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx0f528c46376d44cc8260e-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx0f528c46376d44cc8260e-00665c9e88) test ERROR: Account HEAD returning 503 for [] (txn: tx0f528c46376d44cc8260e-00665c9e88) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx6af54e881e7d4221915f7-00665c9e88) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx010b6420e44b4c54aadc6-00665c9e88) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx010b6420e44b4c54aadc6-00665c9e88) test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name ... ok test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx010b6420e44b4c54aadc6-00665c9e88) test ERROR: Account HEAD returning 503 for [] (txn: tx010b6420e44b4c54aadc6-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx64e05e69329e4ee48d472-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx64e05e69329e4ee48d472-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx64e05e69329e4ee48d472-00665c9e88) test ERROR: Account HEAD returning 503 for [] (txn: tx64e05e69329e4ee48d472-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx87b93239594c491c841a8-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: tx6340faa8482e46f98dc7f-00665c9e88) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: tx6340faa8482e46f98dc7f-00665c9e88) test INFO: Client disconnected on read of '/a/c' (txn: tx6340faa8482e46f98dc7f-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txbfc236b500d045ea8fc64-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx54b0838e42e24946816d0-00665c9e88) test INFO: Client disconnected on read of '/a/c' (txn: tx54b0838e42e24946816d0-00665c9e88) test INFO: Client disconnected on read of '/a/c/' (txn: tx54b0838e42e24946816d0-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx6bbf006b3fc64d9c9e863-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx6bbf006b3fc64d9c9e863-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx6bbf006b3fc64d9c9e863-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx6bbf006b3fc64d9c9e863-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx07f4200a98ad411aacab7-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx07f4200a98ad411aacab7-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx07f4200a98ad411aacab7-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx07f4200a98ad411aacab7-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx08f343c86987409e97d48-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx08f343c86987409e97d48-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx08f343c86987409e97d48-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx08f343c86987409e97d48-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx4c0a2a7a35e34cc09faab-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx4c0a2a7a35e34cc09faab-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx4c0a2a7a35e34cc09faab-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx4c0a2a7a35e34cc09faab-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx1dc43bf241b24c85a675e-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx1dc43bf241b24c85a675e-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx1dc43bf241b24c85a675e-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx1dc43bf241b24c85a675e-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txe2d02ecb1df046b9b9748-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txe2d02ecb1df046b9b9748-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txe2d02ecb1df046b9b9748-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txe2d02ecb1df046b9b9748-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txc3ff0ba139d84a05bfce3-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txc3ff0ba139d84a05bfce3-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txc3ff0ba139d84a05bfce3-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txc3ff0ba139d84a05bfce3-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx08e0ebb5d19d4d68a73f0-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx08e0ebb5d19d4d68a73f0-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx08e0ebb5d19d4d68a73f0-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx08e0ebb5d19d4d68a73f0-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx0a0e5b8c409a400484413-00665c9e88) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: tx0a0e5b8c409a400484413-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx0a0e5b8c409a400484413-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx0a0e5b8c409a400484413-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx0a0e5b8c409a400484413-00665c9e88) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx0a0e5b8c409a400484413-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd1bcc7f703dd4764ade08-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txd1bcc7f703dd4764ade08-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txd1bcc7f703dd4764ade08-00665c9e88) test ERROR: Container GET returning 503 for [503, 503] (txn: txd1bcc7f703dd4764ade08-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx94c6c6e857cf466e857aa-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx94c6c6e857cf466e857aa-00665c9e88) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx94c6c6e857cf466e857aa-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx94c6c6e857cf466e857aa-00665c9e88) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx94c6c6e857cf466e857aa-00665c9e88) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx94c6c6e857cf466e857aa-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx94c6c6e857cf466e857aa-00665c9e88) test ERROR: Container GET returning 503 for [503, 503] (txn: tx94c6c6e857cf466e857aa-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx68dc94befd684704a38c0-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx68dc94befd684704a38c0-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx68dc94befd684704a38c0-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx68dc94befd684704a38c0-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txdc5c60c45bed4f4db7aa3-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txdc5c60c45bed4f4db7aa3-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txdc5c60c45bed4f4db7aa3-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: txdc5c60c45bed4f4db7aa3-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txb608ccfab86440bdb45fa-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txb608ccfab86440bdb45fa-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txb608ccfab86440bdb45fa-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: txb608ccfab86440bdb45fa-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx0dd3ad6a32ef498aba3d4-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx0dd3ad6a32ef498aba3d4-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx0dd3ad6a32ef498aba3d4-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx0dd3ad6a32ef498aba3d4-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txc8f22912ba7b41fd87546-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txc8f22912ba7b41fd87546-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txc8f22912ba7b41fd87546-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: txc8f22912ba7b41fd87546-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx63f370d53a734e87a7e1f-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx63f370d53a734e87a7e1f-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx63f370d53a734e87a7e1f-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx63f370d53a734e87a7e1f-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx1276b7b9981e4caaa4c22-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx1276b7b9981e4caaa4c22-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx1276b7b9981e4caaa4c22-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx1276b7b9981e4caaa4c22-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7b02b49c19214bc4b042d-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7b02b49c19214bc4b042d-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7b02b49c19214bc4b042d-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx7b02b49c19214bc4b042d-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx80c22e0b565144eca8145-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx80c22e0b565144eca8145-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx80c22e0b565144eca8145-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx80c22e0b565144eca8145-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx1229c7321dbe455fa7f79-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx1229c7321dbe455fa7f79-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx1229c7321dbe455fa7f79-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: tx1229c7321dbe455fa7f79-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa9be9d9ca15b4fe893305-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa9be9d9ca15b4fe893305-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa9be9d9ca15b4fe893305-00665c9e88) test ERROR: Container GET returning 503 for [] (txn: txa9be9d9ca15b4fe893305-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7b6546718a5f4c3fa59ae-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7b6546718a5f4c3fa59ae-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7b6546718a5f4c3fa59ae-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx7b6546718a5f4c3fa59ae-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx2e60f9713a1c4172af1a0-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx2e60f9713a1c4172af1a0-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx2e60f9713a1c4172af1a0-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx2e60f9713a1c4172af1a0-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd6a6a5f6cdf644edb2a4d-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd6a6a5f6cdf644edb2a4d-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd6a6a5f6cdf644edb2a4d-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txd6a6a5f6cdf644edb2a4d-00665c9e88) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txee927e9b81574ab1be102-00665c9e88) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txee927e9b81574ab1be102-00665c9e88) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txee927e9b81574ab1be102-00665c9e88) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txee927e9b81574ab1be102-00665c9e88) test INFO: Client disconnected on read of '/a/c' (txn: txbb483fe88a4e4b45aecad-00665c9e88) test INFO: Client disconnected on read of '/a/c/' (txn: txbb483fe88a4e4b45aecad-00665c9e88) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx97cf9b51dbca4c42a7330-00665c9e88) test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx97cf9b51dbca4c42a7330-00665c9e88) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx97cf9b51dbca4c42a7330-00665c9e88) test ERROR: Account HEAD returning 503 for [] (txn: tx97cf9b51dbca4c42a7330-00665c9e88) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx8162c076d215484fb6047-00665c9e88) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx8162c076d215484fb6047-00665c9e88) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx8162c076d215484fb6047-00665c9e88) test ERROR: Account HEAD returning 503 for [] (txn: tx8162c076d215484fb6047-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx04ab7f20564f4530a3240-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx2f06abb285944309b0133-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txf873cba428f54b63a338e-00665c9e88) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpbskb5d4v/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpbskb5d4v/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpbskb5d4v/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpbskb5d4v/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0114 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0093 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0092 "-" 3543394 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0088 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txeaf13ab13bee4575b2171-00665c9e88" "proxy-server 3543394" 0.0016 "-" 3543394 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:58893/sdb1/1/a/c" "txeaf13ab13bee4575b2171-00665c9e88" "container-server 3543394" 0.0015 "-" 3543394 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:32771/sda1/1/a/c" "txeaf13ab13bee4575b2171-00665c9e88" "container-server 3543394" 0.0014 "-" 3543394 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txeaf13ab13bee4575b2171-00665c9e88" "proxy-server 3543394" 0.0160 "-" 3543394 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txeaf13ab13bee4575b2171-00665c9e88" "proxy-server 3543394" 0.0290 "-" 3543394 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/08 PUT /v1/a/c HTTP/1.0 201 - - t - - - txeaf13ab13bee4575b2171-00665c9e88 - 0.0400 - - 1717345928.941273928 1717345928.981234550 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:08 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx34aa5b3b7c1f4b289a3ff-00665c9e88" "proxy-server 3543394" 0.0016 "-" 3543394 - test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_server.TestProxyServer.test_check_response_503 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_server.TestProxyServer.test_denied_host_header ... ok test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:58893/sdb1/1/a1/c1" "tx34aa5b3b7c1f4b289a3ff-00665c9e88" "container-server 3543394" 0.0014 "-" 3543394 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:32771/sda1/1/a1/c1" "tx34aa5b3b7c1f4b289a3ff-00665c9e88" "container-server 3543394" 0.0014 "-" 3543394 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx34aa5b3b7c1f4b289a3ff-00665c9e88" "proxy-server 3543394" 0.0295 "-" 3543394 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx34aa5b3b7c1f4b289a3ff-00665c9e88" "proxy-server 3543394" 0.0190 "-" 3543394 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/09 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx34aa5b3b7c1f4b289a3ff-00665c9e88 - 0.0405 - - 1717345928.983104229 1717345929.023555279 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd535d5db0ecc4eda94f36-00665c9e89" "proxy-server 3543394" 0.0068 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:32771/sda1/2/a/c1" "txd535d5db0ecc4eda94f36-00665c9e89" "container-server 3543394" 0.0053 "-" 3543394 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:58893/sdb1/2/a/c1" "txd535d5db0ecc4eda94f36-00665c9e89" "container-server 3543394" 0.0014 "-" 3543394 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd535d5db0ecc4eda94f36-00665c9e89" "proxy-server 3543394" 0.0310 "-" 3543394 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd535d5db0ecc4eda94f36-00665c9e89" "proxy-server 3543394" 0.0207 "-" 3543394 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/09 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txd535d5db0ecc4eda94f36-00665c9e89 - 0.0477 - - 1717345929.025422335 1717345929.073140144 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx760ed8f20f2543509c648-00665c9e89" "proxy-server 3543394" 0.0058 "-" 3543394 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:32771/sda1/1/a/c2" "tx760ed8f20f2543509c648-00665c9e89" "container-server 3543394" 0.0034 "-" 3543394 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:58893/sdb1/1/a/c2" "tx760ed8f20f2543509c648-00665c9e89" "container-server 3543394" 0.0077 "-" 3543394 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx760ed8f20f2543509c648-00665c9e89" "proxy-server 3543394" 0.0357 "-" 3543394 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:32:09 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx760ed8f20f2543509c648-00665c9e89" "proxy-server 3543394" 0.0258 "-" 3543394 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/32/09 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx760ed8f20f2543509c648-00665c9e89 - 0.0516 - - 1717345929.075039625 1717345929.126644373 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test.unit.proxy.test_server.TestProxyServer.test_error_occurred test.unit.proxy.test_server.TestProxyServer.test_error_occurred ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test.unit.proxy.test_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_server.TestProxyServer.test_exception_occurred ... ok test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_info_defaults test.unit.proxy.test_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_server.TestProxyServer.test_node_timing test.unit.proxy.test_server.TestProxyServer.test_node_timing ... ok test.unit.proxy.test_server.TestProxyServer.test_private_method_request test.unit.proxy.test_server.TestProxyServer.test_private_method_request ... ok test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx8a68201e6e0a4f138db3f-00665c9e89) test INFO: Client disconnected on read of '/a/c' (txn: tx8a68201e6e0a4f138db3f-00665c9e89) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: tx88c040df98554bd8a4bbe-00665c9e89) test INFO: Client disconnected on read of '/a/c' (txn: tx88c040df98554bd8a4bbe-00665c9e89) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: txa981ed8cef27439da0c3f-00665c9e8a) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: txa981ed8cef27439da0c3f-00665c9e8a) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: txa981ed8cef27439da0c3f-00665c9e8a) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: txa981ed8cef27439da0c3f-00665c9e8a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix ... ok test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_server.TestProxyServer.test_valid_api_version ... ok test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) ---------------------------------------------------------------------- Ran 8924 tests in 276.020s OK (skipped=1814) + stestr slowest Test id Runtime (s) --------------------------------------------------------------------------------------------------------------------------- ----------- test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving 50.581 test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug 42.955 test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges 33.104 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance 26.898 test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET 22.172 test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges 20.657 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving 20.069 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match 17.768 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving 15.127 test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache 12.823 + rm -rf .stestr make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_install -O--buildsystem=pybuild debian/rules override_dh_installdocs make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh_installdocs dh_installdocs -A AUTHORS dh_installdocs: warning: Not using auto-detected swift as main doc package for swift-doc: With -A/--all, this would cause file-conflicts. make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' debian/rules override_dh_sphinxdoc make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' LC_ALL=C.UTF-8 PYTHONPATH=. python3 -m sphinx doc/source /build/reproducible-path/swift-2.33.0/debian/swift-doc/usr/share/doc/swift-doc/base Running Sphinx v7.2.6 [openstackdocstheme] version: 3.2.0 [openstackdocstheme] connecting html-page-context event handler making output directory... done [openstackdocstheme] using theme from /usr/lib/python3/dist-packages/openstackdocstheme/theme [openstackdocstheme] cannot find git top directory, assuming "." building [mo]: targets for 0 po files that are out of date writing output... building [html]: targets for 128 source files that are out of date updating environment: [new config] 128 added, 0 changed, 0 removed reading sources... [ 1%] account reading sources... [ 2%] admin/index reading sources... [ 2%] admin/objectstorage-EC reading sources... [ 3%] admin/objectstorage-account-reaper reading sources... [ 4%] admin/objectstorage-arch reading sources... [ 5%] admin/objectstorage-auditors reading sources... [ 5%] admin/objectstorage-characteristics reading sources... [ 6%] admin/objectstorage-components reading sources... [ 7%] admin/objectstorage-features reading sources... [ 8%] admin/objectstorage-intro reading sources... [ 9%] admin/objectstorage-large-objects reading sources... [ 9%] admin/objectstorage-monitoring reading sources... [ 10%] admin/objectstorage-replication reading sources... [ 11%] admin/objectstorage-ringbuilder reading sources... [ 12%] admin/objectstorage-tenant-specific-image-storage reading sources... [ 12%] admin/objectstorage-troubleshoot reading sources... [ 13%] admin_guide reading sources... [ 14%] apache_deployment_guide reading sources... [ 15%] api/authentication reading sources... [ 16%] api/bulk-delete reading sources... [ 16%] api/container_quotas reading sources... [ 17%] api/discoverability reading sources... [ 18%] api/form_post_middleware reading sources... [ 19%] api/large_objects reading sources... [ 20%] api/object-expiration reading sources... [ 20%] api/object_api_v1_overview reading sources... [ 21%] api/object_versioning reading sources... [ 22%] api/pagination reading sources... [ 23%] api/pseudo-hierarchical-folders-directories reading sources... [ 23%] api/serialized-response-formats reading sources... [ 24%] api/static-website reading sources... [ 25%] api/temporary_url_middleware reading sources... [ 26%] api/use_content-encoding_metadata reading sources... [ 27%] api/use_the_content-disposition_metadata reading sources... [ 27%] associated_projects reading sources... [ 28%] audit_watchers reading sources... [ 29%] config/account_server_config reading sources... [ 30%] config/container_server_config reading sources... [ 30%] config/global_memcache_config reading sources... [ 31%] config/index reading sources... [ 32%] config/object_server_config reading sources... [ 33%] config/proxy_server_config reading sources... [ 34%] config/swift_common_config reading sources... [ 34%] container reading sources... [ 35%] contributor/contributing reading sources... [ 36%] contributor/review_guidelines reading sources... [ 37%] cors reading sources... [ 38%] crossdomain reading sources... [ 38%] db reading sources... [ 39%] deployment_guide reading sources... [ 40%] development_auth reading sources... [ 41%] development_guidelines reading sources... [ 41%] development_middleware reading sources... [ 42%] development_ondisk_backends reading sources... [ 43%] development_saio reading sources... [ 44%] development_watchers reading sources... [ 45%] first_contribution_swift reading sources... [ 45%] getting_started reading sources... [ 46%] index reading sources... [ 47%] install/controller-install reading sources... [ 48%] install/controller-install-debian reading sources... [ 48%] install/controller-install-obs reading sources... [ 49%] install/controller-install-rdo reading sources... [ 50%] install/controller-install-ubuntu reading sources... [ 51%] install/environment-networking reading sources... [ 52%] install/finalize-installation reading sources... [ 52%] install/finalize-installation-obs reading sources... [ 53%] install/finalize-installation-rdo reading sources... [ 54%] install/finalize-installation-ubuntu-debian reading sources... [ 55%] install/get_started reading sources... [ 55%] install/index reading sources... [ 56%] install/initial-rings reading sources... [ 57%] install/next-steps reading sources... [ 58%] install/storage-install reading sources... [ 59%] install/storage-install-obs reading sources... [ 59%] install/storage-install-rdo reading sources... [ 60%] install/storage-install-ubuntu-debian reading sources... [ 61%] install/verify reading sources... [ 62%] logs reading sources... [ 62%] metrics/account_auditor reading sources... [ 63%] metrics/account_reaper reading sources... [ 64%] metrics/account_replicator reading sources... [ 65%] metrics/account_server reading sources... [ 66%] metrics/all reading sources... [ 66%] metrics/container_auditor reading sources... [ 67%] metrics/container_replicator reading sources... [ 68%] metrics/container_server reading sources... [ 69%] metrics/container_sync reading sources... [ 70%] metrics/container_updater reading sources... [ 70%] metrics/object_auditor reading sources... [ 71%] metrics/object_expirer reading sources... [ 72%] metrics/object_reconstructor reading sources... [ 73%] metrics/object_replicator reading sources... [ 73%] metrics/object_server reading sources... [ 74%] metrics/object_updater reading sources... [ 75%] metrics/proxy_server reading sources... [ 76%] middleware reading sources... [ 77%] misc reading sources... [ 77%] object reading sources... [ 78%] ops_runbook/diagnose reading sources... [ 79%] ops_runbook/index reading sources... [ 80%] ops_runbook/maintenance reading sources... [ 80%] ops_runbook/procedures reading sources... [ 81%] ops_runbook/troubleshooting reading sources... [ 82%] overview_acl reading sources... [ 83%] overview_architecture reading sources... [ 84%] overview_auth reading sources... [ 84%] overview_backing_store reading sources... [ 85%] overview_container_sharding reading sources... [ 86%] overview_container_sync reading sources... [ 87%] overview_encryption reading sources... [ 88%] overview_erasure_code reading sources... [ 88%] overview_expiring_objects reading sources... [ 89%] overview_global_cluster reading sources... [ 90%] overview_large_objects reading sources... [ 91%] overview_policies reading sources... [ 91%] overview_reaper reading sources... [ 92%] overview_replication reading sources... [ 93%] overview_ring reading sources... [ 94%] overview_wsgi_management reading sources... [ 95%] policies_saio reading sources... [ 95%] proxy reading sources... [ 96%] ratelimit reading sources... [ 97%] replication_network reading sources... [ 98%] ring reading sources... [ 98%] ring_background reading sources... [ 99%] ring_partpower reading sources... [100%] s3_compat looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done copying assets... copying downloadable files... [100%] ../../../etc/proxy-server.conf-sample copying static files... done copying extra files... done done writing output... [ 1%] account [openstackdocstheme] Could not get modification time of doc/source/account.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'account' writing output... [ 2%] admin/index [openstackdocstheme] Could not get modification time of doc/source/admin/index.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/index' writing output... [ 2%] admin/objectstorage-EC [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-EC.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-EC' writing output... [ 3%] admin/objectstorage-account-reaper [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-account-reaper.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-account-reaper' writing output... [ 4%] admin/objectstorage-arch [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-arch.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-arch' writing output... [ 5%] admin/objectstorage-auditors [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-auditors.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-auditors' writing output... [ 5%] admin/objectstorage-characteristics [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-characteristics.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-characteristics' writing output... [ 6%] admin/objectstorage-components [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-components.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-components' writing output... [ 7%] admin/objectstorage-features [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-features.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-features' writing output... [ 8%] admin/objectstorage-intro [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-intro.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-intro' writing output... [ 9%] admin/objectstorage-large-objects [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-large-objects.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-large-objects' writing output... [ 9%] admin/objectstorage-monitoring [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-monitoring.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-monitoring' writing output... [ 10%] admin/objectstorage-replication [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-replication.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-replication' writing output... [ 11%] admin/objectstorage-ringbuilder [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-ringbuilder.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-ringbuilder' writing output... [ 12%] admin/objectstorage-tenant-specific-image-storage [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-tenant-specific-image-storage.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-tenant-specific-image-storage' writing output... [ 12%] admin/objectstorage-troubleshoot [openstackdocstheme] Could not get modification time of doc/source/admin/objectstorage-troubleshoot.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin/objectstorage-troubleshoot' writing output... [ 13%] admin_guide [openstackdocstheme] Could not get modification time of doc/source/admin_guide.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'admin_guide' writing output... [ 14%] apache_deployment_guide [openstackdocstheme] Could not get modification time of doc/source/apache_deployment_guide.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'apache_deployment_guide' writing output... [ 15%] api/authentication [openstackdocstheme] Could not get modification time of doc/source/api/authentication.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/authentication' writing output... [ 16%] api/bulk-delete [openstackdocstheme] Could not get modification time of doc/source/api/bulk-delete.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/bulk-delete' writing output... [ 16%] api/container_quotas [openstackdocstheme] Could not get modification time of doc/source/api/container_quotas.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/container_quotas' writing output... [ 17%] api/discoverability [openstackdocstheme] Could not get modification time of doc/source/api/discoverability.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/discoverability' writing output... [ 18%] api/form_post_middleware [openstackdocstheme] Could not get modification time of doc/source/api/form_post_middleware.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/form_post_middleware' writing output... [ 19%] api/large_objects [openstackdocstheme] Could not get modification time of doc/source/api/large_objects.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/large_objects' writing output... [ 20%] api/object-expiration [openstackdocstheme] Could not get modification time of doc/source/api/object-expiration.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/object-expiration' writing output... [ 20%] api/object_api_v1_overview [openstackdocstheme] Could not get modification time of doc/source/api/object_api_v1_overview.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/object_api_v1_overview' writing output... [ 21%] api/object_versioning [openstackdocstheme] Could not get modification time of doc/source/api/object_versioning.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/object_versioning' writing output... [ 22%] api/pagination [openstackdocstheme] Could not get modification time of doc/source/api/pagination.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/pagination' writing output... [ 23%] api/pseudo-hierarchical-folders-directories [openstackdocstheme] Could not get modification time of doc/source/api/pseudo-hierarchical-folders-directories.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/pseudo-hierarchical-folders-directories' writing output... [ 23%] api/serialized-response-formats [openstackdocstheme] Could not get modification time of doc/source/api/serialized-response-formats.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/serialized-response-formats' writing output... [ 24%] api/static-website [openstackdocstheme] Could not get modification time of doc/source/api/static-website.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/static-website' writing output... [ 25%] api/temporary_url_middleware [openstackdocstheme] Could not get modification time of doc/source/api/temporary_url_middleware.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/temporary_url_middleware' writing output... [ 26%] api/use_content-encoding_metadata [openstackdocstheme] Could not get modification time of doc/source/api/use_content-encoding_metadata.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/use_content-encoding_metadata' writing output... [ 27%] api/use_the_content-disposition_metadata [openstackdocstheme] Could not get modification time of doc/source/api/use_the_content-disposition_metadata.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'api/use_the_content-disposition_metadata' writing output... [ 27%] associated_projects [openstackdocstheme] Could not get modification time of doc/source/associated_projects.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'associated_projects' writing output... [ 28%] audit_watchers [openstackdocstheme] Could not get modification time of doc/source/audit_watchers.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'audit_watchers' writing output... [ 29%] config/account_server_config [openstackdocstheme] Could not get modification time of doc/source/config/account_server_config.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/account_server_config' writing output... [ 30%] config/container_server_config [openstackdocstheme] Could not get modification time of doc/source/config/container_server_config.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/container_server_config' writing output... [ 30%] config/global_memcache_config [openstackdocstheme] Could not get modification time of doc/source/config/global_memcache_config.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/global_memcache_config' writing output... [ 31%] config/index [openstackdocstheme] Could not get modification time of doc/source/config/index.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/index' writing output... [ 32%] config/object_server_config [openstackdocstheme] Could not get modification time of doc/source/config/object_server_config.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/object_server_config' writing output... [ 33%] config/proxy_server_config [openstackdocstheme] Could not get modification time of doc/source/config/proxy_server_config.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/proxy_server_config' writing output... [ 34%] config/swift_common_config [openstackdocstheme] Could not get modification time of doc/source/config/swift_common_config.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'config/swift_common_config' writing output... [ 34%] container [openstackdocstheme] Could not get modification time of doc/source/container.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'container' writing output... [ 35%] contributor/contributing [openstackdocstheme] Could not get modification time of doc/source/contributor/contributing.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'contributor/contributing' writing output... [ 36%] contributor/review_guidelines [openstackdocstheme] Could not get modification time of doc/source/contributor/review_guidelines.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'contributor/review_guidelines' writing output... [ 37%] cors [openstackdocstheme] Could not get modification time of doc/source/cors.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'cors' writing output... [ 38%] crossdomain [openstackdocstheme] Could not get modification time of doc/source/crossdomain.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'crossdomain' writing output... [ 38%] db [openstackdocstheme] Could not get modification time of doc/source/db.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'db' writing output... [ 39%] deployment_guide [openstackdocstheme] Could not get modification time of doc/source/deployment_guide.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'deployment_guide' writing output... [ 40%] development_auth [openstackdocstheme] Could not get modification time of doc/source/development_auth.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'development_auth' writing output... [ 41%] development_guidelines [openstackdocstheme] Could not get modification time of doc/source/development_guidelines.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'development_guidelines' writing output... [ 41%] development_middleware [openstackdocstheme] Could not get modification time of doc/source/development_middleware.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'development_middleware' writing output... [ 42%] development_ondisk_backends [openstackdocstheme] Could not get modification time of doc/source/development_ondisk_backends.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'development_ondisk_backends' writing output... [ 43%] development_saio [openstackdocstheme] Could not get modification time of doc/source/development_saio.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'development_saio' writing output... [ 44%] development_watchers [openstackdocstheme] Could not get modification time of doc/source/development_watchers.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'development_watchers' writing output... [ 45%] first_contribution_swift [openstackdocstheme] Could not get modification time of doc/source/first_contribution_swift.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'first_contribution_swift' writing output... [ 45%] getting_started [openstackdocstheme] Could not get modification time of doc/source/getting_started.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'getting_started' writing output... [ 46%] index [openstackdocstheme] Could not get modification time of doc/source/index.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'index' writing output... [ 47%] install/controller-install [openstackdocstheme] Could not get modification time of doc/source/install/controller-install.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/controller-install' writing output... [ 48%] install/controller-install-debian [openstackdocstheme] Could not get modification time of doc/source/install/controller-install-debian.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/controller-install-debian' writing output... [ 48%] install/controller-install-obs [openstackdocstheme] Could not get modification time of doc/source/install/controller-install-obs.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/controller-install-obs' writing output... [ 49%] install/controller-install-rdo [openstackdocstheme] Could not get modification time of doc/source/install/controller-install-rdo.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/controller-install-rdo' writing output... [ 50%] install/controller-install-ubuntu [openstackdocstheme] Could not get modification time of doc/source/install/controller-install-ubuntu.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/controller-install-ubuntu' writing output... [ 51%] install/environment-networking [openstackdocstheme] Could not get modification time of doc/source/install/environment-networking.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/environment-networking' writing output... [ 52%] install/finalize-installation [openstackdocstheme] Could not get modification time of doc/source/install/finalize-installation.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/finalize-installation' writing output... [ 52%] install/finalize-installation-obs [openstackdocstheme] Could not get modification time of doc/source/install/finalize-installation-obs.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/finalize-installation-obs' writing output... [ 53%] install/finalize-installation-rdo [openstackdocstheme] Could not get modification time of doc/source/install/finalize-installation-rdo.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/finalize-installation-rdo' writing output... [ 54%] install/finalize-installation-ubuntu-debian [openstackdocstheme] Could not get modification time of doc/source/install/finalize-installation-ubuntu-debian.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/finalize-installation-ubuntu-debian' writing output... [ 55%] install/get_started [openstackdocstheme] Could not get modification time of doc/source/install/get_started.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/get_started' writing output... [ 55%] install/index [openstackdocstheme] Could not get modification time of doc/source/install/index.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/index' writing output... [ 56%] install/initial-rings [openstackdocstheme] Could not get modification time of doc/source/install/initial-rings.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/initial-rings' writing output... [ 57%] install/next-steps [openstackdocstheme] Could not get modification time of doc/source/install/next-steps.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/next-steps' writing output... [ 58%] install/storage-install [openstackdocstheme] Could not get modification time of doc/source/install/storage-install.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/storage-install' writing output... [ 59%] install/storage-install-obs [openstackdocstheme] Could not get modification time of doc/source/install/storage-install-obs.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/storage-install-obs' writing output... [ 59%] install/storage-install-rdo [openstackdocstheme] Could not get modification time of doc/source/install/storage-install-rdo.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/storage-install-rdo' writing output... [ 60%] install/storage-install-ubuntu-debian [openstackdocstheme] Could not get modification time of doc/source/install/storage-install-ubuntu-debian.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/storage-install-ubuntu-debian' writing output... [ 61%] install/verify [openstackdocstheme] Could not get modification time of doc/source/install/verify.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'install/verify' writing output... [ 62%] logs [openstackdocstheme] Could not get modification time of doc/source/logs.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'logs' writing output... [ 62%] metrics/account_auditor [openstackdocstheme] Could not get modification time of doc/source/metrics/account_auditor.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/account_auditor' writing output... [ 63%] metrics/account_reaper [openstackdocstheme] Could not get modification time of doc/source/metrics/account_reaper.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/account_reaper' writing output... [ 64%] metrics/account_replicator [openstackdocstheme] Could not get modification time of doc/source/metrics/account_replicator.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/account_replicator' writing output... [ 65%] metrics/account_server [openstackdocstheme] Could not get modification time of doc/source/metrics/account_server.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/account_server' writing output... [ 66%] metrics/all [openstackdocstheme] Could not get modification time of doc/source/metrics/all.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/all' writing output... [ 66%] metrics/container_auditor [openstackdocstheme] Could not get modification time of doc/source/metrics/container_auditor.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/container_auditor' writing output... [ 67%] metrics/container_replicator [openstackdocstheme] Could not get modification time of doc/source/metrics/container_replicator.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/container_replicator' writing output... [ 68%] metrics/container_server [openstackdocstheme] Could not get modification time of doc/source/metrics/container_server.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/container_server' writing output... [ 69%] metrics/container_sync [openstackdocstheme] Could not get modification time of doc/source/metrics/container_sync.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/container_sync' writing output... [ 70%] metrics/container_updater [openstackdocstheme] Could not get modification time of doc/source/metrics/container_updater.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/container_updater' writing output... [ 70%] metrics/object_auditor [openstackdocstheme] Could not get modification time of doc/source/metrics/object_auditor.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/object_auditor' writing output... [ 71%] metrics/object_expirer [openstackdocstheme] Could not get modification time of doc/source/metrics/object_expirer.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/object_expirer' writing output... [ 72%] metrics/object_reconstructor [openstackdocstheme] Could not get modification time of doc/source/metrics/object_reconstructor.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/object_reconstructor' writing output... [ 73%] metrics/object_replicator [openstackdocstheme] Could not get modification time of doc/source/metrics/object_replicator.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/object_replicator' writing output... [ 73%] metrics/object_server [openstackdocstheme] Could not get modification time of doc/source/metrics/object_server.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/object_server' writing output... [ 74%] metrics/object_updater [openstackdocstheme] Could not get modification time of doc/source/metrics/object_updater.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/object_updater' writing output... [ 75%] metrics/proxy_server [openstackdocstheme] Could not get modification time of doc/source/metrics/proxy_server.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'metrics/proxy_server' writing output... [ 76%] middleware [openstackdocstheme] Could not get modification time of doc/source/middleware.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'middleware' writing output... [ 77%] misc [openstackdocstheme] Could not get modification time of doc/source/misc.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'misc' writing output... [ 77%] object [openstackdocstheme] Could not get modification time of doc/source/object.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'object' writing output... [ 78%] ops_runbook/diagnose [openstackdocstheme] Could not get modification time of doc/source/ops_runbook/diagnose.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ops_runbook/diagnose' writing output... [ 79%] ops_runbook/index [openstackdocstheme] Could not get modification time of doc/source/ops_runbook/index.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ops_runbook/index' writing output... [ 80%] ops_runbook/maintenance [openstackdocstheme] Could not get modification time of doc/source/ops_runbook/maintenance.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ops_runbook/maintenance' writing output... [ 80%] ops_runbook/procedures [openstackdocstheme] Could not get modification time of doc/source/ops_runbook/procedures.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ops_runbook/procedures' writing output... [ 81%] ops_runbook/troubleshooting [openstackdocstheme] Could not get modification time of doc/source/ops_runbook/troubleshooting.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ops_runbook/troubleshooting' writing output... [ 82%] overview_acl [openstackdocstheme] Could not get modification time of doc/source/overview_acl.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_acl' writing output... [ 83%] overview_architecture [openstackdocstheme] Could not get modification time of doc/source/overview_architecture.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_architecture' writing output... [ 84%] overview_auth [openstackdocstheme] Could not get modification time of doc/source/overview_auth.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_auth' writing output... [ 84%] overview_backing_store [openstackdocstheme] Could not get modification time of doc/source/overview_backing_store.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_backing_store' writing output... [ 85%] overview_container_sharding [openstackdocstheme] Could not get modification time of doc/source/overview_container_sharding.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_container_sharding' writing output... [ 86%] overview_container_sync [openstackdocstheme] Could not get modification time of doc/source/overview_container_sync.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_container_sync' writing output... [ 87%] overview_encryption [openstackdocstheme] Could not get modification time of doc/source/overview_encryption.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_encryption' writing output... [ 88%] overview_erasure_code [openstackdocstheme] Could not get modification time of doc/source/overview_erasure_code.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_erasure_code' writing output... [ 88%] overview_expiring_objects [openstackdocstheme] Could not get modification time of doc/source/overview_expiring_objects.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_expiring_objects' writing output... [ 89%] overview_global_cluster [openstackdocstheme] Could not get modification time of doc/source/overview_global_cluster.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_global_cluster' writing output... [ 90%] overview_large_objects [openstackdocstheme] Could not get modification time of doc/source/overview_large_objects.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_large_objects' writing output... [ 91%] overview_policies [openstackdocstheme] Could not get modification time of doc/source/overview_policies.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_policies' writing output... [ 91%] overview_reaper [openstackdocstheme] Could not get modification time of doc/source/overview_reaper.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_reaper' writing output... [ 92%] overview_replication [openstackdocstheme] Could not get modification time of doc/source/overview_replication.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_replication' writing output... [ 93%] overview_ring [openstackdocstheme] Could not get modification time of doc/source/overview_ring.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_ring' writing output... [ 94%] overview_wsgi_management [openstackdocstheme] Could not get modification time of doc/source/overview_wsgi_management.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'overview_wsgi_management' writing output... [ 95%] policies_saio [openstackdocstheme] Could not get modification time of doc/source/policies_saio.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'policies_saio' writing output... [ 95%] proxy [openstackdocstheme] Could not get modification time of doc/source/proxy.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'proxy' writing output... [ 96%] ratelimit [openstackdocstheme] Could not get modification time of doc/source/ratelimit.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ratelimit' writing output... [ 97%] replication_network [openstackdocstheme] Could not get modification time of doc/source/replication_network.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'replication_network' writing output... [ 98%] ring [openstackdocstheme] Could not get modification time of doc/source/ring.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ring' writing output... [ 98%] ring_background [openstackdocstheme] Could not get modification time of doc/source/ring_background.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ring_background' writing output... [ 99%] ring_partpower [openstackdocstheme] Could not get modification time of doc/source/ring_partpower.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'ring_partpower' writing output... [100%] s3_compat [openstackdocstheme] Could not get modification time of doc/source/s3_compat.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 's3_compat' WARNING: [openstackdocstheme] cannot get gitsha from git repository generating indices... genindex py-modindex [openstackdocstheme] could not determine last_updated for 'py-modindex' done writing additional pages... search done copying images... [ 4%] admin/figures/objectstorage-arch.png copying images... [ 8%] admin/figures/objectstorage-nodes.png copying images... [ 12%] admin/figures/objectstorage.png copying images... [ 17%] admin/figures/objectstorage-buildingblocks.png copying images... [ 21%] admin/figures/objectstorage-ring.png copying images... [ 25%] admin/figures/objectstorage-zones.png copying images... [ 29%] admin/figures/objectstorage-accountscontainers.png copying images... [ 33%] admin/figures/objectstorage-partitions.png copying images... [ 38%] admin/figures/objectstorage-replication.png copying images... [ 42%] admin/figures/objectstorage-usecase.png copying images... [ 46%] images/sharding_unsharded.svg copying images... [ 50%] images/sharding_scan_basic.svg copying images... [ 54%] images/sharding_db_states.svg copying images... [ 58%] images/sharding_cleave_basic.svg copying images... [ 62%] images/sharding_scan_load.svg copying images... [ 67%] images/sharding_cleave1_load.svg copying images... [ 71%] images/sharding_cleave2_load.svg copying images... [ 75%] images/ec_overview.png copying images... [ 79%] images/reload_process_tree_1.svg copying images... [ 83%] images/reload_process_tree_2.svg copying images... [ 88%] images/reload_process_tree_3.svg copying images... [ 92%] images/reload_process_tree_4.svg copying images... [ 96%] images/reload_process_tree_5.svg copying images... [100%] images/reload_process_tree_6.svg dumping search index in English (code: en)... done dumping object inventory... done build succeeded, 1 warning. The HTML pages are in debian/swift-doc/usr/share/doc/swift-doc/base. LC_ALL=C.UTF-8 PYTHONPATH=. python3 -m sphinx api-ref/source /build/reproducible-path/swift-2.33.0/debian/swift-doc/usr/share/doc/swift-doc/api-ref Running Sphinx v7.2.6 [openstackdocstheme] version: 3.2.0 [openstackdocstheme] connecting html-page-context event handler making output directory... done [openstackdocstheme] overriding configured project name (Object Storage API Reference) with name extracted from the package (swift); you can disable this behavior with the 'openstackdocs_auto_name' option [openstackdocstheme] using theme from /usr/lib/python3/dist-packages/openstackdocstheme/theme [openstackdocstheme] cannot find git top directory, assuming "." building [mo]: targets for 0 po files that are out of date writing output... building [html]: targets for 1 source files that are out of date updating environment: [new config] 1 added, 0 changed, 0 removed reading sources... [100%] index looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done copying assets... copying static files... done copying extra files... done done writing output... [100%] index [openstackdocstheme] Could not get modification time of doc/source/index.rst: [Errno 2] No such file or directory: 'git' [openstackdocstheme] could not determine last_updated for 'index' WARNING: [openstackdocstheme] cannot get gitsha from git repository generating indices... genindex done writing additional pages... search done dumping search index in English (code: en)... done dumping object inventory... done Copying assets: api-site.css, api-site.js, combobox.js build succeeded, 1 warning. The HTML pages are in debian/swift-doc/usr/share/doc/swift-doc/api-ref. dh_sphinxdoc dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/base/_static/js/docs.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/base/_static/js/navigation.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/base/_static/js/bootstrap.min.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/base/_static/js/jquery-3.2.1.min.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/api-ref/_static/combobox.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/api-ref/_static/js/jquery-3.2.1.min.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/api-ref/_static/js/docs.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/api-ref/_static/js/navigation.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/api-ref/_static/js/bootstrap.min.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/swift-doc/usr/share/doc/swift-doc/api-ref/_static/api-site.js make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' debian/rules override_dh_installchangelogs make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' if [ -e /build/reproducible-path/swift-2.33.0/debian/CHANGELOG ] ; then \ dh_installchangelogs /build/reproducible-path/swift-2.33.0/debian/CHANGELOG ; \ else \ dh_installchangelogs ; \ fi make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_installman -O--buildsystem=pybuild dh_installcron -O--buildsystem=pybuild debian/rules override_dh_python3 make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh_python3 --shebang=/usr/bin/python3 I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-container-deleter I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-ring-composer I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-config I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-drive-full-checker I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-recon I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-orphans I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-dispersion-populate I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-reload I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-ring-builder-analyzer I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-oldies I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-form-signature I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-ring-builder I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-get-nodes I: dh_python3 tools:113: replacing shebang in debian/swift/usr/bin/swift-dispersion-report I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-auditor I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-replicator I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-server I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-info I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-reaper I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-server-uwsgi I: dh_python3 tools:113: replacing shebang in debian/swift-account/usr/bin/swift-account-audit I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-sharder I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-server-uwsgi I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-info I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-sync I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-auditor I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-reconciler-enqueue I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-server I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-updater I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-replicator I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-manage-shard-ranges I: dh_python3 tools:113: replacing shebang in debian/swift-container/usr/bin/swift-container-reconciler I: dh_python3 tools:113: replacing shebang in debian/swift-drive-audit/usr/bin/swift-drive-audit I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-reconstructor I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-server I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-info I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-server-uwsgi I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-auditor I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-replicator I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-relinker I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-recon-cron I: dh_python3 tools:113: replacing shebang in debian/swift-object/usr/bin/swift-object-updater I: dh_python3 tools:113: replacing shebang in debian/swift-object-expirer/usr/bin/swift-object-expirer I: dh_python3 tools:113: replacing shebang in debian/swift-proxy/usr/bin/swift-proxy-server-uwsgi I: dh_python3 tools:113: replacing shebang in debian/swift-proxy/usr/bin/swift-proxy-server make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' debian/rules override_dh_installinit make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' echo "Do nothing" Do nothing set -e ; set -x ; \ for FILE in swift-account.swift-account-auditor.init swift-account.swift-account-reaper.init swift-account.swift-account-replicator.init swift-account.swift-account.init swift-container.swift-container-auditor.init swift-container.swift-container-reconciler.init swift-container.swift-container-replicator.init swift-container.swift-container-sharder.init swift-container.swift-container-sync.init swift-container.swift-container-updater.init swift-container.swift-container.init swift-object-expirer.swift-object-expirer.init swift-object.swift-object-auditor.init swift-object.swift-object-reconstructor.init swift-object.swift-object-replicator.init swift-object.swift-object-updater.init swift-object.swift-object.init swift-proxy.swift-proxy.init ; do \ PKG=`echo $FILE | cut -d. -f1` ; \ NAME=`echo $FILE | cut -d. -f2` ; \ dh_installinit --error-handler=true -p$PKG --name=$NAME ;\ done + echo swift-account.swift-account-auditor.init + cut -d. -f1 + PKG=swift-account + cut -d. -f2 + echo swift-account.swift-account-auditor.init + NAME=swift-account-auditor + dh_installinit --error-handler=true -pswift-account --name=swift-account-auditor + echo swift-account.swift-account-reaper.init + cut -d. -f1 + PKG=swift-account + + cutecho swift-account.swift-account-reaper.init -d. -f2 + NAME=swift-account-reaper + dh_installinit --error-handler=true -pswift-account --name=swift-account-reaper + echo swift-account.swift-account-replicator.init+ cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account-replicator.init + cut -d. -f2 + NAME=swift-account-replicator + dh_installinit --error-handler=true -pswift-account --name=swift-account-replicator + echo swift-account.swift-account.init + cut -d. -f1 + PKG=swift-account + cut -d. -f2 + echo swift-account.swift-account.init + NAME=swift-account + dh_installinit --error-handler=true -pswift-account --name=swift-account + cut -d. -f1 + echo swift-container.swift-container-auditor.init + PKG=swift-container + echo swift-container.swift-container-auditor.init + cut -d. -f2 + NAME=swift-container-auditor + dh_installinit --error-handler=true -pswift-container --name=swift-container-auditor + echo swift-container.swift-container-reconciler.init + cut -d. -f1 + PKG=swift-container + cut -d. -f2 + echo swift-container.swift-container-reconciler.init + NAME=swift-container-reconciler + dh_installinit --error-handler=true -pswift-container --name=swift-container-reconciler + echo swift-container.swift-container-replicator.init + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-replicator.init + cut -d. -f2 + NAME=swift-container-replicator + dh_installinit --error-handler=true -pswift-container --name=swift-container-replicator + echo swift-container.swift-container-sharder.init+ cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-sharder.init + cut -d. -f2 + NAME=swift-container-sharder + dh_installinit --error-handler=true -pswift-container --name=swift-container-sharder + echo swift-container.swift-container-sync.init + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-sync.init + cut -d. -f2 + NAME=swift-container-sync + dh_installinit --error-handler=true -pswift-container --name=swift-container-sync + echo+ swift-container.swift-container-updater.initcut -d. -f1 + PKG=swift-container + cut -d. -f2 + echo swift-container.swift-container-updater.init + NAME=swift-container-updater + dh_installinit --error-handler=true -pswift-container --name=swift-container-updater + echo swift-container.swift-container.init+ cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container.init + cut -d. -f2 + NAME=swift-container + dh_installinit --error-handler=true -pswift-container --name=swift-container + + echo swift-object-expirer.swift-object-expirer.init cut -d. -f1 + PKG=swift-object-expirer + echo swift-object-expirer.swift-object-expirer.init + cut -d. -f2 + NAME=swift-object-expirer + dh_installinit --error-handler=true -pswift-object-expirer --name=swift-object-expirer + cut -d. -f1 + echo swift-object.swift-object-auditor.init + PKG=swift-object + echo swift-object.swift-object-auditor.init + cut -d. -f2 + NAME=swift-object-auditor + dh_installinit --error-handler=true -pswift-object --name=swift-object-auditor + echo swift-object.swift-object-reconstructor.init + cut -d. -f1 + PKG=swift-object + echo swift-object.swift-object-reconstructor.init + cut -d. -f2 + NAME=swift-object-reconstructor + dh_installinit --error-handler=true -pswift-object --name=swift-object-reconstructor + echo swift-object.swift-object-replicator.init + cut -d. -f1 + PKG=swift-object + + cutecho swift-object.swift-object-replicator.init -d. -f2 + NAME=swift-object-replicator + dh_installinit --error-handler=true -pswift-object --name=swift-object-replicator + echo swift-object.swift-object-updater.init + cut -d. -f1 + PKG=swift-object + echo swift-object.swift-object-updater.init + cut -d. -f2 + NAME=swift-object-updater + dh_installinit --error-handler=true -pswift-object --name=swift-object-updater + + echo swift-object.swift-object.init cut -d. -f1 + PKG=swift-object + + cutecho swift-object.swift-object.init -d. -f2 + NAME=swift-object + dh_installinit --error-handler=true -pswift-object --name=swift-object + echo+ swift-proxy.swift-proxy.initcut -d. -f1 + PKG=swift-proxy + echo swift-proxy.swift-proxy.init + cut -d. -f2 + NAME=swift-proxy + dh_installinit --error-handler=true -pswift-proxy --name=swift-proxy make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' debian/rules override_dh_installsystemd make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' echo "Do nothing" Do nothing set -e ; set -x ; \ for FILE in swift-account.swift-account-auditor.service swift-account.swift-account-auditor@.service swift-account.swift-account-reaper.service swift-account.swift-account-reaper@.service swift-account.swift-account-replicator.service swift-account.swift-account-replicator@.service swift-account.swift-account.service swift-container.swift-container-auditor.service swift-container.swift-container-auditor@.service swift-container.swift-container-reconciler.service swift-container.swift-container-replicator.service swift-container.swift-container-replicator@.service swift-container.swift-container-sharder.service swift-container.swift-container-sharder@.service swift-container.swift-container-sync.service swift-container.swift-container-sync@.service swift-container.swift-container-updater.service swift-container.swift-container-updater@.service swift-container.swift-container.service swift-object-expirer.swift-object-expirer.service swift-object.swift-object-auditor.service swift-object.swift-object-auditor@.service swift-object.swift-object-reconstructor.service swift-object.swift-object-reconstructor@.service swift-object.swift-object-replicator.service swift-object.swift-object-replicator@.service swift-object.swift-object-updater.service swift-object.swift-object-updater@.service swift-object.swift-object.service swift-object.swift-object@.service swift-proxy.swift-proxy.service ; do \ PKG=`echo $FILE | cut -d. -f1` ; \ NAME=`echo $FILE | cut -d. -f2` ; \ dh_installsystemd -p$PKG --name=$NAME ;\ done + echo swift-account.swift-account-auditor.service + cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account-auditor.service + cut -d. -f2 + NAME=swift-account-auditor + dh_installsystemd -pswift-account --name=swift-account-auditor + echo+ swift-account.swift-account-auditor@.service cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account-auditor@.service + cut -d. -f2 + NAME=swift-account-auditor@ + dh_installsystemd -pswift-account --name=swift-account-auditor@ + echo swift-account.swift-account-reaper.service+ cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account-reaper.service + cut -d. -f2 + NAME=swift-account-reaper + dh_installsystemd -pswift-account --name=swift-account-reaper + echo swift-account.swift-account-reaper@.service + cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account-reaper@.service + cut -d. -f2 + NAME=swift-account-reaper@ + dh_installsystemd -pswift-account --name=swift-account-reaper@ + echo swift-account.swift-account-replicator.service + cut -d. -f1 + PKG=swift-account + + echo swift-account.swift-account-replicator.service cut -d. -f2 + NAME=swift-account-replicator + dh_installsystemd -pswift-account --name=swift-account-replicator + echo swift-account.swift-account-replicator@.service + cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account-replicator@.service + cut -d. -f2 + NAME=swift-account-replicator@ + dh_installsystemd -pswift-account --name=swift-account-replicator@ + echo swift-account.swift-account.service + cut -d. -f1 + PKG=swift-account + echo swift-account.swift-account.service + cut -d. -f2 + NAME=swift-account + dh_installsystemd -pswift-account --name=swift-account + echo swift-container.swift-container-auditor.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-auditor.service + cut -d. -f2 + NAME=swift-container-auditor + dh_installsystemd -pswift-container --name=swift-container-auditor + echo swift-container.swift-container-auditor@.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-auditor@.service + cut -d. -f2 + NAME=swift-container-auditor@ + dh_installsystemd -pswift-container --name=swift-container-auditor@ + echo swift-container.swift-container-reconciler.service + cut -d. -f1 + PKG=swift-container + cut -d. -f2 + echo swift-container.swift-container-reconciler.service + NAME=swift-container-reconciler + dh_installsystemd -pswift-container --name=swift-container-reconciler + echo swift-container.swift-container-replicator.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-replicator.service + cut -d. -f2 + NAME=swift-container-replicator + dh_installsystemd -pswift-container --name=swift-container-replicator + echo swift-container.swift-container-replicator@.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-replicator@.service + cut -d. -f2 + NAME=swift-container-replicator@ + dh_installsystemd -pswift-container --name=swift-container-replicator@ + echo swift-container.swift-container-sharder.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-sharder.service + cut -d. -f2 + NAME=swift-container-sharder + dh_installsystemd -pswift-container --name=swift-container-sharder + echo swift-container.swift-container-sharder@.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-sharder@.service + cut -d. -f2 + NAME=swift-container-sharder@ + dh_installsystemd -pswift-container --name=swift-container-sharder@ + echo swift-container.swift-container-sync.service + cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-sync.service + cut -d. -f2 + NAME=swift-container-sync + dh_installsystemd -pswift-container --name=swift-container-sync + echo swift-container.swift-container-sync@.service + cut -d. -f1 + PKG=swift-container + cut -d. -f2 + echo swift-container.swift-container-sync@.service + NAME=swift-container-sync@ + dh_installsystemd -pswift-container --name=swift-container-sync@ + + echo swift-container.swift-container-updater.service cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-updater.service + cut -d. -f2 + NAME=swift-container-updater + dh_installsystemd -pswift-container --name=swift-container-updater + + echo swift-container.swift-container-updater@.service cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container-updater@.service + cut -d. -f2 + NAME=swift-container-updater@ + dh_installsystemd -pswift-container --name=swift-container-updater@ + + echo swift-container.swift-container.service cut -d. -f1 + PKG=swift-container + echo swift-container.swift-container.service + cut -d. -f2 + NAME=swift-container + dh_installsystemd -pswift-container --name=swift-container + + echo swift-object-expirer.swift-object-expirer.service cut -d. -f1 + PKG=swift-object-expirer + echo swift-object-expirer.swift-object-expirer.service + cut -d. -f2 + NAME=swift-object-expirer + dh_installsystemd -pswift-object-expirer --name=swift-object-expirer + + echocut swift-object.swift-object-auditor.service -d. -f1 + PKG=swift-object + echo swift-object.swift-object-auditor.service + cut -d. -f2 + NAME=swift-object-auditor + dh_installsystemd -pswift-object --name=swift-object-auditor + + echo swift-object.swift-object-auditor@.service cut -d. -f1 + PKG=swift-object + echo swift-object.swift-object-auditor@.service + cut -d. -f2 + NAME=swift-object-auditor@ + dh_installsystemd -pswift-object --name=swift-object-auditor@ + echo swift-object.swift-object-reconstructor.service + cut -d. -f1 + PKG=swift-object + echo+ swift-object.swift-object-reconstructor.servicecut -d. -f2 + NAME=swift-object-reconstructor + dh_installsystemd -pswift-object --name=swift-object-reconstructor + echo+ swift-object.swift-object-reconstructor@.servicecut -d. -f1 + PKG=swift-object + echo swift-object.swift-object-reconstructor@.service + cut -d. -f2 + NAME=swift-object-reconstructor@ + dh_installsystemd -pswift-object --name=swift-object-reconstructor@ + echo+ swift-object.swift-object-replicator.service cut -d. -f1 + PKG=swift-object + + echo swift-object.swift-object-replicator.service cut -d. -f2 + NAME=swift-object-replicator + dh_installsystemd -pswift-object --name=swift-object-replicator + echo swift-object.swift-object-replicator@.service + cut -d. -f1 + PKG=swift-object + echo+ swift-object.swift-object-replicator@.servicecut -d. -f2 + NAME=swift-object-replicator@ + dh_installsystemd -pswift-object --name=swift-object-replicator@ + echo swift-object.swift-object-updater.service + cut -d. -f1 + PKG=swift-object + + echo swift-object.swift-object-updater.service cut -d. -f2 + NAME=swift-object-updater + dh_installsystemd -pswift-object --name=swift-object-updater + echo swift-object.swift-object-updater@.service + cut -d. -f1 + PKG=swift-object + echo swift-object.swift-object-updater@.service + cut -d. -f2 + NAME=swift-object-updater@ + dh_installsystemd -pswift-object --name=swift-object-updater@ + + echo swift-object.swift-object.service cut -d. -f1 + PKG=swift-object + + echo swift-object.swift-object.service cut -d. -f2 + NAME=swift-object + dh_installsystemd -pswift-object --name=swift-object + echo swift-object.swift-object@.service + cut -d. -f1 + PKG=swift-object + + echo swift-object.swift-object@.service cut -d. -f2 + NAME=swift-object@ + dh_installsystemd -pswift-object --name=swift-object@ + echo+ swift-proxy.swift-proxy.servicecut -d. -f1 + PKG=swift-proxy + cut -d. -f2 + echo swift-proxy.swift-proxy.service + NAME=swift-proxy + dh_installsystemd -pswift-proxy --name=swift-proxy make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_installsystemduser -O--buildsystem=pybuild dh_installlogrotate -O--buildsystem=pybuild dh_lintian -O--buildsystem=pybuild dh_perl -O--buildsystem=pybuild dh_link -O--buildsystem=pybuild dh_strip_nondeterminism -O--buildsystem=pybuild dh_compress -O--buildsystem=pybuild dh_fixperms -O--buildsystem=pybuild debian/rules override_dh_missing make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh_missing --fail-missing make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_installdeb -O--buildsystem=pybuild debian/rules override_dh_gencontrol make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' if dpkg-vendor --derives-from ubuntu ; then \ dh_gencontrol -- -T/build/reproducible-path/swift-2.33.0/debian/ubuntu_control_vars ; \ else \ dh_gencontrol -- -T/build/reproducible-path/swift-2.33.0/debian/debian_control_vars ; \ fi dpkg-gencontrol: warning: package swift-doc: substitution variable ${sphinxdoc:Built-Using} unused, but is defined make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_md5sums -O--buildsystem=pybuild dh_builddeb -O--buildsystem=pybuild dpkg-deb: building package 'python3-swift' in '../python3-swift_2.33.0-4_all.deb'. dpkg-deb: building package 'swift' in '../swift_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-account' in '../swift-account_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-container' in '../swift-container_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-doc' in '../swift-doc_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-drive-audit' in '../swift-drive-audit_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-object' in '../swift-object_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-object-expirer' in '../swift-object-expirer_2.33.0-4_all.deb'. dpkg-deb: building package 'swift-proxy' in '../swift-proxy_2.33.0-4_all.deb'. dpkg-genbuildinfo --build=binary -O../swift_2.33.0-4_amd64.buildinfo dpkg-genchanges --build=binary -O../swift_2.33.0-4_amd64.changes dpkg-genchanges: info: binary-only upload (no source code included) dpkg-source --after-build . dpkg-source: info: using options from swift-2.33.0/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/ --extend-diff-ignore=^[.]gitreview$ dpkg-buildpackage: info: binary-only upload (no source included) dpkg-genchanges: info: not including original source code in upload I: copying local configuration I: unmounting dev/ptmx filesystem I: unmounting dev/pts filesystem I: unmounting dev/shm filesystem I: unmounting proc filesystem I: unmounting sys filesystem I: cleaning the build env I: removing directory /srv/workspace/pbuilder/3531167 and its subdirectories I: Current time: Sun Jun 2 04:34:09 -12 2024 I: pbuilder-time-stamp: 1717346049 Sun Jun 2 16:34:11 UTC 2024 I: 1st build successful. Starting 2nd build on remote node ionos5-amd64.debian.net. Sun Jun 2 16:34:11 UTC 2024 I: Preparing to do remote build '2' on ionos5-amd64.debian.net. Sun Jun 2 16:45:06 UTC 2024 I: Deleting $TMPDIR on ionos5-amd64.debian.net. Sun Jun 2 16:45:06 UTC 2024 I: swift_2.33.0-4_amd64.changes: Format: 1.8 Date: Mon, 22 Apr 2024 14:17:48 +0200 Source: swift Binary: python3-swift swift swift-account swift-container swift-doc swift-drive-audit swift-object swift-object-expirer swift-proxy Architecture: all Version: 2.33.0-4 Distribution: unstable Urgency: medium Maintainer: Debian OpenStack Changed-By: Thomas Goirand Description: python3-swift - distributed virtual object store - Python 3 libraries swift - distributed virtual object store - common files swift-account - distributed virtual object store - account server swift-container - distributed virtual object store - container server swift-doc - distributed virtual object store - documentation swift-drive-audit - distributed virtual object store - drive audit swift-object - distributed virtual object store - object server swift-object-expirer - distributed virtual object store - object-expirer swift-proxy - distributed virtual object store - proxy server Changes: swift (2.33.0-4) unstable; urgency=medium . * Removed build-depends on python3-openstack.nose-plugin. Checksums-Sha1: 972d909a24bd2074462de20f9ff30bd8d2c882ba 713268 python3-swift_2.33.0-4_all.deb c6128a3bff713d73b4890a009dc350e5791d5d89 97460 swift-account_2.33.0-4_all.deb 154dc6f57944ed1fbbdf59f4f9ef4f6cdf963b36 101680 swift-container_2.33.0-4_all.deb adef03dba6ec5531cbe5268e6ce19b2333bed106 2332148 swift-doc_2.33.0-4_all.deb 05b7991bc53457e6b258fe8e0868526e4b6def4c 77660 swift-drive-audit_2.33.0-4_all.deb d34751fee5c039d97d79d1b72bafca53b8d3bd4e 79368 swift-object-expirer_2.33.0-4_all.deb 49c56517f673b78e5c5a4523596367145218a22a 107952 swift-object_2.33.0-4_all.deb ec8883a6f7c97e9914aacf04259fdabb42f49a5c 110284 swift-proxy_2.33.0-4_all.deb 12d40e78fb29cf1e98e81b7b78fe7c2a091805c9 105600 swift_2.33.0-4_all.deb 50ebac37d9b5c07aee8c5deb6fe0036088e82234 13564 swift_2.33.0-4_amd64.buildinfo Checksums-Sha256: 519f95932a1537690113b4471df972ca9c78a15ba8fb8f9fa977b22456d3e6e7 713268 python3-swift_2.33.0-4_all.deb 83221236b788e277c0dd1e6ff12a26956043a8c06ed1eb021865c8caf15a269c 97460 swift-account_2.33.0-4_all.deb 88974741edb98e86a5c43a1180a6ef9ed664672ec1ba0ae475dc1cdbda4fb6f6 101680 swift-container_2.33.0-4_all.deb 1cc2f1e9c8646acf2549b13892e643135f0744200a6dfbe34da0e60b4a8c0840 2332148 swift-doc_2.33.0-4_all.deb 6d5bfd267b75e506627585c596e9b7e5a7d364dbcec814edba9467e568e203b5 77660 swift-drive-audit_2.33.0-4_all.deb 6e2a905234cd23e6f7b5cb55da3c044d8b78dfd0a81a8dfe57211b823d122565 79368 swift-object-expirer_2.33.0-4_all.deb 8a082c986e395eea6dd3ae69476324904bf0d62d4ac05bcf31ec949a765c53f7 107952 swift-object_2.33.0-4_all.deb 1848342b42bb5f7c10e220b1c0cd230ad9289df9592594217c2bff3847eb341d 110284 swift-proxy_2.33.0-4_all.deb c8ab87df559c9eca962c79a353b0b8d804bf71219c7b8409a7b5d228e9acd50e 105600 swift_2.33.0-4_all.deb cc02f338edb003e1822ef1489e50300f1a93fff0308b9c798fa2b1a2b14fa512 13564 swift_2.33.0-4_amd64.buildinfo Files: 4f5fadd979b087390b362a8080ac080d 713268 python optional python3-swift_2.33.0-4_all.deb 50a1abf2f3c04425daff4e26aaa3a4cb 97460 net optional swift-account_2.33.0-4_all.deb 07de511c98bddb136f7dd2a654558cb8 101680 net optional swift-container_2.33.0-4_all.deb 3a37dff5400013405161dbdb8444b50a 2332148 doc optional swift-doc_2.33.0-4_all.deb 060b3a522ee34b4dc9e2b62f391cdd83 77660 net optional swift-drive-audit_2.33.0-4_all.deb 19b7dd047a300ddfb0ac88fb2db64be2 79368 net optional swift-object-expirer_2.33.0-4_all.deb 8edc2a37d063675f2caedcd0a052952e 107952 net optional swift-object_2.33.0-4_all.deb 9f7ab8fea6e5b10e69ebb812a91ec59b 110284 net optional swift-proxy_2.33.0-4_all.deb 32276fba12c5f28f1b1931655d58c29f 105600 net optional swift_2.33.0-4_all.deb 1e10b6751c27761b87222ca412c49015 13564 net optional swift_2.33.0-4_amd64.buildinfo Sun Jun 2 16:45:06 UTC 2024 I: Preparing to do remote build '1' on ionos1-amd64.debian.net. Sun Jun 2 17:01:29 UTC 2024 I: Deleting $TMPDIR on ionos1-amd64.debian.net. I: pbuilder: network access will be disabled during build I: Current time: Sun Jun 2 04:45:08 -12 2024 I: pbuilder-time-stamp: 1717346708 I: Building the build Environment I: extracting base tarball [/var/cache/pbuilder/unstable-reproducible-base.tgz] I: copying local configuration W: --override-config is not set; not updating apt.conf Read the manpage for details. I: mounting /proc filesystem I: mounting /sys filesystem I: creating /{dev,run}/shm I: mounting /dev/pts filesystem I: redirecting /dev/ptmx to /dev/pts/ptmx I: policy-rc.d already exists I: Copying source file I: copying [swift_2.33.0-4.dsc] I: copying [./swift_2.33.0.orig.tar.xz] I: copying [./swift_2.33.0-4.debian.tar.xz] I: Extracting source gpgv: Signature made Mon Apr 22 12:28:38 2024 gpgv: using RSA key A0B1A9F3508956130E7A425CD416AD15AC6B43FE gpgv: Can't check signature: No public key dpkg-source: warning: cannot verify inline signature for ./swift_2.33.0-4.dsc: no acceptable signature found dpkg-source: info: extracting swift in swift-2.33.0 dpkg-source: info: unpacking swift_2.33.0.orig.tar.xz dpkg-source: info: unpacking swift_2.33.0-4.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying syslog_log_name.patch dpkg-source: info: applying disable_sphinxcontrib_rsvgconverter.patch dpkg-source: info: applying set-default-workers-value.patch dpkg-source: info: applying Add_tempurl_path_prefix_configuration_option.patch dpkg-source: info: applying swift-recon-only-query-object-servers-once.patch dpkg-source: info: applying Clean_up_watchdog_threads.patch dpkg-source: info: applying drive-full-checker.patch I: Not using root during the build. I: Installing the build-deps I: user script /srv/workspace/pbuilder/3586756/tmp/hooks/D02_print_environment starting I: set BUILDDIR='/build/reproducible-path' BUILDUSERGECOS='first user,first room,first work-phone,first home-phone,first other' BUILDUSERNAME='pbuilder1' BUILD_ARCH='amd64' DEBIAN_FRONTEND='noninteractive' DEB_BUILD_OPTIONS='buildinfo=+all reproducible=+all parallel=20 ' DISTRIBUTION='unstable' HOME='/root' HOST_ARCH='amd64' IFS=' ' INVOCATION_ID='0bebda793bd3484b850e239d362d2262' LANG='C' LANGUAGE='en_US:en' LC_ALL='C' MAIL='/var/mail/root' OPTIND='1' PATH='/usr/sbin:/usr/bin:/sbin:/bin:/usr/games' PBCURRENTCOMMANDLINEOPERATION='build' PBUILDER_OPERATION='build' PBUILDER_PKGDATADIR='/usr/share/pbuilder' PBUILDER_PKGLIBDIR='/usr/lib/pbuilder' PBUILDER_SYSCONFDIR='/etc' PPID='3586756' PS1='# ' PS2='> ' PS4='+ ' PWD='/' SHELL='/bin/bash' SHLVL='2' SUDO_COMMAND='/usr/bin/timeout -k 18.1h 18h /usr/bin/ionice -c 3 /usr/bin/nice /usr/sbin/pbuilder --build --configfile /srv/reproducible-results/rbuild-debian/r-b-build.CLcTtdy7/pbuilderrc_TlYV --distribution unstable --hookdir /etc/pbuilder/first-build-hooks --debbuildopts -b --basetgz /var/cache/pbuilder/unstable-reproducible-base.tgz --buildresult /srv/reproducible-results/rbuild-debian/r-b-build.CLcTtdy7/b1 --logfile b1/build.log swift_2.33.0-4.dsc' SUDO_GID='110' SUDO_UID='105' SUDO_USER='jenkins' TERM='unknown' TZ='/usr/share/zoneinfo/Etc/GMT+12' USER='root' _='/usr/bin/systemd-run' http_proxy='http://46.16.76.132:3128' I: uname -a Linux ionos1-amd64 6.1.0-21-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64 GNU/Linux I: ls -l /bin lrwxrwxrwx 1 root root 7 Jun 1 07:43 /bin -> usr/bin I: user script /srv/workspace/pbuilder/3586756/tmp/hooks/D02_print_environment finished -> Attempting to satisfy build-dependencies -> Creating pbuilder-satisfydepends-dummy package Package: pbuilder-satisfydepends-dummy Version: 0.invalid.0 Architecture: amd64 Maintainer: Debian Pbuilder Team Description: Dummy package to satisfy dependencies with aptitude - created by pbuilder This package was created automatically by pbuilder to satisfy the build-dependencies of the package being currently built. Depends: debhelper-compat (= 12), dh-exec, dh-python, openstack-pkg-tools, python3-all, python3-pbr, python3-setuptools, python3-sphinx, python3-boto3, python3-cffi, python3-coverage, python3-cryptography, python3-eventlet, python3-fixtures, python3-greenlet, python3-keystoneclient, python3-keystonemiddleware, python3-lxml, python3-mock, python3-openssl, python3-openstackclient, python3-openstackdocstheme, python3-os-api-ref, python3-os-testr, python3-oslo.config, python3-oslo.log, python3-oslosphinx, python3-pastedeploy, python3-pyeclib, python3-requests, python3-requests-mock, python3-six, python3-stestr, python3-swiftclient, python3-xattr, subunit dpkg-deb: building package 'pbuilder-satisfydepends-dummy' in '/tmp/satisfydepends-aptitude/pbuilder-satisfydepends-dummy.deb'. Selecting previously unselected package pbuilder-satisfydepends-dummy. (Reading database ... 19719 files and directories currently installed.) Preparing to unpack .../pbuilder-satisfydepends-dummy.deb ... Unpacking pbuilder-satisfydepends-dummy (0.invalid.0) ... dpkg: pbuilder-satisfydepends-dummy: dependency problems, but configuring anyway as you requested: pbuilder-satisfydepends-dummy depends on debhelper-compat (= 12); however: Package debhelper-compat is not installed. pbuilder-satisfydepends-dummy depends on dh-exec; however: Package dh-exec is not installed. pbuilder-satisfydepends-dummy depends on dh-python; however: Package dh-python is not installed. pbuilder-satisfydepends-dummy depends on openstack-pkg-tools; however: Package openstack-pkg-tools is not installed. pbuilder-satisfydepends-dummy depends on python3-all; however: Package python3-all is not installed. pbuilder-satisfydepends-dummy depends on python3-pbr; however: Package python3-pbr is not installed. pbuilder-satisfydepends-dummy depends on python3-setuptools; however: Package python3-setuptools is not installed. pbuilder-satisfydepends-dummy depends on python3-sphinx; however: Package python3-sphinx is not installed. pbuilder-satisfydepends-dummy depends on python3-boto3; however: Package python3-boto3 is not installed. pbuilder-satisfydepends-dummy depends on python3-cffi; however: Package python3-cffi is not installed. pbuilder-satisfydepends-dummy depends on python3-coverage; however: Package python3-coverage is not installed. pbuilder-satisfydepends-dummy depends on python3-cryptography; however: Package python3-cryptography is not installed. pbuilder-satisfydepends-dummy depends on python3-eventlet; however: Package python3-eventlet is not installed. pbuilder-satisfydepends-dummy depends on python3-fixtures; however: Package python3-fixtures is not installed. pbuilder-satisfydepends-dummy depends on python3-greenlet; however: Package python3-greenlet is not installed. pbuilder-satisfydepends-dummy depends on python3-keystoneclient; however: Package python3-keystoneclient is not installed. pbuilder-satisfydepends-dummy depends on python3-keystonemiddleware; however: Package python3-keystonemiddleware is not installed. pbuilder-satisfydepends-dummy depends on python3-lxml; however: Package python3-lxml is not installed. pbuilder-satisfydepends-dummy depends on python3-mock; however: Package python3-mock is not installed. pbuilder-satisfydepends-dummy depends on python3-openssl; however: Package python3-openssl is not installed. pbuilder-satisfydepends-dummy depends on python3-openstackclient; however: Package python3-openstackclient is not installed. pbuilder-satisfydepends-dummy depends on python3-openstackdocstheme; however: Package python3-openstackdocstheme is not installed. pbuilder-satisfydepends-dummy depends on python3-os-api-ref; however: Package python3-os-api-ref is not installed. pbuilder-satisfydepends-dummy depends on python3-os-testr; however: Package python3-os-testr is not installed. pbuilder-satisfydepends-dummy depends on python3-oslo.config; however: Package python3-oslo.config is not installed. pbuilder-satisfydepends-dummy depends on python3-oslo.log; however: Package python3-oslo.log is not installed. pbuilder-satisfydepends-dummy depends on python3-oslosphinx; however: Package python3-oslosphinx is not installed. pbuilder-satisfydepends-dummy depends on python3-pastedeploy; however: Package python3-pastedeploy is not installed. pbuilder-satisfydepends-dummy depends on python3-pyeclib; however: Package python3-pyeclib is not installed. pbuilder-satisfydepends-dummy depends on python3-requests; however: Package python3-requests is not installed. pbuilder-satisfydepends-dummy depends on python3-requests-mock; however: Package python3-requests-mock is not installed. pbuilder-satisfydepends-dummy depends on python3-six; however: Package python3-six is not installed. pbuilder-satisfydepends-dummy depends on python3-stestr; however: Package python3-stestr is not installed. pbuilder-satisfydepends-dummy depends on python3-swiftclient; however: Package python3-swiftclient is not installed. pbuilder-satisfydepends-dummy depends on python3-xattr; however: Package python3-xattr is not installed. pbuilder-satisfydepends-dummy depends on subunit; however: Package subunit is not installed. Setting up pbuilder-satisfydepends-dummy (0.invalid.0) ... Reading package lists... Building dependency tree... Reading state information... Initializing package states... Writing extended state information... Building tag database... pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) The following NEW packages will be installed: autoconf{a} automake{a} autopoint{a} autotools-dev{a} bsdextrautils{a} ca-certificates{a} debhelper{a} dh-autoreconf{a} dh-exec{a} dh-python{a} dh-strip-nondeterminism{a} docutils-common{a} dwz{a} file{a} fonts-font-awesome{a} fonts-glyphicons-halflings{a} gettext{a} gettext-base{a} groff-base{a} ieee-data{a} intltool-debian{a} jq{a} libarchive-zip-perl{a} libbsd0{a} libdebhelper-perl{a} libelf1t64{a} liberasurecode1{a} libexpat1{a} libfile-stripnondeterminism-perl{a} libice6{a} libicu72{a} libjq1{a} libjs-bootstrap{a} libjs-jquery{a} libjs-sphinxdoc{a} libjs-underscore{a} libjson-perl{a} libmagic-mgc{a} libmagic1t64{a} libonig5{a} libpipeline1{a} libpsl5t64{a} libpython3-stdlib{a} libpython3.11-minimal{a} libpython3.11-stdlib{a} libpython3.12-minimal{a} libpython3.12-stdlib{a} libreadline8t64{a} libsm6{a} libsubunit-perl{a} libtool{a} libuchardet0{a} libx11-6{a} libx11-data{a} libxau6{a} libxcb1{a} libxdmcp6{a} libxext6{a} libxml2{a} libxmu6{a} libxslt1.1{a} libxt6t64{a} libyaml-0-2{a} m4{a} man-db{a} media-types{a} netbase{a} openssl{a} openstack-pkg-tools{a} po-debconf{a} pycadf-common{a} python-babel-localedata{a} python-os-api-ref-common{a} python-oslosphinx-common{a} python3{a} python3-alabaster{a} python3-all{a} python3-attr{a} python3-autopage{a} python3-babel{a} python3-binary-memcached{a} python3-blinker{a} python3-boto3{a} python3-botocore{a} python3-certifi{a} python3-cffi{a} python3-cffi-backend{a} python3-chardet{a} python3-charset-normalizer{a} python3-cinderclient{a} python3-cliff{a} python3-cmd2{a} python3-coverage{a} python3-cryptography{a} python3-dateutil{a} python3-debtcollector{a} python3-decorator{a} python3-deprecation{a} python3-distutils{a} python3-dnspython{a} python3-docutils{a} python3-dogpile.cache{a} python3-dulwich{a} python3-eventlet{a} python3-extras{a} python3-fixtures{a} python3-glanceclient{a} python3-greenlet{a} python3-hatch-vcs{a} python3-hatchling{a} python3-idna{a} python3-imagesize{a} python3-importlib-metadata{a} python3-iso8601{a} python3-jaraco.classes{a} python3-jaraco.context{a} python3-jaraco.functools{a} python3-jeepney{a} python3-jinja2{a} python3-jmespath{a} python3-json-pointer{a} python3-jsonpatch{a} python3-jsonschema{a} python3-jsonschema-specifications{a} python3-jwt{a} python3-keyring{a} python3-keystoneauth1{a} python3-keystoneclient{a} python3-keystonemiddleware{a} python3-lib2to3{a} python3-lxml{a} python3-mako{a} python3-markupsafe{a} python3-memcache{a} python3-minimal{a} python3-mock{a} python3-more-itertools{a} python3-msgpack{a} python3-netaddr{a} python3-netifaces{a} python3-neutronclient{a} python3-novaclient{a} python3-oauthlib{a} python3-openssl{a} python3-openstackclient{a} python3-openstackdocstheme{a} python3-openstacksdk{a} python3-os-api-ref{a} python3-os-client-config{a} python3-os-service-types{a} python3-os-testr{a} python3-osc-lib{a} python3-oslo.cache{a} python3-oslo.config{a} python3-oslo.context{a} python3-oslo.i18n{a} python3-oslo.log{a} python3-oslo.serialization{a} python3-oslo.utils{a} python3-oslosphinx{a} python3-packaging{a} python3-paste{a} python3-pastedeploy{a} python3-pastedeploy-tpl{a} python3-pathspec{a} python3-pbr{a} python3-pip{a} python3-pkg-resources{a} python3-platformdirs{a} python3-pluggy{a} python3-ply{a} python3-prettytable{a} python3-pyasyncore{a} python3-pycadf{a} python3-pycparser{a} python3-pyeclib{a} python3-pygments{a} python3-pyinotify{a} python3-pyparsing{a} python3-pyperclip{a} python3-referencing{a} python3-requests{a} python3-requests-mock{a} python3-requestsexceptions{a} python3-rfc3986{a} python3-roman{a} python3-rpds-py{a} python3-s3transfer{a} python3-secretstorage{a} python3-setuptools{a} python3-setuptools-scm{a} python3-simplejson{a} python3-six{a} python3-snowballstemmer{a} python3-sphinx{a} python3-stestr{a} python3-stevedore{a} python3-subunit{a} python3-swiftclient{a} python3-tempita{a} python3-testtools{a} python3-tomlkit{a} python3-trove-classifiers{a} python3-typing-extensions{a} python3-tz{a} python3-uhashring{a} python3-urllib3{a} python3-voluptuous{a} python3-warlock{a} python3-wcwidth{a} python3-webob{a} python3-wheel{a} python3-wrapt{a} python3-xattr{a} python3-yaml{a} python3-zipp{a} python3.11{a} python3.11-minimal{a} python3.12{a} python3.12-minimal{a} readline-common{a} sensible-utils{a} sgml-base{a} sphinx-common{a} subunit{a} tzdata{a} wget{a} x11-common{a} xclip{a} xml-core{a} The following packages are RECOMMENDED but will NOT be installed: autopkgtest javascript-common libarchive-cpio-perl libjson-xs-perl libltdl-dev libmail-sendmail-perl libpaper-utils madison-lite pristine-tar publicsuffix python-openstackclient-doc python3-bs4 python3-cssselect python3-dev python3-fastimport python3-h2 python3-html5lib python3-httpcore python3-httpx python3-pastescript python3-pil python3-rfc3987 python3-uritemplate python3-webcolors xauth 0 packages upgraded, 230 newly installed, 0 to remove and 0 not upgraded. Need to get 69.8 MB of archives. After unpacking 379 MB will be used. Writing extended state information... Get: 1 http://deb.debian.org/debian unstable/main amd64 libpython3.11-minimal amd64 3.11.9-1 [817 kB] Get: 2 http://deb.debian.org/debian unstable/main amd64 libexpat1 amd64 2.6.2-1 [103 kB] Get: 3 http://deb.debian.org/debian unstable/main amd64 python3.11-minimal amd64 3.11.9-1 [1879 kB] Get: 4 http://deb.debian.org/debian unstable/main amd64 python3-minimal amd64 3.11.8-1 [26.3 kB] Get: 5 http://deb.debian.org/debian unstable/main amd64 media-types all 10.1.0 [26.9 kB] Get: 6 http://deb.debian.org/debian unstable/main amd64 netbase all 6.4 [12.8 kB] Get: 7 http://deb.debian.org/debian unstable/main amd64 tzdata all 2024a-4 [255 kB] Get: 8 http://deb.debian.org/debian unstable/main amd64 readline-common all 8.2-4 [69.3 kB] Get: 9 http://deb.debian.org/debian unstable/main amd64 libreadline8t64 amd64 8.2-4 [167 kB] Get: 10 http://deb.debian.org/debian unstable/main amd64 libpython3.11-stdlib amd64 3.11.9-1 [1792 kB] Get: 11 http://deb.debian.org/debian unstable/main amd64 python3.11 amd64 3.11.9-1 [602 kB] Get: 12 http://deb.debian.org/debian unstable/main amd64 libpython3-stdlib amd64 3.11.8-1 [9332 B] Get: 13 http://deb.debian.org/debian unstable/main amd64 python3 amd64 3.11.8-1 [27.4 kB] Get: 14 http://deb.debian.org/debian unstable/main amd64 libpython3.12-minimal amd64 3.12.3-1 [809 kB] Get: 15 http://deb.debian.org/debian unstable/main amd64 python3.12-minimal amd64 3.12.3-1 [2139 kB] Get: 16 http://deb.debian.org/debian unstable/main amd64 sgml-base all 1.31 [15.4 kB] Get: 17 http://deb.debian.org/debian unstable/main amd64 sensible-utils all 0.0.22 [22.4 kB] Get: 18 http://deb.debian.org/debian unstable/main amd64 openssl amd64 3.2.1-3 [1360 kB] Get: 19 http://deb.debian.org/debian unstable/main amd64 ca-certificates all 20240203 [158 kB] Get: 20 http://deb.debian.org/debian unstable/main amd64 libmagic-mgc amd64 1:5.45-3 [314 kB] Get: 21 http://deb.debian.org/debian unstable/main amd64 libmagic1t64 amd64 1:5.45-3 [105 kB] Get: 22 http://deb.debian.org/debian unstable/main amd64 file amd64 1:5.45-3 [42.9 kB] Get: 23 http://deb.debian.org/debian unstable/main amd64 gettext-base amd64 0.21-14+b1 [161 kB] Get: 24 http://deb.debian.org/debian unstable/main amd64 libuchardet0 amd64 0.0.8-1+b1 [68.8 kB] Get: 25 http://deb.debian.org/debian unstable/main amd64 groff-base amd64 1.23.0-4 [1180 kB] Get: 26 http://deb.debian.org/debian unstable/main amd64 bsdextrautils amd64 2.40.1-4 [95.7 kB] Get: 27 http://deb.debian.org/debian unstable/main amd64 libpipeline1 amd64 1.5.7-2 [38.0 kB] Get: 28 http://deb.debian.org/debian unstable/main amd64 man-db amd64 2.12.1-1 [1411 kB] Get: 29 http://deb.debian.org/debian unstable/main amd64 libpsl5t64 amd64 0.21.2-1.1 [56.8 kB] Get: 30 http://deb.debian.org/debian unstable/main amd64 wget amd64 1.24.5-1 [979 kB] Get: 31 http://deb.debian.org/debian unstable/main amd64 m4 amd64 1.4.19-4 [287 kB] Get: 32 http://deb.debian.org/debian unstable/main amd64 autoconf all 2.71-3 [332 kB] Get: 33 http://deb.debian.org/debian unstable/main amd64 autotools-dev all 20220109.1 [51.6 kB] Get: 34 http://deb.debian.org/debian unstable/main amd64 automake all 1:1.16.5-1.3 [823 kB] Get: 35 http://deb.debian.org/debian unstable/main amd64 autopoint all 0.21-14 [496 kB] Get: 36 http://deb.debian.org/debian unstable/main amd64 libdebhelper-perl all 13.15.3 [88.0 kB] Get: 37 http://deb.debian.org/debian unstable/main amd64 libtool all 2.4.7-7 [517 kB] Get: 38 http://deb.debian.org/debian unstable/main amd64 dh-autoreconf all 20 [17.1 kB] Get: 39 http://deb.debian.org/debian unstable/main amd64 libarchive-zip-perl all 1.68-1 [104 kB] Get: 40 http://deb.debian.org/debian unstable/main amd64 libfile-stripnondeterminism-perl all 1.14.0-1 [19.5 kB] Get: 41 http://deb.debian.org/debian unstable/main amd64 dh-strip-nondeterminism all 1.14.0-1 [8448 B] Get: 42 http://deb.debian.org/debian unstable/main amd64 libelf1t64 amd64 0.191-1+b1 [189 kB] Get: 43 http://deb.debian.org/debian unstable/main amd64 dwz amd64 0.15-1+b1 [110 kB] Get: 44 http://deb.debian.org/debian unstable/main amd64 libicu72 amd64 72.1-4+b1 [9395 kB] Get: 45 http://deb.debian.org/debian unstable/main amd64 libxml2 amd64 2.12.7+dfsg-2 [670 kB] Get: 46 http://deb.debian.org/debian unstable/main amd64 gettext amd64 0.21-14+b1 [1301 kB] Get: 47 http://deb.debian.org/debian unstable/main amd64 intltool-debian all 0.35.0+20060710.6 [22.9 kB] Get: 48 http://deb.debian.org/debian unstable/main amd64 po-debconf all 1.0.21+nmu1 [248 kB] Get: 49 http://deb.debian.org/debian unstable/main amd64 debhelper all 13.15.3 [901 kB] Get: 50 http://deb.debian.org/debian unstable/main amd64 dh-exec amd64 0.30 [25.6 kB] Get: 51 http://deb.debian.org/debian unstable/main amd64 python3-pkg-resources all 68.1.2-2 [241 kB] Get: 52 http://deb.debian.org/debian unstable/main amd64 python3-lib2to3 all 3.12.3-3.1 [77.7 kB] Get: 53 http://deb.debian.org/debian unstable/main amd64 python3-distutils all 3.12.3-3.1 [131 kB] Get: 54 http://deb.debian.org/debian unstable/main amd64 python3-setuptools all 68.1.2-2 [468 kB] Get: 55 http://deb.debian.org/debian unstable/main amd64 dh-python all 6.20240422 [107 kB] Get: 56 http://deb.debian.org/debian unstable/main amd64 xml-core all 0.19 [20.1 kB] Get: 57 http://deb.debian.org/debian unstable/main amd64 docutils-common all 0.20.1+dfsg-3 [128 kB] Get: 58 http://deb.debian.org/debian unstable/main amd64 fonts-font-awesome all 5.0.10+really4.7.0~dfsg-4.1 [517 kB] Get: 59 http://deb.debian.org/debian unstable/main amd64 fonts-glyphicons-halflings all 1.009~3.4.1+dfsg-3 [162 kB] Get: 60 http://deb.debian.org/debian unstable/main amd64 ieee-data all 20220827.1 [2029 kB] Get: 61 http://deb.debian.org/debian unstable/main amd64 libonig5 amd64 6.9.9-1 [189 kB] Get: 62 http://deb.debian.org/debian unstable/main amd64 libjq1 amd64 1.7.1-3 [163 kB] Get: 63 http://deb.debian.org/debian unstable/main amd64 jq amd64 1.7.1-3 [77.6 kB] Get: 64 http://deb.debian.org/debian unstable/main amd64 libbsd0 amd64 0.12.2-1 [131 kB] Get: 65 http://deb.debian.org/debian unstable/main amd64 liberasurecode1 amd64 1.6.3-1 [41.9 kB] Get: 66 http://deb.debian.org/debian unstable/main amd64 x11-common all 1:7.7+23 [252 kB] Get: 67 http://deb.debian.org/debian unstable/main amd64 libice6 amd64 2:1.0.10-1+b1 [56.7 kB] Get: 68 http://deb.debian.org/debian unstable/main amd64 libjs-bootstrap all 3.4.1+dfsg-3 [172 kB] Get: 69 http://deb.debian.org/debian unstable/main amd64 libjs-jquery all 3.6.1+dfsg+~3.5.14-1 [326 kB] Get: 70 http://deb.debian.org/debian unstable/main amd64 libjs-underscore all 1.13.4~dfsg+~1.11.4-3 [116 kB] Get: 71 http://deb.debian.org/debian unstable/main amd64 libjs-sphinxdoc all 7.2.6-8 [150 kB] Get: 72 http://deb.debian.org/debian unstable/main amd64 libjson-perl all 4.10000-1 [87.5 kB] Get: 73 http://deb.debian.org/debian unstable/main amd64 libpython3.12-stdlib amd64 3.12.3-1 [1951 kB] Get: 74 http://deb.debian.org/debian unstable/main amd64 libsm6 amd64 2:1.2.3-1+b1 [33.6 kB] Get: 75 http://deb.debian.org/debian unstable/main amd64 libsubunit-perl all 1.4.2-3 [5724 B] Get: 76 http://deb.debian.org/debian unstable/main amd64 libxau6 amd64 1:1.0.9-1+b1 [18.1 kB] Get: 77 http://deb.debian.org/debian unstable/main amd64 libxdmcp6 amd64 1:1.1.2-3+b1 [24.3 kB] Get: 78 http://deb.debian.org/debian unstable/main amd64 libxcb1 amd64 1.17.0-2 [144 kB] Get: 79 http://deb.debian.org/debian unstable/main amd64 libx11-data all 2:1.8.7-1 [328 kB] Get: 80 http://deb.debian.org/debian unstable/main amd64 libx11-6 amd64 2:1.8.7-1+b1 [799 kB] Get: 81 http://deb.debian.org/debian unstable/main amd64 libxext6 amd64 2:1.3.4-1+b1 [52.9 kB] Get: 82 http://deb.debian.org/debian unstable/main amd64 libxt6t64 amd64 1:1.2.1-1.2 [187 kB] Get: 83 http://deb.debian.org/debian unstable/main amd64 libxmu6 amd64 2:1.1.3-3+b2 [58.7 kB] Get: 84 http://deb.debian.org/debian unstable/main amd64 libxslt1.1 amd64 1.1.35-1+b1 [232 kB] Get: 85 http://deb.debian.org/debian unstable/main amd64 libyaml-0-2 amd64 0.2.5-1+b1 [52.6 kB] Get: 86 http://deb.debian.org/debian unstable/main amd64 python3-wheel all 0.43.0-1 [52.5 kB] Get: 87 http://deb.debian.org/debian unstable/main amd64 python3-pip all 24.0+dfsg-2 [1517 kB] Get: 88 http://deb.debian.org/debian unstable/main amd64 openstack-pkg-tools all 126 [94.2 kB] Get: 89 http://deb.debian.org/debian unstable/main amd64 pycadf-common all 3.1.1-3 [4492 B] Get: 90 http://deb.debian.org/debian unstable/main amd64 python-babel-localedata all 2.14.0-1 [5701 kB] Get: 91 http://deb.debian.org/debian unstable/main amd64 python-os-api-ref-common all 2.3.0+ds1-3 [9548 B] Get: 92 http://deb.debian.org/debian unstable/main amd64 python-oslosphinx-common all 4.18.0-5 [30.0 kB] Get: 93 http://deb.debian.org/debian unstable/main amd64 python3-alabaster all 0.7.12-1 [20.8 kB] Get: 94 http://deb.debian.org/debian unstable/main amd64 python3.12 amd64 3.12.3-1 [659 kB] Get: 95 http://deb.debian.org/debian unstable/main amd64 python3-all amd64 3.11.8-1 [1056 B] Get: 96 http://deb.debian.org/debian unstable/main amd64 python3-typing-extensions all 4.12.0-1 [72.1 kB] Get: 97 http://deb.debian.org/debian unstable/main amd64 python3-more-itertools all 10.2.0-1 [59.9 kB] Get: 98 http://deb.debian.org/debian unstable/main amd64 python3-zipp all 1.0.0-6 [6696 B] Get: 99 http://deb.debian.org/debian unstable/main amd64 python3-importlib-metadata all 7.1.0-1 [19.2 kB] Get: 100 http://deb.debian.org/debian unstable/main amd64 python3-attr all 23.2.0-2 [65.5 kB] Get: 101 http://deb.debian.org/debian unstable/main amd64 python3-autopage all 0.4.0-3 [12.7 kB] Get: 102 http://deb.debian.org/debian unstable/main amd64 python3-tz all 2024.1-2 [30.9 kB] Get: 103 http://deb.debian.org/debian unstable/main amd64 python3-babel all 2.14.0-1 [111 kB] Get: 104 http://deb.debian.org/debian unstable/main amd64 python3-six all 1.16.0-6 [16.3 kB] Get: 105 http://deb.debian.org/debian unstable/main amd64 python3-memcache all 1.62-1 [26.0 kB] Get: 106 http://deb.debian.org/debian unstable/main amd64 python3-uhashring all 2.3-2 [12.3 kB] Get: 107 http://deb.debian.org/debian unstable/main amd64 python3-binary-memcached all 0.31.2+dfsg1-2 [13.7 kB] Get: 108 http://deb.debian.org/debian unstable/main amd64 python3-blinker all 1.8.2-1 [13.1 kB] Get: 109 http://deb.debian.org/debian unstable/main amd64 python3-dateutil all 2.9.0-2 [79.4 kB] Get: 110 http://deb.debian.org/debian unstable/main amd64 python3-jmespath all 1.0.1-1 [21.1 kB] Get: 111 http://deb.debian.org/debian unstable/main amd64 python3-urllib3 all 1.26.18-2 [116 kB] Get: 112 http://deb.debian.org/debian unstable/main amd64 python3-certifi all 2023.11.17-1 [155 kB] Get: 113 http://deb.debian.org/debian unstable/main amd64 python3-charset-normalizer all 3.3.2-1 [51.6 kB] Get: 114 http://deb.debian.org/debian unstable/main amd64 python3-idna all 3.6-2 [37.0 kB] Get: 115 http://deb.debian.org/debian unstable/main amd64 python3-chardet all 5.2.0+dfsg-1 [107 kB] Get: 116 http://deb.debian.org/debian unstable/main amd64 python3-requests all 2.31.0+dfsg-2 [68.7 kB] Get: 117 http://deb.debian.org/debian unstable/main amd64 python3-botocore all 1.34.46+repack-1 [6464 kB] Get: 118 http://deb.debian.org/debian unstable/main amd64 python3-s3transfer all 0.10.1-1 [55.6 kB] Get: 119 http://deb.debian.org/debian unstable/main amd64 python3-boto3 all 1.34.46+dfsg-1 [269 kB] Get: 120 http://deb.debian.org/debian unstable/main amd64 python3-cffi-backend amd64 1.16.0-2+b2 [112 kB] Get: 121 http://deb.debian.org/debian unstable/main amd64 python3-ply all 3.11-6 [63.7 kB] Get: 122 http://deb.debian.org/debian unstable/main amd64 python3-pycparser all 2.22-1 [78.0 kB] Get: 123 http://deb.debian.org/debian unstable/main amd64 python3-cffi all 1.16.0-2 [88.5 kB] Get: 124 http://deb.debian.org/debian unstable/main amd64 python3-iso8601 all 1.0.2-1 [15.7 kB] Get: 125 http://deb.debian.org/debian unstable/main amd64 python3-lxml amd64 5.2.1-1 [1995 kB] Get: 126 http://deb.debian.org/debian unstable/main amd64 python3-cryptography amd64 42.0.5-2 [1186 kB] Get: 127 http://deb.debian.org/debian unstable/main amd64 python3-jwt all 2.7.0-1 [29.7 kB] Get: 128 http://deb.debian.org/debian unstable/main amd64 python3-oauthlib all 3.2.2-1 [95.4 kB] Get: 129 http://deb.debian.org/debian unstable/main amd64 python3-pbr all 6.0.0-1 [55.7 kB] Get: 130 http://deb.debian.org/debian unstable/main amd64 python3-os-service-types all 1.7.0-3 [12.5 kB] Get: 131 http://deb.debian.org/debian unstable/main amd64 python3-stevedore all 5.2.0-2 [20.8 kB] Get: 132 http://deb.debian.org/debian unstable/main amd64 python3-keystoneauth1 all 5.6.0-2 [155 kB] Get: 133 http://deb.debian.org/debian unstable/main amd64 python3-oslo.i18n all 6.3.0-2 [19.1 kB] Get: 134 http://deb.debian.org/debian unstable/main amd64 python3-wrapt amd64 1.15.0-2+b1 [49.3 kB] Get: 135 http://deb.debian.org/debian unstable/main amd64 python3-debtcollector all 3.0.0-2 [13.1 kB] Get: 136 http://deb.debian.org/debian unstable/main amd64 python3-netaddr all 0.10.1-1 [315 kB] Get: 137 http://deb.debian.org/debian unstable/main amd64 python3-netifaces amd64 0.11.0-2+b2 [19.4 kB] Get: 138 http://deb.debian.org/debian unstable/main amd64 python3-packaging all 24.0-1 [45.5 kB] Get: 139 http://deb.debian.org/debian unstable/main amd64 python3-pyparsing all 3.1.2-1 [146 kB] Get: 140 http://deb.debian.org/debian unstable/main amd64 python3-yaml amd64 6.0.1-2 [177 kB] Get: 141 http://deb.debian.org/debian unstable/main amd64 python3-oslo.utils all 7.1.0-2 [59.7 kB] Get: 142 http://deb.debian.org/debian unstable/main amd64 python3-wcwidth all 0.2.13+dfsg1-1 [24.9 kB] Get: 143 http://deb.debian.org/debian unstable/main amd64 python3-prettytable all 3.6.0-1 [36.0 kB] Get: 144 http://deb.debian.org/debian unstable/main amd64 python3-cinderclient all 1:9.5.0-2 [134 kB] Get: 145 http://deb.debian.org/debian unstable/main amd64 xclip amd64 0.13-4 [21.3 kB] Get: 146 http://deb.debian.org/debian unstable/main amd64 python3-pyperclip all 1.8.2-2 [13.2 kB] Get: 147 http://deb.debian.org/debian unstable/main amd64 python3-cmd2 all 2.4.3+ds-2 [144 kB] Get: 148 http://deb.debian.org/debian unstable/main amd64 python3-roman all 3.3-3 [9880 B] Get: 149 http://deb.debian.org/debian unstable/main amd64 python3-docutils all 0.20.1+dfsg-3 [389 kB] Get: 150 http://deb.debian.org/debian unstable/main amd64 python3-cliff all 4.6.0-2 [42.3 kB] Get: 151 http://deb.debian.org/debian unstable/main amd64 python3-coverage amd64 7.2.7+dfsg1-1+b1 [170 kB] Get: 152 http://deb.debian.org/debian unstable/main amd64 python3-decorator all 5.1.1-5 [15.1 kB] Get: 153 http://deb.debian.org/debian unstable/main amd64 python3-deprecation all 2.1.0-3 [14.5 kB] Get: 154 http://deb.debian.org/debian unstable/main amd64 python3-dnspython all 2.6.1-1 [161 kB] Get: 155 http://deb.debian.org/debian unstable/main amd64 python3-markupsafe amd64 2.1.5-1 [14.5 kB] Get: 156 http://deb.debian.org/debian unstable/main amd64 python3-mako all 1.3.2-1 [82.4 kB] Get: 157 http://deb.debian.org/debian unstable/main amd64 python3-dogpile.cache all 1.3.1-1 [45.0 kB] Get: 158 http://deb.debian.org/debian unstable/main amd64 python3-dulwich amd64 0.21.6-1+b1 [328 kB] Get: 159 http://deb.debian.org/debian unstable/main amd64 python3-greenlet amd64 3.0.1-3 [177 kB] Get: 160 http://deb.debian.org/debian unstable/main amd64 python3-eventlet all 0.35.1-1 [175 kB] Get: 161 http://deb.debian.org/debian unstable/main amd64 python3-extras all 1.0.0-5 [7536 B] Get: 162 http://deb.debian.org/debian unstable/main amd64 python3-testtools all 2.7.1-3 [121 kB] Get: 163 http://deb.debian.org/debian unstable/main amd64 python3-fixtures all 4.1.0-2 [34.1 kB] Get: 164 http://deb.debian.org/debian unstable/main amd64 python3-openssl all 24.1.0-1 [51.9 kB] Get: 165 http://deb.debian.org/debian unstable/main amd64 python3-json-pointer all 2.4-2 [14.6 kB] Get: 166 http://deb.debian.org/debian unstable/main amd64 python3-jsonpatch all 1.32-3 [11.9 kB] Get: 167 http://deb.debian.org/debian unstable/main amd64 python3-pathspec all 0.12.1-1 [28.1 kB] Get: 168 http://deb.debian.org/debian unstable/main amd64 python3-pluggy all 1.5.0-1 [26.9 kB] Get: 169 http://deb.debian.org/debian unstable/main amd64 python3-trove-classifiers all 2024.1.31-1 [9968 B] Get: 170 http://deb.debian.org/debian unstable/main amd64 python3-hatchling all 1.24.2-1 [53.2 kB] Get: 171 http://deb.debian.org/debian unstable/main amd64 python3-setuptools-scm all 8.0.4-2 [39.5 kB] Get: 172 http://deb.debian.org/debian unstable/main amd64 python3-hatch-vcs all 0.4.0-1 [8336 B] Get: 173 http://deb.debian.org/debian unstable/main amd64 python3-rpds-py amd64 0.12.0-3 [276 kB] Get: 174 http://deb.debian.org/debian unstable/main amd64 python3-referencing all 0.31.0-1 [22.2 kB] Get: 175 http://deb.debian.org/debian unstable/main amd64 python3-jsonschema-specifications all 2023.12.1-1 [8976 B] Get: 176 http://deb.debian.org/debian unstable/main amd64 python3-jsonschema all 4.19.2-3 [71.3 kB] Get: 177 http://deb.debian.org/debian unstable/main amd64 python3-warlock all 2.0.1-4 [9224 B] Get: 178 http://deb.debian.org/debian unstable/main amd64 python3-glanceclient all 1:4.5.0-2 [129 kB] Get: 179 http://deb.debian.org/debian unstable/main amd64 python3-imagesize all 1.4.1-1 [6688 B] Get: 180 http://deb.debian.org/debian unstable/main amd64 python3-jaraco.classes all 3.2.1-4 [7232 B] Get: 181 http://deb.debian.org/debian unstable/main amd64 python3-jaraco.context all 4.3.0-1 [7616 B] Get: 182 http://deb.debian.org/debian unstable/main amd64 python3-jaraco.functools all 4.0.0-1 [10.6 kB] Get: 183 http://deb.debian.org/debian unstable/main amd64 python3-jeepney all 0.8.0-3 [34.0 kB] Get: 184 http://deb.debian.org/debian unstable/main amd64 python3-jinja2 all 3.1.3-1 [119 kB] Get: 185 http://deb.debian.org/debian unstable/main amd64 python3-secretstorage all 3.3.3-3 [16.1 kB] Get: 186 http://deb.debian.org/debian unstable/main amd64 python3-keyring all 25.2.1-1 [53.7 kB] Get: 187 http://deb.debian.org/debian unstable/main amd64 python3-oslo.context all 5.5.0-2 [13.3 kB] Get: 188 http://deb.debian.org/debian unstable/main amd64 python3-msgpack amd64 1.0.3-3+b1 [96.5 kB] Get: 189 http://deb.debian.org/debian unstable/main amd64 python3-oslo.serialization all 5.4.0-2 [14.7 kB] Get: 190 http://deb.debian.org/debian unstable/main amd64 python3-pyasyncore all 1.0.2-2 [10.0 kB] Get: 191 http://deb.debian.org/debian unstable/main amd64 python3-pyinotify all 0.9.6-2 [27.4 kB] Get: 192 http://deb.debian.org/debian unstable/main amd64 python3-oslo.log all 5.5.1-2 [44.0 kB] Get: 193 http://deb.debian.org/debian unstable/main amd64 python3-rfc3986 all 2.0.0-2 [21.6 kB] Get: 194 http://deb.debian.org/debian unstable/main amd64 python3-oslo.config all 1:9.4.0-2 [105 kB] Get: 195 http://deb.debian.org/debian unstable/main amd64 python3-keystoneclient all 1:5.4.0-2 [199 kB] Get: 196 http://deb.debian.org/debian unstable/main amd64 python3-oslo.cache all 3.7.0-2 [34.8 kB] Get: 197 http://deb.debian.org/debian unstable/main amd64 python3-pycadf all 3.1.1-3 [17.9 kB] Get: 198 http://deb.debian.org/debian unstable/main amd64 python3-webob all 1:1.8.7-1 [88.2 kB] Get: 199 http://deb.debian.org/debian unstable/main amd64 python3-keystonemiddleware all 10.6.0-2 [79.0 kB] Get: 200 http://deb.debian.org/debian unstable/main amd64 python3-mock all 5.1.0-1 [70.2 kB] Get: 201 http://deb.debian.org/debian unstable/main amd64 python3-novaclient all 2:18.6.0-3 [248 kB] Get: 202 http://deb.debian.org/debian unstable/main amd64 python3-platformdirs all 4.2.1-1 [16.0 kB] Get: 203 http://deb.debian.org/debian unstable/main amd64 python3-requestsexceptions all 1.4.0-3 [4188 B] Get: 204 http://deb.debian.org/debian unstable/main amd64 python3-openstacksdk all 3.0.0-3 [666 kB] Get: 205 http://deb.debian.org/debian unstable/main amd64 python3-simplejson amd64 3.19.2-1+b1 [69.4 kB] Get: 206 http://deb.debian.org/debian unstable/main amd64 python3-osc-lib all 3.0.1-2 [52.7 kB] Get: 207 http://deb.debian.org/debian unstable/main amd64 python3-openstackclient all 6.6.0-3 [494 kB] Get: 208 http://deb.debian.org/debian unstable/main amd64 python3-os-client-config all 2.1.0-2 [17.4 kB] Get: 209 http://deb.debian.org/debian unstable/main amd64 python3-neutronclient all 1:11.2.0-3 [120 kB] Get: 210 http://deb.debian.org/debian unstable/main amd64 sphinx-common all 7.2.6-8 [702 kB] Get: 211 http://deb.debian.org/debian unstable/main amd64 python3-pygments all 2.17.2+dfsg-1 [818 kB] Get: 212 http://deb.debian.org/debian unstable/main amd64 python3-snowballstemmer all 2.2.0-4 [58.0 kB] Get: 213 http://deb.debian.org/debian unstable/main amd64 python3-sphinx all 7.2.6-8 [552 kB] Get: 214 http://deb.debian.org/debian unstable/main amd64 python3-openstackdocstheme all 3.2.0-1 [684 kB] Get: 215 http://deb.debian.org/debian unstable/main amd64 python3-oslosphinx all 4.18.0-5 [34.5 kB] Get: 216 http://deb.debian.org/debian unstable/main amd64 python3-os-api-ref all 2.3.0+ds1-3 [16.9 kB] Get: 217 http://deb.debian.org/debian unstable/main amd64 python3-tomlkit all 0.12.5-1 [42.0 kB] Get: 218 http://deb.debian.org/debian unstable/main amd64 python3-voluptuous all 0.14.2-1 [45.8 kB] Get: 219 http://deb.debian.org/debian unstable/main amd64 python3-subunit all 1.4.2-3 [66.4 kB] Get: 220 http://deb.debian.org/debian unstable/main amd64 subunit all 1.4.2-3 [5248 B] Get: 221 http://deb.debian.org/debian unstable/main amd64 python3-stestr all 4.1.0-2 [56.7 kB] Get: 222 http://deb.debian.org/debian unstable/main amd64 python3-os-testr all 3.0.0-3 [17.9 kB] Get: 223 http://deb.debian.org/debian unstable/main amd64 python3-tempita all 0.5.2-7 [15.0 kB] Get: 224 http://deb.debian.org/debian unstable/main amd64 python3-paste all 3.10.0-1 [222 kB] Get: 225 http://deb.debian.org/debian unstable/main amd64 python3-pastedeploy-tpl all 3.1-1 [8268 B] Get: 226 http://deb.debian.org/debian unstable/main amd64 python3-pastedeploy all 3.1-1 [18.3 kB] Get: 227 http://deb.debian.org/debian unstable/main amd64 python3-pyeclib amd64 1.6.1-2 [29.0 kB] Get: 228 http://deb.debian.org/debian unstable/main amd64 python3-requests-mock all 1.11.0-1 [17.4 kB] Get: 229 http://deb.debian.org/debian unstable/main amd64 python3-swiftclient all 1:4.5.0-2 [76.1 kB] Get: 230 http://deb.debian.org/debian unstable/main amd64 python3-xattr amd64 0.10.1-1 [16.2 kB] Fetched 69.8 MB in 1s (70.3 MB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package libpython3.11-minimal:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 19719 files and directories currently installed.) Preparing to unpack .../libpython3.11-minimal_3.11.9-1_amd64.deb ... Unpacking libpython3.11-minimal:amd64 (3.11.9-1) ... Selecting previously unselected package libexpat1:amd64. Preparing to unpack .../libexpat1_2.6.2-1_amd64.deb ... Unpacking libexpat1:amd64 (2.6.2-1) ... Selecting previously unselected package python3.11-minimal. Preparing to unpack .../python3.11-minimal_3.11.9-1_amd64.deb ... Unpacking python3.11-minimal (3.11.9-1) ... Setting up libpython3.11-minimal:amd64 (3.11.9-1) ... Setting up libexpat1:amd64 (2.6.2-1) ... Setting up python3.11-minimal (3.11.9-1) ... Selecting previously unselected package python3-minimal. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 20035 files and directories currently installed.) Preparing to unpack .../0-python3-minimal_3.11.8-1_amd64.deb ... Unpacking python3-minimal (3.11.8-1) ... Selecting previously unselected package media-types. Preparing to unpack .../1-media-types_10.1.0_all.deb ... Unpacking media-types (10.1.0) ... Selecting previously unselected package netbase. Preparing to unpack .../2-netbase_6.4_all.deb ... Unpacking netbase (6.4) ... Selecting previously unselected package tzdata. Preparing to unpack .../3-tzdata_2024a-4_all.deb ... Unpacking tzdata (2024a-4) ... Selecting previously unselected package readline-common. Preparing to unpack .../4-readline-common_8.2-4_all.deb ... Unpacking readline-common (8.2-4) ... Selecting previously unselected package libreadline8t64:amd64. Preparing to unpack .../5-libreadline8t64_8.2-4_amd64.deb ... Adding 'diversion of /lib/x86_64-linux-gnu/libhistory.so.8 to /lib/x86_64-linux-gnu/libhistory.so.8.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libhistory.so.8.2 to /lib/x86_64-linux-gnu/libhistory.so.8.2.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libreadline.so.8 to /lib/x86_64-linux-gnu/libreadline.so.8.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libreadline.so.8.2 to /lib/x86_64-linux-gnu/libreadline.so.8.2.usr-is-merged by libreadline8t64' Unpacking libreadline8t64:amd64 (8.2-4) ... Selecting previously unselected package libpython3.11-stdlib:amd64. Preparing to unpack .../6-libpython3.11-stdlib_3.11.9-1_amd64.deb ... Unpacking libpython3.11-stdlib:amd64 (3.11.9-1) ... Selecting previously unselected package python3.11. Preparing to unpack .../7-python3.11_3.11.9-1_amd64.deb ... Unpacking python3.11 (3.11.9-1) ... Selecting previously unselected package libpython3-stdlib:amd64. Preparing to unpack .../8-libpython3-stdlib_3.11.8-1_amd64.deb ... Unpacking libpython3-stdlib:amd64 (3.11.8-1) ... Setting up python3-minimal (3.11.8-1) ... Selecting previously unselected package python3. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 21027 files and directories currently installed.) Preparing to unpack .../000-python3_3.11.8-1_amd64.deb ... Unpacking python3 (3.11.8-1) ... Selecting previously unselected package libpython3.12-minimal:amd64. Preparing to unpack .../001-libpython3.12-minimal_3.12.3-1_amd64.deb ... Unpacking libpython3.12-minimal:amd64 (3.12.3-1) ... Selecting previously unselected package python3.12-minimal. Preparing to unpack .../002-python3.12-minimal_3.12.3-1_amd64.deb ... Unpacking python3.12-minimal (3.12.3-1) ... Selecting previously unselected package sgml-base. Preparing to unpack .../003-sgml-base_1.31_all.deb ... Unpacking sgml-base (1.31) ... Selecting previously unselected package sensible-utils. Preparing to unpack .../004-sensible-utils_0.0.22_all.deb ... Unpacking sensible-utils (0.0.22) ... Selecting previously unselected package openssl. Preparing to unpack .../005-openssl_3.2.1-3_amd64.deb ... Unpacking openssl (3.2.1-3) ... Selecting previously unselected package ca-certificates. Preparing to unpack .../006-ca-certificates_20240203_all.deb ... Unpacking ca-certificates (20240203) ... Selecting previously unselected package libmagic-mgc. Preparing to unpack .../007-libmagic-mgc_1%3a5.45-3_amd64.deb ... Unpacking libmagic-mgc (1:5.45-3) ... Selecting previously unselected package libmagic1t64:amd64. Preparing to unpack .../008-libmagic1t64_1%3a5.45-3_amd64.deb ... Unpacking libmagic1t64:amd64 (1:5.45-3) ... Selecting previously unselected package file. Preparing to unpack .../009-file_1%3a5.45-3_amd64.deb ... Unpacking file (1:5.45-3) ... Selecting previously unselected package gettext-base. Preparing to unpack .../010-gettext-base_0.21-14+b1_amd64.deb ... Unpacking gettext-base (0.21-14+b1) ... Selecting previously unselected package libuchardet0:amd64. Preparing to unpack .../011-libuchardet0_0.0.8-1+b1_amd64.deb ... Unpacking libuchardet0:amd64 (0.0.8-1+b1) ... Selecting previously unselected package groff-base. Preparing to unpack .../012-groff-base_1.23.0-4_amd64.deb ... Unpacking groff-base (1.23.0-4) ... Selecting previously unselected package bsdextrautils. Preparing to unpack .../013-bsdextrautils_2.40.1-4_amd64.deb ... Unpacking bsdextrautils (2.40.1-4) ... Selecting previously unselected package libpipeline1:amd64. Preparing to unpack .../014-libpipeline1_1.5.7-2_amd64.deb ... Unpacking libpipeline1:amd64 (1.5.7-2) ... Selecting previously unselected package man-db. Preparing to unpack .../015-man-db_2.12.1-1_amd64.deb ... Unpacking man-db (2.12.1-1) ... Selecting previously unselected package libpsl5t64:amd64. Preparing to unpack .../016-libpsl5t64_0.21.2-1.1_amd64.deb ... Unpacking libpsl5t64:amd64 (0.21.2-1.1) ... Selecting previously unselected package wget. Preparing to unpack .../017-wget_1.24.5-1_amd64.deb ... Unpacking wget (1.24.5-1) ... Selecting previously unselected package m4. Preparing to unpack .../018-m4_1.4.19-4_amd64.deb ... Unpacking m4 (1.4.19-4) ... Selecting previously unselected package autoconf. Preparing to unpack .../019-autoconf_2.71-3_all.deb ... Unpacking autoconf (2.71-3) ... Selecting previously unselected package autotools-dev. Preparing to unpack .../020-autotools-dev_20220109.1_all.deb ... Unpacking autotools-dev (20220109.1) ... Selecting previously unselected package automake. Preparing to unpack .../021-automake_1%3a1.16.5-1.3_all.deb ... Unpacking automake (1:1.16.5-1.3) ... Selecting previously unselected package autopoint. Preparing to unpack .../022-autopoint_0.21-14_all.deb ... Unpacking autopoint (0.21-14) ... Selecting previously unselected package libdebhelper-perl. Preparing to unpack .../023-libdebhelper-perl_13.15.3_all.deb ... Unpacking libdebhelper-perl (13.15.3) ... Selecting previously unselected package libtool. Preparing to unpack .../024-libtool_2.4.7-7_all.deb ... Unpacking libtool (2.4.7-7) ... Selecting previously unselected package dh-autoreconf. Preparing to unpack .../025-dh-autoreconf_20_all.deb ... Unpacking dh-autoreconf (20) ... Selecting previously unselected package libarchive-zip-perl. Preparing to unpack .../026-libarchive-zip-perl_1.68-1_all.deb ... Unpacking libarchive-zip-perl (1.68-1) ... Selecting previously unselected package libfile-stripnondeterminism-perl. Preparing to unpack .../027-libfile-stripnondeterminism-perl_1.14.0-1_all.deb ... Unpacking libfile-stripnondeterminism-perl (1.14.0-1) ... Selecting previously unselected package dh-strip-nondeterminism. Preparing to unpack .../028-dh-strip-nondeterminism_1.14.0-1_all.deb ... Unpacking dh-strip-nondeterminism (1.14.0-1) ... Selecting previously unselected package libelf1t64:amd64. Preparing to unpack .../029-libelf1t64_0.191-1+b1_amd64.deb ... Unpacking libelf1t64:amd64 (0.191-1+b1) ... Selecting previously unselected package dwz. Preparing to unpack .../030-dwz_0.15-1+b1_amd64.deb ... Unpacking dwz (0.15-1+b1) ... Selecting previously unselected package libicu72:amd64. Preparing to unpack .../031-libicu72_72.1-4+b1_amd64.deb ... Unpacking libicu72:amd64 (72.1-4+b1) ... Selecting previously unselected package libxml2:amd64. Preparing to unpack .../032-libxml2_2.12.7+dfsg-2_amd64.deb ... Unpacking libxml2:amd64 (2.12.7+dfsg-2) ... Selecting previously unselected package gettext. Preparing to unpack .../033-gettext_0.21-14+b1_amd64.deb ... Unpacking gettext (0.21-14+b1) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../034-intltool-debian_0.35.0+20060710.6_all.deb ... Unpacking intltool-debian (0.35.0+20060710.6) ... Selecting previously unselected package po-debconf. Preparing to unpack .../035-po-debconf_1.0.21+nmu1_all.deb ... Unpacking po-debconf (1.0.21+nmu1) ... Selecting previously unselected package debhelper. Preparing to unpack .../036-debhelper_13.15.3_all.deb ... Unpacking debhelper (13.15.3) ... Selecting previously unselected package dh-exec. Preparing to unpack .../037-dh-exec_0.30_amd64.deb ... Unpacking dh-exec (0.30) ... Selecting previously unselected package python3-pkg-resources. Preparing to unpack .../038-python3-pkg-resources_68.1.2-2_all.deb ... Unpacking python3-pkg-resources (68.1.2-2) ... Selecting previously unselected package python3-lib2to3. Preparing to unpack .../039-python3-lib2to3_3.12.3-3.1_all.deb ... Unpacking python3-lib2to3 (3.12.3-3.1) ... Selecting previously unselected package python3-distutils. Preparing to unpack .../040-python3-distutils_3.12.3-3.1_all.deb ... Unpacking python3-distutils (3.12.3-3.1) ... Selecting previously unselected package python3-setuptools. Preparing to unpack .../041-python3-setuptools_68.1.2-2_all.deb ... Unpacking python3-setuptools (68.1.2-2) ... Selecting previously unselected package dh-python. Preparing to unpack .../042-dh-python_6.20240422_all.deb ... Unpacking dh-python (6.20240422) ... Selecting previously unselected package xml-core. Preparing to unpack .../043-xml-core_0.19_all.deb ... Unpacking xml-core (0.19) ... Selecting previously unselected package docutils-common. Preparing to unpack .../044-docutils-common_0.20.1+dfsg-3_all.deb ... Unpacking docutils-common (0.20.1+dfsg-3) ... Selecting previously unselected package fonts-font-awesome. Preparing to unpack .../045-fonts-font-awesome_5.0.10+really4.7.0~dfsg-4.1_all.deb ... Unpacking fonts-font-awesome (5.0.10+really4.7.0~dfsg-4.1) ... Selecting previously unselected package fonts-glyphicons-halflings. Preparing to unpack .../046-fonts-glyphicons-halflings_1.009~3.4.1+dfsg-3_all.deb ... Unpacking fonts-glyphicons-halflings (1.009~3.4.1+dfsg-3) ... Selecting previously unselected package ieee-data. Preparing to unpack .../047-ieee-data_20220827.1_all.deb ... Unpacking ieee-data (20220827.1) ... Selecting previously unselected package libonig5:amd64. Preparing to unpack .../048-libonig5_6.9.9-1_amd64.deb ... Unpacking libonig5:amd64 (6.9.9-1) ... Selecting previously unselected package libjq1:amd64. Preparing to unpack .../049-libjq1_1.7.1-3_amd64.deb ... Unpacking libjq1:amd64 (1.7.1-3) ... Selecting previously unselected package jq. Preparing to unpack .../050-jq_1.7.1-3_amd64.deb ... Unpacking jq (1.7.1-3) ... Selecting previously unselected package libbsd0:amd64. Preparing to unpack .../051-libbsd0_0.12.2-1_amd64.deb ... Unpacking libbsd0:amd64 (0.12.2-1) ... Selecting previously unselected package liberasurecode1:amd64. Preparing to unpack .../052-liberasurecode1_1.6.3-1_amd64.deb ... Unpacking liberasurecode1:amd64 (1.6.3-1) ... Selecting previously unselected package x11-common. Preparing to unpack .../053-x11-common_1%3a7.7+23_all.deb ... Unpacking x11-common (1:7.7+23) ... Selecting previously unselected package libice6:amd64. Preparing to unpack .../054-libice6_2%3a1.0.10-1+b1_amd64.deb ... Unpacking libice6:amd64 (2:1.0.10-1+b1) ... Selecting previously unselected package libjs-bootstrap. Preparing to unpack .../055-libjs-bootstrap_3.4.1+dfsg-3_all.deb ... Unpacking libjs-bootstrap (3.4.1+dfsg-3) ... Selecting previously unselected package libjs-jquery. Preparing to unpack .../056-libjs-jquery_3.6.1+dfsg+~3.5.14-1_all.deb ... Unpacking libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... Selecting previously unselected package libjs-underscore. Preparing to unpack .../057-libjs-underscore_1.13.4~dfsg+~1.11.4-3_all.deb ... Unpacking libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... Selecting previously unselected package libjs-sphinxdoc. Preparing to unpack .../058-libjs-sphinxdoc_7.2.6-8_all.deb ... Unpacking libjs-sphinxdoc (7.2.6-8) ... Selecting previously unselected package libjson-perl. Preparing to unpack .../059-libjson-perl_4.10000-1_all.deb ... Unpacking libjson-perl (4.10000-1) ... Selecting previously unselected package libpython3.12-stdlib:amd64. Preparing to unpack .../060-libpython3.12-stdlib_3.12.3-1_amd64.deb ... Unpacking libpython3.12-stdlib:amd64 (3.12.3-1) ... Selecting previously unselected package libsm6:amd64. Preparing to unpack .../061-libsm6_2%3a1.2.3-1+b1_amd64.deb ... Unpacking libsm6:amd64 (2:1.2.3-1+b1) ... Selecting previously unselected package libsubunit-perl. Preparing to unpack .../062-libsubunit-perl_1.4.2-3_all.deb ... Unpacking libsubunit-perl (1.4.2-3) ... Selecting previously unselected package libxau6:amd64. Preparing to unpack .../063-libxau6_1%3a1.0.9-1+b1_amd64.deb ... Unpacking libxau6:amd64 (1:1.0.9-1+b1) ... Selecting previously unselected package libxdmcp6:amd64. Preparing to unpack .../064-libxdmcp6_1%3a1.1.2-3+b1_amd64.deb ... Unpacking libxdmcp6:amd64 (1:1.1.2-3+b1) ... Selecting previously unselected package libxcb1:amd64. Preparing to unpack .../065-libxcb1_1.17.0-2_amd64.deb ... Unpacking libxcb1:amd64 (1.17.0-2) ... Selecting previously unselected package libx11-data. Preparing to unpack .../066-libx11-data_2%3a1.8.7-1_all.deb ... Unpacking libx11-data (2:1.8.7-1) ... Selecting previously unselected package libx11-6:amd64. Preparing to unpack .../067-libx11-6_2%3a1.8.7-1+b1_amd64.deb ... Unpacking libx11-6:amd64 (2:1.8.7-1+b1) ... Selecting previously unselected package libxext6:amd64. Preparing to unpack .../068-libxext6_2%3a1.3.4-1+b1_amd64.deb ... Unpacking libxext6:amd64 (2:1.3.4-1+b1) ... Selecting previously unselected package libxt6t64:amd64. Preparing to unpack .../069-libxt6t64_1%3a1.2.1-1.2_amd64.deb ... Unpacking libxt6t64:amd64 (1:1.2.1-1.2) ... Selecting previously unselected package libxmu6:amd64. Preparing to unpack .../070-libxmu6_2%3a1.1.3-3+b2_amd64.deb ... Unpacking libxmu6:amd64 (2:1.1.3-3+b2) ... Selecting previously unselected package libxslt1.1:amd64. Preparing to unpack .../071-libxslt1.1_1.1.35-1+b1_amd64.deb ... Unpacking libxslt1.1:amd64 (1.1.35-1+b1) ... Selecting previously unselected package libyaml-0-2:amd64. Preparing to unpack .../072-libyaml-0-2_0.2.5-1+b1_amd64.deb ... Unpacking libyaml-0-2:amd64 (0.2.5-1+b1) ... Selecting previously unselected package python3-wheel. Preparing to unpack .../073-python3-wheel_0.43.0-1_all.deb ... Unpacking python3-wheel (0.43.0-1) ... Selecting previously unselected package python3-pip. Preparing to unpack .../074-python3-pip_24.0+dfsg-2_all.deb ... Unpacking python3-pip (24.0+dfsg-2) ... Selecting previously unselected package openstack-pkg-tools. Preparing to unpack .../075-openstack-pkg-tools_126_all.deb ... Unpacking openstack-pkg-tools (126) ... Selecting previously unselected package pycadf-common. Preparing to unpack .../076-pycadf-common_3.1.1-3_all.deb ... Unpacking pycadf-common (3.1.1-3) ... Selecting previously unselected package python-babel-localedata. Preparing to unpack .../077-python-babel-localedata_2.14.0-1_all.deb ... Unpacking python-babel-localedata (2.14.0-1) ... Selecting previously unselected package python-os-api-ref-common. Preparing to unpack .../078-python-os-api-ref-common_2.3.0+ds1-3_all.deb ... Unpacking python-os-api-ref-common (2.3.0+ds1-3) ... Selecting previously unselected package python-oslosphinx-common. Preparing to unpack .../079-python-oslosphinx-common_4.18.0-5_all.deb ... Unpacking python-oslosphinx-common (4.18.0-5) ... Selecting previously unselected package python3-alabaster. Preparing to unpack .../080-python3-alabaster_0.7.12-1_all.deb ... Unpacking python3-alabaster (0.7.12-1) ... Selecting previously unselected package python3.12. Preparing to unpack .../081-python3.12_3.12.3-1_amd64.deb ... Unpacking python3.12 (3.12.3-1) ... Selecting previously unselected package python3-all. Preparing to unpack .../082-python3-all_3.11.8-1_amd64.deb ... Unpacking python3-all (3.11.8-1) ... Selecting previously unselected package python3-typing-extensions. Preparing to unpack .../083-python3-typing-extensions_4.12.0-1_all.deb ... Unpacking python3-typing-extensions (4.12.0-1) ... Selecting previously unselected package python3-more-itertools. Preparing to unpack .../084-python3-more-itertools_10.2.0-1_all.deb ... Unpacking python3-more-itertools (10.2.0-1) ... Selecting previously unselected package python3-zipp. Preparing to unpack .../085-python3-zipp_1.0.0-6_all.deb ... Unpacking python3-zipp (1.0.0-6) ... Selecting previously unselected package python3-importlib-metadata. Preparing to unpack .../086-python3-importlib-metadata_7.1.0-1_all.deb ... Unpacking python3-importlib-metadata (7.1.0-1) ... Selecting previously unselected package python3-attr. Preparing to unpack .../087-python3-attr_23.2.0-2_all.deb ... Unpacking python3-attr (23.2.0-2) ... Selecting previously unselected package python3-autopage. Preparing to unpack .../088-python3-autopage_0.4.0-3_all.deb ... Unpacking python3-autopage (0.4.0-3) ... Selecting previously unselected package python3-tz. Preparing to unpack .../089-python3-tz_2024.1-2_all.deb ... Unpacking python3-tz (2024.1-2) ... Selecting previously unselected package python3-babel. Preparing to unpack .../090-python3-babel_2.14.0-1_all.deb ... Unpacking python3-babel (2.14.0-1) ... Selecting previously unselected package python3-six. Preparing to unpack .../091-python3-six_1.16.0-6_all.deb ... Unpacking python3-six (1.16.0-6) ... Selecting previously unselected package python3-memcache. Preparing to unpack .../092-python3-memcache_1.62-1_all.deb ... Unpacking python3-memcache (1.62-1) ... Selecting previously unselected package python3-uhashring. Preparing to unpack .../093-python3-uhashring_2.3-2_all.deb ... Unpacking python3-uhashring (2.3-2) ... Selecting previously unselected package python3-binary-memcached. Preparing to unpack .../094-python3-binary-memcached_0.31.2+dfsg1-2_all.deb ... Unpacking python3-binary-memcached (0.31.2+dfsg1-2) ... Selecting previously unselected package python3-blinker. Preparing to unpack .../095-python3-blinker_1.8.2-1_all.deb ... Unpacking python3-blinker (1.8.2-1) ... Selecting previously unselected package python3-dateutil. Preparing to unpack .../096-python3-dateutil_2.9.0-2_all.deb ... Unpacking python3-dateutil (2.9.0-2) ... Selecting previously unselected package python3-jmespath. Preparing to unpack .../097-python3-jmespath_1.0.1-1_all.deb ... Unpacking python3-jmespath (1.0.1-1) ... Selecting previously unselected package python3-urllib3. Preparing to unpack .../098-python3-urllib3_1.26.18-2_all.deb ... Unpacking python3-urllib3 (1.26.18-2) ... Selecting previously unselected package python3-certifi. Preparing to unpack .../099-python3-certifi_2023.11.17-1_all.deb ... Unpacking python3-certifi (2023.11.17-1) ... Selecting previously unselected package python3-charset-normalizer. Preparing to unpack .../100-python3-charset-normalizer_3.3.2-1_all.deb ... Unpacking python3-charset-normalizer (3.3.2-1) ... Selecting previously unselected package python3-idna. Preparing to unpack .../101-python3-idna_3.6-2_all.deb ... Unpacking python3-idna (3.6-2) ... Selecting previously unselected package python3-chardet. Preparing to unpack .../102-python3-chardet_5.2.0+dfsg-1_all.deb ... Unpacking python3-chardet (5.2.0+dfsg-1) ... Selecting previously unselected package python3-requests. Preparing to unpack .../103-python3-requests_2.31.0+dfsg-2_all.deb ... Unpacking python3-requests (2.31.0+dfsg-2) ... Selecting previously unselected package python3-botocore. Preparing to unpack .../104-python3-botocore_1.34.46+repack-1_all.deb ... Unpacking python3-botocore (1.34.46+repack-1) ... Selecting previously unselected package python3-s3transfer. Preparing to unpack .../105-python3-s3transfer_0.10.1-1_all.deb ... Unpacking python3-s3transfer (0.10.1-1) ... Selecting previously unselected package python3-boto3. Preparing to unpack .../106-python3-boto3_1.34.46+dfsg-1_all.deb ... Unpacking python3-boto3 (1.34.46+dfsg-1) ... Selecting previously unselected package python3-cffi-backend:amd64. Preparing to unpack .../107-python3-cffi-backend_1.16.0-2+b2_amd64.deb ... Unpacking python3-cffi-backend:amd64 (1.16.0-2+b2) ... Selecting previously unselected package python3-ply. Preparing to unpack .../108-python3-ply_3.11-6_all.deb ... Unpacking python3-ply (3.11-6) ... Selecting previously unselected package python3-pycparser. Preparing to unpack .../109-python3-pycparser_2.22-1_all.deb ... Unpacking python3-pycparser (2.22-1) ... Selecting previously unselected package python3-cffi. Preparing to unpack .../110-python3-cffi_1.16.0-2_all.deb ... Unpacking python3-cffi (1.16.0-2) ... Selecting previously unselected package python3-iso8601. Preparing to unpack .../111-python3-iso8601_1.0.2-1_all.deb ... Unpacking python3-iso8601 (1.0.2-1) ... Selecting previously unselected package python3-lxml:amd64. Preparing to unpack .../112-python3-lxml_5.2.1-1_amd64.deb ... Unpacking python3-lxml:amd64 (5.2.1-1) ... Selecting previously unselected package python3-cryptography. Preparing to unpack .../113-python3-cryptography_42.0.5-2_amd64.deb ... Unpacking python3-cryptography (42.0.5-2) ... Selecting previously unselected package python3-jwt. Preparing to unpack .../114-python3-jwt_2.7.0-1_all.deb ... Unpacking python3-jwt (2.7.0-1) ... Selecting previously unselected package python3-oauthlib. Preparing to unpack .../115-python3-oauthlib_3.2.2-1_all.deb ... Unpacking python3-oauthlib (3.2.2-1) ... Selecting previously unselected package python3-pbr. Preparing to unpack .../116-python3-pbr_6.0.0-1_all.deb ... Unpacking python3-pbr (6.0.0-1) ... Selecting previously unselected package python3-os-service-types. Preparing to unpack .../117-python3-os-service-types_1.7.0-3_all.deb ... Unpacking python3-os-service-types (1.7.0-3) ... Selecting previously unselected package python3-stevedore. Preparing to unpack .../118-python3-stevedore_5.2.0-2_all.deb ... Unpacking python3-stevedore (5.2.0-2) ... Selecting previously unselected package python3-keystoneauth1. Preparing to unpack .../119-python3-keystoneauth1_5.6.0-2_all.deb ... Unpacking python3-keystoneauth1 (5.6.0-2) ... Selecting previously unselected package python3-oslo.i18n. Preparing to unpack .../120-python3-oslo.i18n_6.3.0-2_all.deb ... Unpacking python3-oslo.i18n (6.3.0-2) ... Selecting previously unselected package python3-wrapt. Preparing to unpack .../121-python3-wrapt_1.15.0-2+b1_amd64.deb ... Unpacking python3-wrapt (1.15.0-2+b1) ... Selecting previously unselected package python3-debtcollector. Preparing to unpack .../122-python3-debtcollector_3.0.0-2_all.deb ... Unpacking python3-debtcollector (3.0.0-2) ... Selecting previously unselected package python3-netaddr. Preparing to unpack .../123-python3-netaddr_0.10.1-1_all.deb ... Unpacking python3-netaddr (0.10.1-1) ... Selecting previously unselected package python3-netifaces:amd64. Preparing to unpack .../124-python3-netifaces_0.11.0-2+b2_amd64.deb ... Unpacking python3-netifaces:amd64 (0.11.0-2+b2) ... Selecting previously unselected package python3-packaging. Preparing to unpack .../125-python3-packaging_24.0-1_all.deb ... Unpacking python3-packaging (24.0-1) ... Selecting previously unselected package python3-pyparsing. Preparing to unpack .../126-python3-pyparsing_3.1.2-1_all.deb ... Unpacking python3-pyparsing (3.1.2-1) ... Selecting previously unselected package python3-yaml. Preparing to unpack .../127-python3-yaml_6.0.1-2_amd64.deb ... Unpacking python3-yaml (6.0.1-2) ... Selecting previously unselected package python3-oslo.utils. Preparing to unpack .../128-python3-oslo.utils_7.1.0-2_all.deb ... Unpacking python3-oslo.utils (7.1.0-2) ... Selecting previously unselected package python3-wcwidth. Preparing to unpack .../129-python3-wcwidth_0.2.13+dfsg1-1_all.deb ... Unpacking python3-wcwidth (0.2.13+dfsg1-1) ... Selecting previously unselected package python3-prettytable. Preparing to unpack .../130-python3-prettytable_3.6.0-1_all.deb ... Unpacking python3-prettytable (3.6.0-1) ... Selecting previously unselected package python3-cinderclient. Preparing to unpack .../131-python3-cinderclient_1%3a9.5.0-2_all.deb ... Unpacking python3-cinderclient (1:9.5.0-2) ... Selecting previously unselected package xclip. Preparing to unpack .../132-xclip_0.13-4_amd64.deb ... Unpacking xclip (0.13-4) ... Selecting previously unselected package python3-pyperclip. Preparing to unpack .../133-python3-pyperclip_1.8.2-2_all.deb ... Unpacking python3-pyperclip (1.8.2-2) ... Selecting previously unselected package python3-cmd2. Preparing to unpack .../134-python3-cmd2_2.4.3+ds-2_all.deb ... Unpacking python3-cmd2 (2.4.3+ds-2) ... Selecting previously unselected package python3-roman. Preparing to unpack .../135-python3-roman_3.3-3_all.deb ... Unpacking python3-roman (3.3-3) ... Selecting previously unselected package python3-docutils. Preparing to unpack .../136-python3-docutils_0.20.1+dfsg-3_all.deb ... Unpacking python3-docutils (0.20.1+dfsg-3) ... Selecting previously unselected package python3-cliff. Preparing to unpack .../137-python3-cliff_4.6.0-2_all.deb ... Unpacking python3-cliff (4.6.0-2) ... Selecting previously unselected package python3-coverage. Preparing to unpack .../138-python3-coverage_7.2.7+dfsg1-1+b1_amd64.deb ... Unpacking python3-coverage (7.2.7+dfsg1-1+b1) ... Selecting previously unselected package python3-decorator. Preparing to unpack .../139-python3-decorator_5.1.1-5_all.deb ... Unpacking python3-decorator (5.1.1-5) ... Selecting previously unselected package python3-deprecation. Preparing to unpack .../140-python3-deprecation_2.1.0-3_all.deb ... Unpacking python3-deprecation (2.1.0-3) ... Selecting previously unselected package python3-dnspython. Preparing to unpack .../141-python3-dnspython_2.6.1-1_all.deb ... Unpacking python3-dnspython (2.6.1-1) ... Selecting previously unselected package python3-markupsafe. Preparing to unpack .../142-python3-markupsafe_2.1.5-1_amd64.deb ... Unpacking python3-markupsafe (2.1.5-1) ... Selecting previously unselected package python3-mako. Preparing to unpack .../143-python3-mako_1.3.2-1_all.deb ... Unpacking python3-mako (1.3.2-1) ... Selecting previously unselected package python3-dogpile.cache. Preparing to unpack .../144-python3-dogpile.cache_1.3.1-1_all.deb ... Unpacking python3-dogpile.cache (1.3.1-1) ... Selecting previously unselected package python3-dulwich. Preparing to unpack .../145-python3-dulwich_0.21.6-1+b1_amd64.deb ... Unpacking python3-dulwich (0.21.6-1+b1) ... Selecting previously unselected package python3-greenlet. Preparing to unpack .../146-python3-greenlet_3.0.1-3_amd64.deb ... Unpacking python3-greenlet (3.0.1-3) ... Selecting previously unselected package python3-eventlet. Preparing to unpack .../147-python3-eventlet_0.35.1-1_all.deb ... Unpacking python3-eventlet (0.35.1-1) ... Selecting previously unselected package python3-extras. Preparing to unpack .../148-python3-extras_1.0.0-5_all.deb ... Unpacking python3-extras (1.0.0-5) ... Selecting previously unselected package python3-testtools. Preparing to unpack .../149-python3-testtools_2.7.1-3_all.deb ... Unpacking python3-testtools (2.7.1-3) ... Selecting previously unselected package python3-fixtures. Preparing to unpack .../150-python3-fixtures_4.1.0-2_all.deb ... Unpacking python3-fixtures (4.1.0-2) ... Selecting previously unselected package python3-openssl. Preparing to unpack .../151-python3-openssl_24.1.0-1_all.deb ... Unpacking python3-openssl (24.1.0-1) ... Selecting previously unselected package python3-json-pointer. Preparing to unpack .../152-python3-json-pointer_2.4-2_all.deb ... Unpacking python3-json-pointer (2.4-2) ... Selecting previously unselected package python3-jsonpatch. Preparing to unpack .../153-python3-jsonpatch_1.32-3_all.deb ... Unpacking python3-jsonpatch (1.32-3) ... Selecting previously unselected package python3-pathspec. Preparing to unpack .../154-python3-pathspec_0.12.1-1_all.deb ... Unpacking python3-pathspec (0.12.1-1) ... Selecting previously unselected package python3-pluggy. Preparing to unpack .../155-python3-pluggy_1.5.0-1_all.deb ... Unpacking python3-pluggy (1.5.0-1) ... Selecting previously unselected package python3-trove-classifiers. Preparing to unpack .../156-python3-trove-classifiers_2024.1.31-1_all.deb ... Unpacking python3-trove-classifiers (2024.1.31-1) ... Selecting previously unselected package python3-hatchling. Preparing to unpack .../157-python3-hatchling_1.24.2-1_all.deb ... Unpacking python3-hatchling (1.24.2-1) ... Selecting previously unselected package python3-setuptools-scm. Preparing to unpack .../158-python3-setuptools-scm_8.0.4-2_all.deb ... Unpacking python3-setuptools-scm (8.0.4-2) ... Selecting previously unselected package python3-hatch-vcs. Preparing to unpack .../159-python3-hatch-vcs_0.4.0-1_all.deb ... Unpacking python3-hatch-vcs (0.4.0-1) ... Selecting previously unselected package python3-rpds-py. Preparing to unpack .../160-python3-rpds-py_0.12.0-3_amd64.deb ... Unpacking python3-rpds-py (0.12.0-3) ... Selecting previously unselected package python3-referencing. Preparing to unpack .../161-python3-referencing_0.31.0-1_all.deb ... Unpacking python3-referencing (0.31.0-1) ... Selecting previously unselected package python3-jsonschema-specifications. Preparing to unpack .../162-python3-jsonschema-specifications_2023.12.1-1_all.deb ... Unpacking python3-jsonschema-specifications (2023.12.1-1) ... Selecting previously unselected package python3-jsonschema. Preparing to unpack .../163-python3-jsonschema_4.19.2-3_all.deb ... Unpacking python3-jsonschema (4.19.2-3) ... Selecting previously unselected package python3-warlock. Preparing to unpack .../164-python3-warlock_2.0.1-4_all.deb ... Unpacking python3-warlock (2.0.1-4) ... Selecting previously unselected package python3-glanceclient. Preparing to unpack .../165-python3-glanceclient_1%3a4.5.0-2_all.deb ... Unpacking python3-glanceclient (1:4.5.0-2) ... Selecting previously unselected package python3-imagesize. Preparing to unpack .../166-python3-imagesize_1.4.1-1_all.deb ... Unpacking python3-imagesize (1.4.1-1) ... Selecting previously unselected package python3-jaraco.classes. Preparing to unpack .../167-python3-jaraco.classes_3.2.1-4_all.deb ... Unpacking python3-jaraco.classes (3.2.1-4) ... Selecting previously unselected package python3-jaraco.context. Preparing to unpack .../168-python3-jaraco.context_4.3.0-1_all.deb ... Unpacking python3-jaraco.context (4.3.0-1) ... Selecting previously unselected package python3-jaraco.functools. Preparing to unpack .../169-python3-jaraco.functools_4.0.0-1_all.deb ... Unpacking python3-jaraco.functools (4.0.0-1) ... Selecting previously unselected package python3-jeepney. Preparing to unpack .../170-python3-jeepney_0.8.0-3_all.deb ... Unpacking python3-jeepney (0.8.0-3) ... Selecting previously unselected package python3-jinja2. Preparing to unpack .../171-python3-jinja2_3.1.3-1_all.deb ... Unpacking python3-jinja2 (3.1.3-1) ... Selecting previously unselected package python3-secretstorage. Preparing to unpack .../172-python3-secretstorage_3.3.3-3_all.deb ... Unpacking python3-secretstorage (3.3.3-3) ... Selecting previously unselected package python3-keyring. Preparing to unpack .../173-python3-keyring_25.2.1-1_all.deb ... Unpacking python3-keyring (25.2.1-1) ... Selecting previously unselected package python3-oslo.context. Preparing to unpack .../174-python3-oslo.context_5.5.0-2_all.deb ... Unpacking python3-oslo.context (5.5.0-2) ... Selecting previously unselected package python3-msgpack. Preparing to unpack .../175-python3-msgpack_1.0.3-3+b1_amd64.deb ... Unpacking python3-msgpack (1.0.3-3+b1) ... Selecting previously unselected package python3-oslo.serialization. Preparing to unpack .../176-python3-oslo.serialization_5.4.0-2_all.deb ... Unpacking python3-oslo.serialization (5.4.0-2) ... Selecting previously unselected package python3-pyasyncore. Preparing to unpack .../177-python3-pyasyncore_1.0.2-2_all.deb ... Unpacking python3-pyasyncore (1.0.2-2) ... Selecting previously unselected package python3-pyinotify. Preparing to unpack .../178-python3-pyinotify_0.9.6-2_all.deb ... Unpacking python3-pyinotify (0.9.6-2) ... Selecting previously unselected package python3-oslo.log. Preparing to unpack .../179-python3-oslo.log_5.5.1-2_all.deb ... Unpacking python3-oslo.log (5.5.1-2) ... Selecting previously unselected package python3-rfc3986. Preparing to unpack .../180-python3-rfc3986_2.0.0-2_all.deb ... Unpacking python3-rfc3986 (2.0.0-2) ... Selecting previously unselected package python3-oslo.config. Preparing to unpack .../181-python3-oslo.config_1%3a9.4.0-2_all.deb ... Unpacking python3-oslo.config (1:9.4.0-2) ... Selecting previously unselected package python3-keystoneclient. Preparing to unpack .../182-python3-keystoneclient_1%3a5.4.0-2_all.deb ... Unpacking python3-keystoneclient (1:5.4.0-2) ... Selecting previously unselected package python3-oslo.cache. Preparing to unpack .../183-python3-oslo.cache_3.7.0-2_all.deb ... Unpacking python3-oslo.cache (3.7.0-2) ... Selecting previously unselected package python3-pycadf. Preparing to unpack .../184-python3-pycadf_3.1.1-3_all.deb ... Unpacking python3-pycadf (3.1.1-3) ... Selecting previously unselected package python3-webob. Preparing to unpack .../185-python3-webob_1%3a1.8.7-1_all.deb ... Unpacking python3-webob (1:1.8.7-1) ... Selecting previously unselected package python3-keystonemiddleware. Preparing to unpack .../186-python3-keystonemiddleware_10.6.0-2_all.deb ... Unpacking python3-keystonemiddleware (10.6.0-2) ... Selecting previously unselected package python3-mock. Preparing to unpack .../187-python3-mock_5.1.0-1_all.deb ... Unpacking python3-mock (5.1.0-1) ... Selecting previously unselected package python3-novaclient. Preparing to unpack .../188-python3-novaclient_2%3a18.6.0-3_all.deb ... Unpacking python3-novaclient (2:18.6.0-3) ... Selecting previously unselected package python3-platformdirs. Preparing to unpack .../189-python3-platformdirs_4.2.1-1_all.deb ... Unpacking python3-platformdirs (4.2.1-1) ... Selecting previously unselected package python3-requestsexceptions. Preparing to unpack .../190-python3-requestsexceptions_1.4.0-3_all.deb ... Unpacking python3-requestsexceptions (1.4.0-3) ... Selecting previously unselected package python3-openstacksdk. Preparing to unpack .../191-python3-openstacksdk_3.0.0-3_all.deb ... Unpacking python3-openstacksdk (3.0.0-3) ... Selecting previously unselected package python3-simplejson. Preparing to unpack .../192-python3-simplejson_3.19.2-1+b1_amd64.deb ... Unpacking python3-simplejson (3.19.2-1+b1) ... Selecting previously unselected package python3-osc-lib. Preparing to unpack .../193-python3-osc-lib_3.0.1-2_all.deb ... Unpacking python3-osc-lib (3.0.1-2) ... Selecting previously unselected package python3-openstackclient. Preparing to unpack .../194-python3-openstackclient_6.6.0-3_all.deb ... Unpacking python3-openstackclient (6.6.0-3) ... Selecting previously unselected package python3-os-client-config. Preparing to unpack .../195-python3-os-client-config_2.1.0-2_all.deb ... Unpacking python3-os-client-config (2.1.0-2) ... Selecting previously unselected package python3-neutronclient. Preparing to unpack .../196-python3-neutronclient_1%3a11.2.0-3_all.deb ... Unpacking python3-neutronclient (1:11.2.0-3) ... Selecting previously unselected package sphinx-common. Preparing to unpack .../197-sphinx-common_7.2.6-8_all.deb ... Unpacking sphinx-common (7.2.6-8) ... Selecting previously unselected package python3-pygments. Preparing to unpack .../198-python3-pygments_2.17.2+dfsg-1_all.deb ... Unpacking python3-pygments (2.17.2+dfsg-1) ... Selecting previously unselected package python3-snowballstemmer. Preparing to unpack .../199-python3-snowballstemmer_2.2.0-4_all.deb ... Unpacking python3-snowballstemmer (2.2.0-4) ... Selecting previously unselected package python3-sphinx. Preparing to unpack .../200-python3-sphinx_7.2.6-8_all.deb ... Unpacking python3-sphinx (7.2.6-8) ... Selecting previously unselected package python3-openstackdocstheme. Preparing to unpack .../201-python3-openstackdocstheme_3.2.0-1_all.deb ... Unpacking python3-openstackdocstheme (3.2.0-1) ... Selecting previously unselected package python3-oslosphinx. Preparing to unpack .../202-python3-oslosphinx_4.18.0-5_all.deb ... Unpacking python3-oslosphinx (4.18.0-5) ... Selecting previously unselected package python3-os-api-ref. Preparing to unpack .../203-python3-os-api-ref_2.3.0+ds1-3_all.deb ... Unpacking python3-os-api-ref (2.3.0+ds1-3) ... Selecting previously unselected package python3-tomlkit. Preparing to unpack .../204-python3-tomlkit_0.12.5-1_all.deb ... Unpacking python3-tomlkit (0.12.5-1) ... Selecting previously unselected package python3-voluptuous. Preparing to unpack .../205-python3-voluptuous_0.14.2-1_all.deb ... Unpacking python3-voluptuous (0.14.2-1) ... Selecting previously unselected package python3-subunit. Preparing to unpack .../206-python3-subunit_1.4.2-3_all.deb ... Unpacking python3-subunit (1.4.2-3) ... Selecting previously unselected package subunit. Preparing to unpack .../207-subunit_1.4.2-3_all.deb ... Unpacking subunit (1.4.2-3) ... Selecting previously unselected package python3-stestr. Preparing to unpack .../208-python3-stestr_4.1.0-2_all.deb ... Unpacking python3-stestr (4.1.0-2) ... Selecting previously unselected package python3-os-testr. Preparing to unpack .../209-python3-os-testr_3.0.0-3_all.deb ... Unpacking python3-os-testr (3.0.0-3) ... Selecting previously unselected package python3-tempita. Preparing to unpack .../210-python3-tempita_0.5.2-7_all.deb ... Unpacking python3-tempita (0.5.2-7) ... Selecting previously unselected package python3-paste. Preparing to unpack .../211-python3-paste_3.10.0-1_all.deb ... Unpacking python3-paste (3.10.0-1) ... Selecting previously unselected package python3-pastedeploy-tpl. Preparing to unpack .../212-python3-pastedeploy-tpl_3.1-1_all.deb ... Unpacking python3-pastedeploy-tpl (3.1-1) ... Selecting previously unselected package python3-pastedeploy. Preparing to unpack .../213-python3-pastedeploy_3.1-1_all.deb ... Unpacking python3-pastedeploy (3.1-1) ... Selecting previously unselected package python3-pyeclib. Preparing to unpack .../214-python3-pyeclib_1.6.1-2_amd64.deb ... Unpacking python3-pyeclib (1.6.1-2) ... Selecting previously unselected package python3-requests-mock. Preparing to unpack .../215-python3-requests-mock_1.11.0-1_all.deb ... Unpacking python3-requests-mock (1.11.0-1) ... Selecting previously unselected package python3-swiftclient. Preparing to unpack .../216-python3-swiftclient_1%3a4.5.0-2_all.deb ... Unpacking python3-swiftclient (1:4.5.0-2) ... Selecting previously unselected package python3-xattr. Preparing to unpack .../217-python3-xattr_0.10.1-1_amd64.deb ... Unpacking python3-xattr (0.10.1-1) ... Setting up media-types (10.1.0) ... Setting up libpipeline1:amd64 (1.5.7-2) ... Setting up libxau6:amd64 (1:1.0.9-1+b1) ... Setting up libsubunit-perl (1.4.2-3) ... Setting up libicu72:amd64 (72.1-4+b1) ... Setting up bsdextrautils (2.40.1-4) ... Setting up libmagic-mgc (1:5.45-3) ... Setting up libarchive-zip-perl (1.68-1) ... Setting up libyaml-0-2:amd64 (0.2.5-1+b1) ... Setting up fonts-glyphicons-halflings (1.009~3.4.1+dfsg-3) ... Setting up libdebhelper-perl (13.15.3) ... Setting up libmagic1t64:amd64 (1:5.45-3) ... Setting up libpython3.12-minimal:amd64 (3.12.3-1) ... Setting up x11-common (1:7.7+23) ... invoke-rc.d: could not determine current runlevel Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. Setting up libpsl5t64:amd64 (0.21.2-1.1) ... Setting up gettext-base (0.21-14+b1) ... Setting up m4 (1.4.19-4) ... Setting up file (1:5.45-3) ... Setting up libelf1t64:amd64 (0.191-1+b1) ... Setting up python-babel-localedata (2.14.0-1) ... Setting up tzdata (2024a-4) ... Current default time zone: 'Etc/UTC' Local time is now: Sun Jun 2 16:46:05 UTC 2024. Universal Time is now: Sun Jun 2 16:46:05 UTC 2024. Run 'dpkg-reconfigure tzdata' if you wish to change it. Setting up autotools-dev (20220109.1) ... Setting up libx11-data (2:1.8.7-1) ... Setting up python3-pastedeploy-tpl (3.1-1) ... Setting up autopoint (0.21-14) ... Setting up autoconf (2.71-3) ... Setting up liberasurecode1:amd64 (1.6.3-1) ... Setting up pycadf-common (3.1.1-3) ... Setting up dwz (0.15-1+b1) ... Setting up sensible-utils (0.0.22) ... Setting up libuchardet0:amd64 (0.0.8-1+b1) ... Setting up python-oslosphinx-common (4.18.0-5) ... Setting up libjson-perl (4.10000-1) ... Setting up netbase (6.4) ... Setting up sgml-base (1.31) ... Setting up libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... Setting up openssl (3.2.1-3) ... Setting up libbsd0:amd64 (0.12.2-1) ... Setting up readline-common (8.2-4) ... Setting up libxml2:amd64 (2.12.7+dfsg-2) ... Setting up fonts-font-awesome (5.0.10+really4.7.0~dfsg-4.1) ... Setting up libonig5:amd64 (6.9.9-1) ... Setting up libjs-bootstrap (3.4.1+dfsg-3) ... Setting up libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... Setting up automake (1:1.16.5-1.3) ... update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode Setting up libfile-stripnondeterminism-perl (1.14.0-1) ... Setting up python3.12-minimal (3.12.3-1) ... Setting up libice6:amd64 (2:1.0.10-1+b1) ... Setting up libxdmcp6:amd64 (1:1.1.2-3+b1) ... Setting up libxcb1:amd64 (1.17.0-2) ... Setting up gettext (0.21-14+b1) ... Setting up wget (1.24.5-1) ... Setting up libtool (2.4.7-7) ... Setting up libjq1:amd64 (1.7.1-3) ... Setting up intltool-debian (0.35.0+20060710.6) ... Setting up dh-autoreconf (20) ... Setting up ca-certificates (20240203) ... Updating certificates in /etc/ssl/certs... 146 added, 0 removed; done. Setting up python-os-api-ref-common (2.3.0+ds1-3) ... Setting up libjs-sphinxdoc (7.2.6-8) ... Setting up libreadline8t64:amd64 (8.2-4) ... Setting up dh-strip-nondeterminism (1.14.0-1) ... Setting up ieee-data (20220827.1) ... Setting up groff-base (1.23.0-4) ... Setting up xml-core (0.19) ... Setting up jq (1.7.1-3) ... Setting up libxslt1.1:amd64 (1.1.35-1+b1) ... Setting up libx11-6:amd64 (2:1.8.7-1+b1) ... Setting up libsm6:amd64 (2:1.2.3-1+b1) ... Setting up libpython3.12-stdlib:amd64 (3.12.3-1) ... Setting up po-debconf (1.0.21+nmu1) ... Setting up libpython3.11-stdlib:amd64 (3.11.9-1) ... Setting up python3.12 (3.12.3-1) ... Setting up libxext6:amd64 (2:1.3.4-1+b1) ... Setting up man-db (2.12.1-1) ... Not building database; man-db/auto-update is not 'true'. Setting up sphinx-common (7.2.6-8) ... Setting up libxt6t64:amd64 (1:1.2.1-1.2) ... Setting up libpython3-stdlib:amd64 (3.11.8-1) ... Setting up python3.11 (3.11.9-1) ... Setting up libxmu6:amd64 (2:1.1.3-3+b2) ... Setting up debhelper (13.15.3) ... Setting up python3 (3.11.8-1) ... Setting up dh-exec (0.30) ... Setting up python3-markupsafe (2.1.5-1) ... Setting up python3-wheel (0.43.0-1) ... Setting up python3-platformdirs (4.2.1-1) ... Setting up python3-tz (2024.1-2) ... Setting up python3-mock (5.1.0-1) ... Setting up python3-six (1.16.0-6) ... Setting up python3-simplejson (3.19.2-1+b1) ... Setting up python3-roman (3.3-3) ... Setting up python3-decorator (5.1.1-5) ... Setting up python3-jinja2 (3.1.3-1) ... Setting up python3-tempita (0.5.2-7) ... Setting up python3-packaging (24.0-1) ... Setting up python3-iso8601 (1.0.2-1) ... Setting up python3-wcwidth (0.2.13+dfsg1-1) ... Setting up python3-pyparsing (3.1.2-1) ... Setting up python3-certifi (2023.11.17-1) ... Setting up python3-snowballstemmer (2.2.0-4) ... Setting up python3-pyeclib (1.6.1-2) ... Setting up python3-jmespath (1.0.1-1) ... Setting up python3-greenlet (3.0.1-3) ... Setting up python3-idna (3.6-2) ... Setting up python3-wrapt (1.15.0-2+b1) ... Setting up xclip (0.13-4) ... Setting up python3-deprecation (2.1.0-3) ... Setting up python3-typing-extensions (4.12.0-1) ... Setting up python3-pip (24.0+dfsg-2) ... Setting up python3-jeepney (0.8.0-3) ... Setting up python3-urllib3 (1.26.18-2) ... Setting up python3-pluggy (1.5.0-1) ... Setting up python3-netifaces:amd64 (0.11.0-2+b2) ... Setting up python3-rpds-py (0.12.0-3) ... Setting up python3-netaddr (0.10.1-1) ... Setting up python3-trove-classifiers (2024.1.31-1) ... Setting up python3-json-pointer (2.4-2) ... Setting up python3-lxml:amd64 (5.2.1-1) ... Setting up python3-dnspython (2.6.1-1) ... Setting up python3-dulwich (0.21.6-1+b1) ... Setting up python3-dateutil (2.9.0-2) ... Setting up python3-msgpack (1.0.3-3+b1) ... Setting up python3-voluptuous (0.14.2-1) ... Setting up python3-pyasyncore (1.0.2-2) ... Setting up python3-prettytable (3.6.0-1) ... Setting up python3-tomlkit (0.12.5-1) ... Setting up python3-pathspec (0.12.1-1) ... Setting up python3-lib2to3 (3.12.3-3.1) ... Setting up python3-cffi-backend:amd64 (1.16.0-2+b2) ... Setting up python3-webob (1:1.8.7-1) ... Setting up python3-imagesize (1.4.1-1) ... Setting up python3-blinker (1.8.2-1) ... Setting up python3-pkg-resources (68.1.2-2) ... Setting up python3-distutils (3.12.3-3.1) ... python3.12: can't get files for byte-compilation Setting up python3-more-itertools (10.2.0-1) ... Setting up python3-attr (23.2.0-2) ... Setting up python3-jaraco.classes (3.2.1-4) ... Setting up python3-jaraco.functools (4.0.0-1) ... Setting up python3-setuptools (68.1.2-2) ... Setting up python3-jaraco.context (4.3.0-1) ... Setting up python3-jwt (2.7.0-1) ... Setting up python3-babel (2.14.0-1) ... update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode Setting up python3-pbr (6.0.0-1) ... Setting up python3-charset-normalizer (3.3.2-1) ... Setting up python3-alabaster (0.7.12-1) ... Setting up python3-jsonpatch (1.32-3) ... update-alternatives: using /usr/bin/json-patch-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode Setting up python3-ply (3.11-6) ... Setting up python3-autopage (0.4.0-3) ... Setting up python3-pyinotify (0.9.6-2) ... Setting up python3-extras (1.0.0-5) ... Setting up python3-memcache (1.62-1) ... Setting up python3-all (3.11.8-1) ... Setting up python3-requestsexceptions (1.4.0-3) ... Setting up python3-coverage (7.2.7+dfsg1-1+b1) ... Setting up python3-yaml (6.0.1-2) ... Setting up python3-pyperclip (1.8.2-2) ... Setting up python3-debtcollector (3.0.0-2) ... Setting up python3-zipp (1.0.0-6) ... Setting up python3-setuptools-scm (8.0.4-2) ... Setting up openstack-pkg-tools (126) ... Setting up python3-pycparser (2.22-1) ... Setting up python3-eventlet (0.35.1-1) ... Setting up python3-pygments (2.17.2+dfsg-1) ... Setting up python3-chardet (5.2.0+dfsg-1) ... Setting up python3-rfc3986 (2.0.0-2) ... Setting up python3-paste (3.10.0-1) ... Setting up python3-cmd2 (2.4.3+ds-2) ... Setting up python3-cryptography (42.0.5-2) ... Setting up python3-requests (2.31.0+dfsg-2) ... Setting up python3-oslosphinx (4.18.0-5) ... Setting up python3-oslo.context (5.5.0-2) ... Setting up python3-uhashring (2.3-2) ... Setting up python3-xattr (0.10.1-1) ... Setting up python3-os-service-types (1.7.0-3) ... Setting up python3-stevedore (5.2.0-2) ... Setting up python3-hatchling (1.24.2-1) ... Setting up python3-swiftclient (1:4.5.0-2) ... Setting up python3-binary-memcached (0.31.2+dfsg1-2) ... Setting up python3-oslo.i18n (6.3.0-2) ... Setting up python3-mako (1.3.2-1) ... Setting up python3-hatch-vcs (0.4.0-1) ... Setting up python3-referencing (0.31.0-1) ... Setting up dh-python (6.20240422) ... Setting up python3-importlib-metadata (7.1.0-1) ... Setting up python3-openssl (24.1.0-1) ... Setting up python3-pastedeploy (3.1-1) ... Setting up python3-cffi (1.16.0-2) ... Setting up python3-oslo.utils (7.1.0-2) ... Setting up python3-dogpile.cache (1.3.1-1) ... Setting up python3-oauthlib (3.2.2-1) ... Setting up python3-secretstorage (3.3.3-3) ... Setting up python3-requests-mock (1.11.0-1) ... Setting up python3-botocore (1.34.46+repack-1) ... Setting up python3-keyring (25.2.1-1) ... Setting up python3-jsonschema-specifications (2023.12.1-1) ... Setting up python3-keystoneauth1 (5.6.0-2) ... Setting up python3-s3transfer (0.10.1-1) ... Setting up python3-oslo.serialization (5.4.0-2) ... Setting up python3-novaclient (2:18.6.0-3) ... Setting up python3-jsonschema (4.19.2-3) ... Setting up python3-boto3 (1.34.46+dfsg-1) ... Setting up python3-openstacksdk (3.0.0-3) ... Setting up python3-cinderclient (1:9.5.0-2) ... Setting up python3-os-client-config (2.1.0-2) ... Setting up python3-warlock (2.0.1-4) ... Setting up python3-glanceclient (1:4.5.0-2) ... Setting up python3-testtools (2.7.1-3) ... Setting up python3-subunit (1.4.2-3) ... Setting up python3-fixtures (4.1.0-2) ... Setting up subunit (1.4.2-3) ... Processing triggers for libc-bin (2.38-12) ... Processing triggers for sgml-base (1.31) ... Setting up docutils-common (0.20.1+dfsg-3) ... Processing triggers for sgml-base (1.31) ... Setting up python3-docutils (0.20.1+dfsg-3) ... Setting up python3-cliff (4.6.0-2) ... Setting up python3-oslo.config (1:9.4.0-2) ... Setting up python3-keystoneclient (1:5.4.0-2) ... Setting up python3-osc-lib (3.0.1-2) ... Setting up python3-sphinx (7.2.6-8) ... Setting up python3-oslo.log (5.5.1-2) ... Setting up python3-pycadf (3.1.1-3) ... Setting up python3-stestr (4.1.0-2) ... Setting up python3-openstackclient (6.6.0-3) ... Setting up python3-openstackdocstheme (3.2.0-1) ... Setting up python3-neutronclient (1:11.2.0-3) ... Setting up python3-oslo.cache (3.7.0-2) ... Setting up python3-os-testr (3.0.0-3) ... Setting up python3-keystonemiddleware (10.6.0-2) ... Setting up python3-os-api-ref (2.3.0+ds1-3) ... Processing triggers for ca-certificates (20240203) ... Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done. Reading package lists... Building dependency tree... Reading state information... Reading extended state information... Initializing package states... Writing extended state information... Building tag database... -> Finished parsing the build-deps I: Building the package I: Running cd /build/reproducible-path/swift-2.33.0/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games" HOME="/nonexistent/first-build" dpkg-buildpackage -us -uc -b && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games" HOME="/nonexistent/first-build" dpkg-genchanges -S > ../swift_2.33.0-4_source.changes dpkg-buildpackage: info: source package swift dpkg-buildpackage: info: source version 2.33.0-4 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Thomas Goirand dpkg-source --before-build . dpkg-buildpackage: info: host architecture amd64 dpkg-source: info: using options from swift-2.33.0/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/ --extend-diff-ignore=^[.]gitreview$ debian/rules clean make: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh clean --buildsystem=pybuild --with python3,sphinxdoc debian/rules override_dh_auto_clean make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' rm -rf build .stestr *.egg-info .pybuild find . -iname '*.pyc' -delete for i in $(find . -type d -iname __pycache__) ; do rm -rf $i ; done make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' dh_autoreconf_clean -O--buildsystem=pybuild dh_clean -O--buildsystem=pybuild debian/rules binary make: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh binary --buildsystem=pybuild --with python3,sphinxdoc dh_update_autotools_config -O--buildsystem=pybuild dh_autoreconf -O--buildsystem=pybuild dh_auto_configure -O--buildsystem=pybuild I: pybuild base:311: python3.12 setup.py config /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running config I: pybuild base:311: python3.11 setup.py config /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running config debian/rules override_dh_auto_build make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' dh_auto_build I: pybuild base:311: /usr/bin/python3.12 setup.py build /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running build running build_py creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/reconciler.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/sync_store.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/sharder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container copying swift/container/sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/container creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/account.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/container.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy/controllers copying swift/proxy/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy copying swift/proxy/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/proxy creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/mem_diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/ssync_receiver.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/reconstructor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/mem_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/ssync_sender.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/expirer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj copying swift/obj/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/libc.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/timestamp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils copying swift/common/utils/ipaddrs.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/utils creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3response.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/subresource.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_handlers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/etree.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3api.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/exception.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3token.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3request.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/reaper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/account/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/account copying swift/common/registry.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/container_sync_realms.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/db_auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/error_limiter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/daemon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/direct_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/constraints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/digest.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/swob.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/db_replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/header_key_dict.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/wsgi.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/storage_policy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/linkat.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/internal_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/bufferedhttp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/request_helpers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/db.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/splice.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/http_protocol.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/manager.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/base_storage_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/memcached.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common copying swift/common/http.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/composite_builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring copying swift/common/ring/ring.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/ring creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/html_viewer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/profile_model.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/x_profile creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/legacy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/object_versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/versioned_writes creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj/watchers copying swift/obj/watchers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj/watchers copying swift/obj/watchers/dark_data.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/obj/watchers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kmip_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kms_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/encrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/crypto_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/decrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/crypto creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/bucket.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/tagging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/s3_acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/service.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_delete.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/location.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/object_lock.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_upload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/controllers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/shard-info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/drive_full_checker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/relinker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/container_deleter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/form_signature.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/recon_cron.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/reload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/manage_shard_ranges.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/ringcomposer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/ringbuilder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/dispersion_report.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/cli/ring_builder_analyzer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/cli copying swift/common/middleware/keystoneauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/xprofile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/memcache.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/read_only.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/list_endpoints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/listing_formats.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/proxy_logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/copy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/container_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/symlink.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/staticweb.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/crossdomain.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/slo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/tempauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/container_sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/domain_remap.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/backend_ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/etag_quoter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/formpost.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/bulk.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/gatekeeper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/tempurl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/account_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/name_check.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/dlo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/catch_errors.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/healthcheck.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/cname_lookup.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/common/middleware/ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware copying swift/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift running egg_info creating swift.egg-info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Processing SOURCES.txt writing manifest file 'swift.egg-info/SOURCES.txt' warning: no files found matching 'ChangeLog' warning: no previously-included files found matching '.gitignore' warning: no previously-included files found matching '.gitreview' warning: no previously-included files matching '*.pyc' found anywhere in distribution reading manifest template 'MANIFEST.in' warning: no files found matching 'babel.cfg' adding license file 'LICENSE' adding license file 'AUTHORS' writing manifest file 'swift.egg-info/SOURCES.txt' /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/access_control_policy.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/bucket_logging_status.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/common.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_object_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_part_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/error.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_bucket_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_parts_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_versions_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/location_constraint.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/versioning_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/common/middleware/s3api/schema creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/de creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/de/LC_MESSAGES copying swift/locale/de/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/de/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/en_GB creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/en_GB/LC_MESSAGES copying swift/locale/en_GB/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/en_GB/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/es creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/es/LC_MESSAGES copying swift/locale/es/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/es/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/fr creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/fr/LC_MESSAGES copying swift/locale/fr/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/fr/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/it creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/it/LC_MESSAGES copying swift/locale/it/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/it/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ja creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ja/LC_MESSAGES copying swift/locale/ja/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ja/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ko_KR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ko_KR/LC_MESSAGES copying swift/locale/ko_KR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ko_KR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/pt_BR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/pt_BR/LC_MESSAGES copying swift/locale/pt_BR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/pt_BR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ru creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ru/LC_MESSAGES copying swift/locale/ru/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/ru/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/tr_TR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/tr_TR/LC_MESSAGES copying swift/locale/tr_TR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/tr_TR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_CN creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_CN/LC_MESSAGES copying swift/locale/zh_CN/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_CN/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_TW creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_TW/LC_MESSAGES copying swift/locale/zh_TW/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.12/build/swift/locale/zh_TW/LC_MESSAGES running build_scripts creating build creating build/scripts-3.12 copying and adjusting bin/swift-account-audit -> build/scripts-3.12 copying and adjusting bin/swift-account-auditor -> build/scripts-3.12 copying and adjusting bin/swift-account-info -> build/scripts-3.12 copying and adjusting bin/swift-account-reaper -> build/scripts-3.12 copying and adjusting bin/swift-account-replicator -> build/scripts-3.12 copying and adjusting bin/swift-account-server -> build/scripts-3.12 copying and adjusting bin/swift-config -> build/scripts-3.12 copying and adjusting bin/swift-container-auditor -> build/scripts-3.12 copying and adjusting bin/swift-container-info -> build/scripts-3.12 copying and adjusting bin/swift-container-replicator -> build/scripts-3.12 copying and adjusting bin/swift-container-server -> build/scripts-3.12 copying and adjusting bin/swift-container-sharder -> build/scripts-3.12 copying and adjusting bin/swift-container-sync -> build/scripts-3.12 copying and adjusting bin/swift-container-updater -> build/scripts-3.12 copying and adjusting bin/swift-container-reconciler -> build/scripts-3.12 copying and adjusting bin/swift-reconciler-enqueue -> build/scripts-3.12 copying and adjusting bin/swift-dispersion-populate -> build/scripts-3.12 copying and adjusting bin/swift-dispersion-report -> build/scripts-3.12 copying and adjusting bin/swift-drive-audit -> build/scripts-3.12 copying and adjusting bin/swift-form-signature -> build/scripts-3.12 copying and adjusting bin/swift-get-nodes -> build/scripts-3.12 copying and adjusting bin/swift-init -> build/scripts-3.12 copying and adjusting bin/swift-object-auditor -> build/scripts-3.12 copying and adjusting bin/swift-object-expirer -> build/scripts-3.12 copying and adjusting bin/swift-object-info -> build/scripts-3.12 copying and adjusting bin/swift-object-replicator -> build/scripts-3.12 copying and adjusting bin/swift-object-reconstructor -> build/scripts-3.12 copying and adjusting bin/swift-object-relinker -> build/scripts-3.12 copying and adjusting bin/swift-object-server -> build/scripts-3.12 copying and adjusting bin/swift-object-updater -> build/scripts-3.12 copying and adjusting bin/swift-oldies -> build/scripts-3.12 copying and adjusting bin/swift-orphans -> build/scripts-3.12 copying and adjusting bin/swift-proxy-server -> build/scripts-3.12 copying and adjusting bin/swift-recon -> build/scripts-3.12 copying and adjusting bin/swift-recon-cron -> build/scripts-3.12 copying and adjusting bin/swift-ring-builder -> build/scripts-3.12 copying and adjusting bin/swift-ring-builder-analyzer -> build/scripts-3.12 copying and adjusting bin/swift-ring-composer -> build/scripts-3.12 changing mode of build/scripts-3.12/swift-account-audit from 644 to 755 changing mode of build/scripts-3.12/swift-account-auditor from 644 to 755 changing mode of build/scripts-3.12/swift-account-info from 644 to 755 changing mode of build/scripts-3.12/swift-account-reaper from 644 to 755 changing mode of build/scripts-3.12/swift-account-replicator from 644 to 755 changing mode of build/scripts-3.12/swift-account-server from 644 to 755 changing mode of build/scripts-3.12/swift-config from 644 to 755 changing mode of build/scripts-3.12/swift-container-auditor from 644 to 755 changing mode of build/scripts-3.12/swift-container-info from 644 to 755 changing mode of build/scripts-3.12/swift-container-replicator from 644 to 755 changing mode of build/scripts-3.12/swift-container-server from 644 to 755 changing mode of build/scripts-3.12/swift-container-sharder from 644 to 755 changing mode of build/scripts-3.12/swift-container-sync from 644 to 755 changing mode of build/scripts-3.12/swift-container-updater from 644 to 755 changing mode of build/scripts-3.12/swift-container-reconciler from 644 to 755 changing mode of build/scripts-3.12/swift-reconciler-enqueue from 644 to 755 changing mode of build/scripts-3.12/swift-dispersion-populate from 644 to 755 changing mode of build/scripts-3.12/swift-dispersion-report from 644 to 755 changing mode of build/scripts-3.12/swift-drive-audit from 644 to 755 changing mode of build/scripts-3.12/swift-form-signature from 644 to 755 changing mode of build/scripts-3.12/swift-get-nodes from 644 to 755 changing mode of build/scripts-3.12/swift-init from 644 to 755 changing mode of build/scripts-3.12/swift-object-auditor from 644 to 755 changing mode of build/scripts-3.12/swift-object-expirer from 644 to 755 changing mode of build/scripts-3.12/swift-object-info from 644 to 755 changing mode of build/scripts-3.12/swift-object-replicator from 644 to 755 changing mode of build/scripts-3.12/swift-object-reconstructor from 644 to 755 changing mode of build/scripts-3.12/swift-object-relinker from 644 to 755 changing mode of build/scripts-3.12/swift-object-server from 644 to 755 changing mode of build/scripts-3.12/swift-object-updater from 644 to 755 changing mode of build/scripts-3.12/swift-oldies from 644 to 755 changing mode of build/scripts-3.12/swift-orphans from 644 to 755 changing mode of build/scripts-3.12/swift-proxy-server from 644 to 755 changing mode of build/scripts-3.12/swift-recon from 644 to 755 changing mode of build/scripts-3.12/swift-recon-cron from 644 to 755 changing mode of build/scripts-3.12/swift-ring-builder from 644 to 755 changing mode of build/scripts-3.12/swift-ring-builder-analyzer from 644 to 755 changing mode of build/scripts-3.12/swift-ring-composer from 644 to 755 I: pybuild base:311: /usr/bin/python3 setup.py build /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running build running build_py creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/reconciler.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/sync_store.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/sharder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container copying swift/container/sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/container creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3response.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/subresource.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_handlers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/etree.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3api.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/exception.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3token.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3request.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api copying swift/common/middleware/keystoneauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/xprofile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/memcache.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/read_only.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/list_endpoints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/listing_formats.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/proxy_logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/copy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/container_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/symlink.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/staticweb.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/crossdomain.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/slo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/tempauth.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/container_sync.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/domain_remap.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/backend_ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/etag_quoter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/formpost.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/bulk.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/gatekeeper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/tempurl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/account_quotas.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/name_check.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/dlo.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/catch_errors.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/healthcheck.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/cname_lookup.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/middleware/ratelimit.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware copying swift/common/registry.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/container_sync_realms.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/db_auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/error_limiter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/daemon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/direct_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/constraints.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/digest.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/swob.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/db_replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/header_key_dict.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/wsgi.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/storage_policy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/linkat.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/internal_client.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/bufferedhttp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/request_helpers.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/db.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/splice.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/http_protocol.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/manager.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/base_storage_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/memcached.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common copying swift/common/http.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/shard-info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/drive_full_checker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/relinker.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/container_deleter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/form_signature.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/recon_cron.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/reload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/manage_shard_ranges.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/ringcomposer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/ringbuilder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/dispersion_report.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/recon.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli copying swift/cli/ring_builder_analyzer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/cli creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/composite_builder.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring copying swift/common/ring/ring.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/ring creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/logging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/bucket.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/tagging.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/s3_acl.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/service.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_delete.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/location.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/object_lock.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_upload.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/controllers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj/watchers copying swift/obj/watchers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj/watchers copying swift/obj/watchers/dark_data.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj/watchers creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/reaper.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/backend.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account copying swift/account/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/account creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/libc.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/timestamp.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils copying swift/common/utils/ipaddrs.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/utils creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/html_viewer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/exceptions.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/profile_model.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/x_profile creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/base.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/account.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/container.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/info.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/proxy/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy/controllers copying swift/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift copying swift/obj/mem_diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/ssync_receiver.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/reconstructor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/mem_server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/ssync_sender.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/updater.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/auditor.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/diskfile.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/expirer.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj copying swift/obj/replicator.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/obj creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kmip_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/kms_keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/keymaster.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/encrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/crypto_utils.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto copying swift/common/middleware/crypto/decrypter.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/crypto creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/legacy.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/object_versioning.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/versioned_writes copying swift/proxy/__init__.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy copying swift/proxy/server.py -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/proxy running egg_info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Reusing existing SOURCES.txt /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/access_control_policy.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/bucket_logging_status.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/common.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_object_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_part_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/error.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_bucket_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_parts_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_versions_result.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/location_constraint.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/versioning_configuration.rng -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/common/middleware/s3api/schema creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/de creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/de/LC_MESSAGES copying swift/locale/de/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/de/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/en_GB creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/en_GB/LC_MESSAGES copying swift/locale/en_GB/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/en_GB/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/es creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/es/LC_MESSAGES copying swift/locale/es/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/es/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/fr creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/fr/LC_MESSAGES copying swift/locale/fr/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/fr/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/it creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/it/LC_MESSAGES copying swift/locale/it/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/it/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ja creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ja/LC_MESSAGES copying swift/locale/ja/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ja/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ko_KR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ko_KR/LC_MESSAGES copying swift/locale/ko_KR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ko_KR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/pt_BR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/pt_BR/LC_MESSAGES copying swift/locale/pt_BR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/pt_BR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ru creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ru/LC_MESSAGES copying swift/locale/ru/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/ru/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/tr_TR creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/tr_TR/LC_MESSAGES copying swift/locale/tr_TR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/tr_TR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_CN creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_CN/LC_MESSAGES copying swift/locale/zh_CN/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_CN/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_TW creating /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_TW/LC_MESSAGES copying swift/locale/zh_TW/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/.pybuild/cpython3_3.11/build/swift/locale/zh_TW/LC_MESSAGES running build_scripts creating build/scripts-3.11 copying and adjusting bin/swift-account-audit -> build/scripts-3.11 copying and adjusting bin/swift-account-auditor -> build/scripts-3.11 copying and adjusting bin/swift-account-info -> build/scripts-3.11 copying and adjusting bin/swift-account-reaper -> build/scripts-3.11 copying and adjusting bin/swift-account-replicator -> build/scripts-3.11 copying and adjusting bin/swift-account-server -> build/scripts-3.11 copying and adjusting bin/swift-config -> build/scripts-3.11 copying and adjusting bin/swift-container-auditor -> build/scripts-3.11 copying and adjusting bin/swift-container-info -> build/scripts-3.11 copying and adjusting bin/swift-container-replicator -> build/scripts-3.11 copying and adjusting bin/swift-container-server -> build/scripts-3.11 copying and adjusting bin/swift-container-sharder -> build/scripts-3.11 copying and adjusting bin/swift-container-sync -> build/scripts-3.11 copying and adjusting bin/swift-container-updater -> build/scripts-3.11 copying and adjusting bin/swift-container-reconciler -> build/scripts-3.11 copying and adjusting bin/swift-reconciler-enqueue -> build/scripts-3.11 copying and adjusting bin/swift-dispersion-populate -> build/scripts-3.11 copying and adjusting bin/swift-dispersion-report -> build/scripts-3.11 copying and adjusting bin/swift-drive-audit -> build/scripts-3.11 copying and adjusting bin/swift-form-signature -> build/scripts-3.11 copying and adjusting bin/swift-get-nodes -> build/scripts-3.11 copying and adjusting bin/swift-init -> build/scripts-3.11 copying and adjusting bin/swift-object-auditor -> build/scripts-3.11 copying and adjusting bin/swift-object-expirer -> build/scripts-3.11 copying and adjusting bin/swift-object-info -> build/scripts-3.11 copying and adjusting bin/swift-object-replicator -> build/scripts-3.11 copying and adjusting bin/swift-object-reconstructor -> build/scripts-3.11 copying and adjusting bin/swift-object-relinker -> build/scripts-3.11 copying and adjusting bin/swift-object-server -> build/scripts-3.11 copying and adjusting bin/swift-object-updater -> build/scripts-3.11 copying and adjusting bin/swift-oldies -> build/scripts-3.11 copying and adjusting bin/swift-orphans -> build/scripts-3.11 copying and adjusting bin/swift-proxy-server -> build/scripts-3.11 copying and adjusting bin/swift-recon -> build/scripts-3.11 copying and adjusting bin/swift-recon-cron -> build/scripts-3.11 copying and adjusting bin/swift-ring-builder -> build/scripts-3.11 copying and adjusting bin/swift-ring-builder-analyzer -> build/scripts-3.11 copying and adjusting bin/swift-ring-composer -> build/scripts-3.11 changing mode of build/scripts-3.11/swift-account-audit from 644 to 755 changing mode of build/scripts-3.11/swift-account-auditor from 644 to 755 changing mode of build/scripts-3.11/swift-account-info from 644 to 755 changing mode of build/scripts-3.11/swift-account-reaper from 644 to 755 changing mode of build/scripts-3.11/swift-account-replicator from 644 to 755 changing mode of build/scripts-3.11/swift-account-server from 644 to 755 changing mode of build/scripts-3.11/swift-config from 644 to 755 changing mode of build/scripts-3.11/swift-container-auditor from 644 to 755 changing mode of build/scripts-3.11/swift-container-info from 644 to 755 changing mode of build/scripts-3.11/swift-container-replicator from 644 to 755 changing mode of build/scripts-3.11/swift-container-server from 644 to 755 changing mode of build/scripts-3.11/swift-container-sharder from 644 to 755 changing mode of build/scripts-3.11/swift-container-sync from 644 to 755 changing mode of build/scripts-3.11/swift-container-updater from 644 to 755 changing mode of build/scripts-3.11/swift-container-reconciler from 644 to 755 changing mode of build/scripts-3.11/swift-reconciler-enqueue from 644 to 755 changing mode of build/scripts-3.11/swift-dispersion-populate from 644 to 755 changing mode of build/scripts-3.11/swift-dispersion-report from 644 to 755 changing mode of build/scripts-3.11/swift-drive-audit from 644 to 755 changing mode of build/scripts-3.11/swift-form-signature from 644 to 755 changing mode of build/scripts-3.11/swift-get-nodes from 644 to 755 changing mode of build/scripts-3.11/swift-init from 644 to 755 changing mode of build/scripts-3.11/swift-object-auditor from 644 to 755 changing mode of build/scripts-3.11/swift-object-expirer from 644 to 755 changing mode of build/scripts-3.11/swift-object-info from 644 to 755 changing mode of build/scripts-3.11/swift-object-replicator from 644 to 755 changing mode of build/scripts-3.11/swift-object-reconstructor from 644 to 755 changing mode of build/scripts-3.11/swift-object-relinker from 644 to 755 changing mode of build/scripts-3.11/swift-object-server from 644 to 755 changing mode of build/scripts-3.11/swift-object-updater from 644 to 755 changing mode of build/scripts-3.11/swift-oldies from 644 to 755 changing mode of build/scripts-3.11/swift-orphans from 644 to 755 changing mode of build/scripts-3.11/swift-proxy-server from 644 to 755 changing mode of build/scripts-3.11/swift-recon from 644 to 755 changing mode of build/scripts-3.11/swift-recon-cron from 644 to 755 changing mode of build/scripts-3.11/swift-ring-builder from 644 to 755 changing mode of build/scripts-3.11/swift-ring-builder-analyzer from 644 to 755 changing mode of build/scripts-3.11/swift-ring-composer from 644 to 755 /usr/share/openstack-pkg-tools/pkgos_insert_include pkgos_func python3-swift.postinst set -e ; set -x ; \ for i in debian/swift-account.swift-account-auditor.init.in \ debian/swift-account.swift-account-reaper.init.in \ debian/swift-account.swift-account-replicator.init.in \ debian/swift-container.swift-container-auditor.init.in \ debian/swift-container.swift-container-reconciler.init.in \ debian/swift-container.swift-container-replicator.init.in \ debian/swift-container.swift-container-sharder.init.in \ debian/swift-container.swift-container-sync.init.in \ debian/swift-container.swift-container-updater.init.in \ debian/swift-object-expirer.swift-object-expirer.init.in \ debian/swift-object.swift-object-auditor.init.in \ debian/swift-object.swift-object.init.in \ debian/swift-object.swift-object-reconstructor.init.in \ debian/swift-object.swift-object-replicator.init.in \ debian/swift-object.swift-object-updater.init.in \ ; do \ debian/gen-init $i ; \ done + debian/gen-init debian/swift-account.swift-account-auditor.init.in + debian/gen-init debian/swift-account.swift-account-reaper.init.in + debian/gen-init debian/swift-account.swift-account-replicator.init.in + debian/gen-init debian/swift-container.swift-container-auditor.init.in + debian/gen-init debian/swift-container.swift-container-reconciler.init.in + debian/gen-init debian/swift-container.swift-container-replicator.init.in + debian/gen-init debian/swift-container.swift-container-sharder.init.in + debian/gen-init debian/swift-container.swift-container-sync.init.in + debian/gen-init debian/swift-container.swift-container-updater.init.in + debian/gen-init debian/swift-object-expirer.swift-object-expirer.init.in + debian/gen-init debian/swift-object.swift-object-auditor.init.in + debian/gen-init debian/swift-object.swift-object.init.in + debian/gen-init debian/swift-object.swift-object-reconstructor.init.in + debian/gen-init debian/swift-object.swift-object-replicator.init.in + debian/gen-init debian/swift-object.swift-object-updater.init.in set -e ; set -x ; \ for i in debian/swift-proxy.swift-proxy.init \ debian/swift-account.swift-account.init \ debian/swift-container.swift-container.init \ ; do \ if [ $DEB_USE_UWSGI = "yes" ] ; then \ cp $i.in $i ; \ cat /usr/share/openstack-pkg-tools/init-script-template >>$i ; \ pkgos-gen-systemd-unit $i.in ; \ else \ CONFIG=`echo $i | cut -d/ -f2 | cut -d. -f1 | sed -e s/swift-//`-server debian/gen-init $i.in ; \ fi ; \ done + [ no = yes ] + echo debian/swift-proxy.swift-proxy.init + cut -d/ -f2 + cut -d. -f1 + sed -e s/swift-// + CONFIG=proxy-server debian/gen-init debian/swift-proxy.swift-proxy.init.in + [ no = yes ] + echo debian/swift-account.swift-account.init + cut -d/ -f2 + cut -d. -f1 + sed -e s/swift-// + CONFIG=account-server debian/gen-init debian/swift-account.swift-account.init.in + [ no = yes ] + echo debian/swift-container.swift-container.init + cut -d/ -f2 + cut -d. -f1 + sed -e s/swift-// + CONFIG=container-server debian/gen-init debian/swift-container.swift-container.init.in make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' debian/rules override_dh_auto_test make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' echo "Do nothing..." Do nothing... make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' create-stamp debian/debhelper-build-stamp dh_testroot -O--buildsystem=pybuild dh_prep -O--buildsystem=pybuild debian/rules override_dh_auto_install make[1]: Entering directory '/build/reproducible-path/swift-2.33.0' make[1]: pyversions: No such file or directory py3versions: no X-Python3-Version in control file, using supported versions debian/rules:69: warning: overriding recipe for target 'override_dh_installinit' /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit' debian/rules:77: warning: overriding recipe for target 'override_dh_installsystemd' /usr/share/openstack-pkg-tools/pkgos.make:40: warning: ignoring old recipe for target 'override_dh_installsystemd' debian/rules:85: warning: overriding recipe for target 'gen-init-configurations' /usr/share/openstack-pkg-tools/pkgos.make:19: warning: ignoring old recipe for target 'gen-init-configurations' pkgos-dh_auto_install --no-py2 --in-tmp + PKGOS_IN_TMP=no + echo WARNING: --no-py2 is deprecated and always on. WARNING: --no-py2 is deprecated and always on. + shift + PKGOS_IN_TMP=yes + shift + dpkg-parsechangelog -SSource + SRC_PKG_NAME=swift + echo swift + sed s/python-// + PY_MODULE_NAME=swift + py3versions -vr + PYTHON3S=3.12 3.11 + [ yes = yes ] + TARGET_DIR=tmp + pwd + python3.12 setup.py install --install-layout=deb --root /build/reproducible-path/swift-2.33.0/debian/tmp /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running install running build running build_py creating build/lib creating build/lib/swift creating build/lib/swift/common creating build/lib/swift/common/ring copying swift/common/ring/builder.py -> build/lib/swift/common/ring copying swift/common/ring/__init__.py -> build/lib/swift/common/ring copying swift/common/ring/composite_builder.py -> build/lib/swift/common/ring copying swift/common/ring/utils.py -> build/lib/swift/common/ring copying swift/common/ring/ring.py -> build/lib/swift/common/ring creating build/lib/swift/cli copying swift/cli/shard-info.py -> build/lib/swift/cli copying swift/cli/drive_full_checker.py -> build/lib/swift/cli copying swift/cli/relinker.py -> build/lib/swift/cli copying swift/cli/container_deleter.py -> build/lib/swift/cli copying swift/cli/__init__.py -> build/lib/swift/cli copying swift/cli/form_signature.py -> build/lib/swift/cli copying swift/cli/recon_cron.py -> build/lib/swift/cli copying swift/cli/reload.py -> build/lib/swift/cli copying swift/cli/info.py -> build/lib/swift/cli copying swift/cli/manage_shard_ranges.py -> build/lib/swift/cli copying swift/cli/ringcomposer.py -> build/lib/swift/cli copying swift/cli/ringbuilder.py -> build/lib/swift/cli copying swift/cli/dispersion_report.py -> build/lib/swift/cli copying swift/cli/recon.py -> build/lib/swift/cli copying swift/cli/ring_builder_analyzer.py -> build/lib/swift/cli creating build/lib/swift/common/middleware creating build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3response.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/subresource.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_utils.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/acl_handlers.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/__init__.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/etree.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3api.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/exception.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3token.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/utils.py -> build/lib/swift/common/middleware/s3api copying swift/common/middleware/s3api/s3request.py -> build/lib/swift/common/middleware/s3api creating build/lib/swift/proxy creating build/lib/swift/proxy/controllers copying swift/proxy/controllers/base.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/account.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/container.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/__init__.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/info.py -> build/lib/swift/proxy/controllers copying swift/proxy/controllers/obj.py -> build/lib/swift/proxy/controllers creating build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/logging.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/bucket.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/base.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/acl.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/tagging.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/s3_acl.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/service.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_delete.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/__init__.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/location.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/obj.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/object_lock.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/multi_upload.py -> build/lib/swift/common/middleware/s3api/controllers copying swift/common/middleware/s3api/controllers/versioning.py -> build/lib/swift/common/middleware/s3api/controllers creating build/lib/swift/common/utils copying swift/common/utils/libc.py -> build/lib/swift/common/utils copying swift/common/utils/timestamp.py -> build/lib/swift/common/utils copying swift/common/utils/__init__.py -> build/lib/swift/common/utils copying swift/common/utils/ipaddrs.py -> build/lib/swift/common/utils creating build/lib/swift/obj creating build/lib/swift/obj/watchers copying swift/obj/watchers/__init__.py -> build/lib/swift/obj/watchers copying swift/obj/watchers/dark_data.py -> build/lib/swift/obj/watchers creating build/lib/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/legacy.py -> build/lib/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/__init__.py -> build/lib/swift/common/middleware/versioned_writes copying swift/common/middleware/versioned_writes/object_versioning.py -> build/lib/swift/common/middleware/versioned_writes copying swift/proxy/__init__.py -> build/lib/swift/proxy copying swift/proxy/server.py -> build/lib/swift/proxy copying swift/obj/mem_diskfile.py -> build/lib/swift/obj copying swift/obj/ssync_receiver.py -> build/lib/swift/obj copying swift/obj/reconstructor.py -> build/lib/swift/obj copying swift/obj/mem_server.py -> build/lib/swift/obj copying swift/obj/ssync_sender.py -> build/lib/swift/obj copying swift/obj/updater.py -> build/lib/swift/obj copying swift/obj/__init__.py -> build/lib/swift/obj copying swift/obj/auditor.py -> build/lib/swift/obj copying swift/obj/diskfile.py -> build/lib/swift/obj copying swift/obj/expirer.py -> build/lib/swift/obj copying swift/obj/server.py -> build/lib/swift/obj copying swift/obj/replicator.py -> build/lib/swift/obj creating build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/html_viewer.py -> build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/exceptions.py -> build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/__init__.py -> build/lib/swift/common/middleware/x_profile copying swift/common/middleware/x_profile/profile_model.py -> build/lib/swift/common/middleware/x_profile creating build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/kmip_keymaster.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/__init__.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/kms_keymaster.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/keymaster.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/encrypter.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/crypto_utils.py -> build/lib/swift/common/middleware/crypto copying swift/common/middleware/crypto/decrypter.py -> build/lib/swift/common/middleware/crypto creating build/lib/swift/account copying swift/account/reaper.py -> build/lib/swift/account copying swift/account/__init__.py -> build/lib/swift/account copying swift/account/auditor.py -> build/lib/swift/account copying swift/account/backend.py -> build/lib/swift/account copying swift/account/server.py -> build/lib/swift/account copying swift/account/utils.py -> build/lib/swift/account copying swift/account/replicator.py -> build/lib/swift/account copying swift/common/middleware/keystoneauth.py -> build/lib/swift/common/middleware copying swift/common/middleware/xprofile.py -> build/lib/swift/common/middleware copying swift/common/middleware/memcache.py -> build/lib/swift/common/middleware copying swift/common/middleware/read_only.py -> build/lib/swift/common/middleware copying swift/common/middleware/list_endpoints.py -> build/lib/swift/common/middleware copying swift/common/middleware/listing_formats.py -> build/lib/swift/common/middleware copying swift/common/middleware/proxy_logging.py -> build/lib/swift/common/middleware copying swift/common/middleware/copy.py -> build/lib/swift/common/middleware copying swift/common/middleware/container_quotas.py -> build/lib/swift/common/middleware copying swift/common/middleware/symlink.py -> build/lib/swift/common/middleware copying swift/common/middleware/acl.py -> build/lib/swift/common/middleware copying swift/common/middleware/staticweb.py -> build/lib/swift/common/middleware copying swift/common/middleware/crossdomain.py -> build/lib/swift/common/middleware copying swift/common/middleware/__init__.py -> build/lib/swift/common/middleware copying swift/common/middleware/slo.py -> build/lib/swift/common/middleware copying swift/common/middleware/tempauth.py -> build/lib/swift/common/middleware copying swift/common/middleware/container_sync.py -> build/lib/swift/common/middleware copying swift/common/middleware/domain_remap.py -> build/lib/swift/common/middleware copying swift/common/middleware/backend_ratelimit.py -> build/lib/swift/common/middleware copying swift/common/middleware/etag_quoter.py -> build/lib/swift/common/middleware copying swift/common/middleware/formpost.py -> build/lib/swift/common/middleware copying swift/common/middleware/bulk.py -> build/lib/swift/common/middleware copying swift/common/middleware/gatekeeper.py -> build/lib/swift/common/middleware copying swift/common/middleware/tempurl.py -> build/lib/swift/common/middleware copying swift/common/middleware/account_quotas.py -> build/lib/swift/common/middleware copying swift/common/middleware/name_check.py -> build/lib/swift/common/middleware copying swift/common/middleware/dlo.py -> build/lib/swift/common/middleware copying swift/common/middleware/catch_errors.py -> build/lib/swift/common/middleware copying swift/common/middleware/healthcheck.py -> build/lib/swift/common/middleware copying swift/common/middleware/cname_lookup.py -> build/lib/swift/common/middleware copying swift/common/middleware/recon.py -> build/lib/swift/common/middleware copying swift/common/middleware/ratelimit.py -> build/lib/swift/common/middleware copying swift/__init__.py -> build/lib/swift copying swift/common/registry.py -> build/lib/swift/common copying swift/common/container_sync_realms.py -> build/lib/swift/common copying swift/common/db_auditor.py -> build/lib/swift/common copying swift/common/error_limiter.py -> build/lib/swift/common copying swift/common/daemon.py -> build/lib/swift/common copying swift/common/direct_client.py -> build/lib/swift/common copying swift/common/constraints.py -> build/lib/swift/common copying swift/common/digest.py -> build/lib/swift/common copying swift/common/swob.py -> build/lib/swift/common copying swift/common/db_replicator.py -> build/lib/swift/common copying swift/common/header_key_dict.py -> build/lib/swift/common copying swift/common/wsgi.py -> build/lib/swift/common copying swift/common/storage_policy.py -> build/lib/swift/common copying swift/common/exceptions.py -> build/lib/swift/common copying swift/common/linkat.py -> build/lib/swift/common copying swift/common/__init__.py -> build/lib/swift/common copying swift/common/internal_client.py -> build/lib/swift/common copying swift/common/bufferedhttp.py -> build/lib/swift/common copying swift/common/request_helpers.py -> build/lib/swift/common copying swift/common/db.py -> build/lib/swift/common copying swift/common/splice.py -> build/lib/swift/common copying swift/common/http_protocol.py -> build/lib/swift/common copying swift/common/manager.py -> build/lib/swift/common copying swift/common/base_storage_server.py -> build/lib/swift/common copying swift/common/recon.py -> build/lib/swift/common copying swift/common/memcached.py -> build/lib/swift/common copying swift/common/http.py -> build/lib/swift/common creating build/lib/swift/container copying swift/container/reconciler.py -> build/lib/swift/container copying swift/container/updater.py -> build/lib/swift/container copying swift/container/__init__.py -> build/lib/swift/container copying swift/container/auditor.py -> build/lib/swift/container copying swift/container/backend.py -> build/lib/swift/container copying swift/container/sync_store.py -> build/lib/swift/container copying swift/container/server.py -> build/lib/swift/container copying swift/container/replicator.py -> build/lib/swift/container copying swift/container/sharder.py -> build/lib/swift/container copying swift/container/sync.py -> build/lib/swift/container running egg_info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Reusing existing SOURCES.txt /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) creating build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/access_control_policy.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/bucket_logging_status.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/common.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_object_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/copy_part_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/delete_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/error.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_bucket_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_parts_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/list_versions_result.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/location_constraint.rng -> build/lib/swift/common/middleware/s3api/schema copying swift/common/middleware/s3api/schema/versioning_configuration.rng -> build/lib/swift/common/middleware/s3api/schema creating build/lib/swift/locale creating build/lib/swift/locale/de creating build/lib/swift/locale/de/LC_MESSAGES copying swift/locale/de/LC_MESSAGES/swift.po -> build/lib/swift/locale/de/LC_MESSAGES creating build/lib/swift/locale/en_GB creating build/lib/swift/locale/en_GB/LC_MESSAGES copying swift/locale/en_GB/LC_MESSAGES/swift.po -> build/lib/swift/locale/en_GB/LC_MESSAGES creating build/lib/swift/locale/es creating build/lib/swift/locale/es/LC_MESSAGES copying swift/locale/es/LC_MESSAGES/swift.po -> build/lib/swift/locale/es/LC_MESSAGES creating build/lib/swift/locale/fr creating build/lib/swift/locale/fr/LC_MESSAGES copying swift/locale/fr/LC_MESSAGES/swift.po -> build/lib/swift/locale/fr/LC_MESSAGES creating build/lib/swift/locale/it creating build/lib/swift/locale/it/LC_MESSAGES copying swift/locale/it/LC_MESSAGES/swift.po -> build/lib/swift/locale/it/LC_MESSAGES creating build/lib/swift/locale/ja creating build/lib/swift/locale/ja/LC_MESSAGES copying swift/locale/ja/LC_MESSAGES/swift.po -> build/lib/swift/locale/ja/LC_MESSAGES creating build/lib/swift/locale/ko_KR creating build/lib/swift/locale/ko_KR/LC_MESSAGES copying swift/locale/ko_KR/LC_MESSAGES/swift.po -> build/lib/swift/locale/ko_KR/LC_MESSAGES creating build/lib/swift/locale/pt_BR creating build/lib/swift/locale/pt_BR/LC_MESSAGES copying swift/locale/pt_BR/LC_MESSAGES/swift.po -> build/lib/swift/locale/pt_BR/LC_MESSAGES creating build/lib/swift/locale/ru creating build/lib/swift/locale/ru/LC_MESSAGES copying swift/locale/ru/LC_MESSAGES/swift.po -> build/lib/swift/locale/ru/LC_MESSAGES creating build/lib/swift/locale/tr_TR creating build/lib/swift/locale/tr_TR/LC_MESSAGES copying swift/locale/tr_TR/LC_MESSAGES/swift.po -> build/lib/swift/locale/tr_TR/LC_MESSAGES creating build/lib/swift/locale/zh_CN creating build/lib/swift/locale/zh_CN/LC_MESSAGES copying swift/locale/zh_CN/LC_MESSAGES/swift.po -> build/lib/swift/locale/zh_CN/LC_MESSAGES creating build/lib/swift/locale/zh_TW creating build/lib/swift/locale/zh_TW/LC_MESSAGES copying swift/locale/zh_TW/LC_MESSAGES/swift.po -> build/lib/swift/locale/zh_TW/LC_MESSAGES running build_scripts running install_lib creating /build/reproducible-path/swift-2.33.0/debian/tmp creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3 creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_CN creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_CN/LC_MESSAGES copying build/lib/swift/locale/zh_CN/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_CN/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/it creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/it/LC_MESSAGES copying build/lib/swift/locale/it/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/it/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/en_GB creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/en_GB/LC_MESSAGES copying build/lib/swift/locale/en_GB/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/en_GB/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/es creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/es/LC_MESSAGES copying build/lib/swift/locale/es/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/es/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/de creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/de/LC_MESSAGES copying build/lib/swift/locale/de/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/de/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/fr creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/fr/LC_MESSAGES copying build/lib/swift/locale/fr/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/fr/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ja creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ja/LC_MESSAGES copying build/lib/swift/locale/ja/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ja/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/tr_TR creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/tr_TR/LC_MESSAGES copying build/lib/swift/locale/tr_TR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/tr_TR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/pt_BR creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/pt_BR/LC_MESSAGES copying build/lib/swift/locale/pt_BR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/pt_BR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ko_KR creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ko_KR/LC_MESSAGES copying build/lib/swift/locale/ko_KR/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ko_KR/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ru creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ru/LC_MESSAGES copying build/lib/swift/locale/ru/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/ru/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_TW creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_TW/LC_MESSAGES copying build/lib/swift/locale/zh_TW/LC_MESSAGES/swift.po -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/locale/zh_TW/LC_MESSAGES creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy copying build/lib/swift/proxy/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/base.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/account.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/container.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/info.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers copying build/lib/swift/proxy/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy copying build/lib/swift/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/reconciler.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/updater.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/backend.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/sync_store.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/sharder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container copying build/lib/swift/container/sync.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/registry.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/container_sync_realms.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/db_auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/error_limiter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/daemon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/direct_client.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/constraints.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/libc.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/timestamp.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/utils/ipaddrs.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils copying build/lib/swift/common/digest.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/swob.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/db_replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/header_key_dict.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/wsgi.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/storage_policy.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/keystoneauth.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/xprofile.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/memcache.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/read_only.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/list_endpoints.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/listing_formats.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/proxy_logging.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/copy.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/container_quotas.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/symlink.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/acl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/staticweb.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/crossdomain.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/s3response.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/subresource.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/acl_utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/acl_handlers.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/etree.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/bucket_logging_status.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/common.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/complete_multipart_upload.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_versions_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/complete_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/delete.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/delete_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_all_my_buckets_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_multipart_uploads_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/create_bucket_configuration.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/versioning_configuration.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/copy_object_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/copy_part_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/lifecycle_configuration.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/error.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/access_control_policy.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/initiate_multipart_upload_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_parts_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/location_constraint.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema copying build/lib/swift/common/middleware/s3api/schema/list_bucket_result.rng -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/schema creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/logging.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/bucket.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/base.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/acl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/tagging.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/s3_acl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/service.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/multi_delete.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/location.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/obj.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/object_lock.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/multi_upload.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/controllers/versioning.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers copying build/lib/swift/common/middleware/s3api/s3api.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/exception.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/s3token.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/s3api/s3request.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api copying build/lib/swift/common/middleware/slo.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/tempauth.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/container_sync.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/domain_remap.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/backend_ratelimit.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/html_viewer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/exceptions.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/x_profile/profile_model.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile copying build/lib/swift/common/middleware/etag_quoter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/formpost.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/bulk.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/gatekeeper.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/tempurl.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/account_quotas.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/name_check.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/versioned_writes/legacy.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/versioned_writes/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/versioned_writes/object_versioning.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes copying build/lib/swift/common/middleware/dlo.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/catch_errors.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/healthcheck.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/kmip_keymaster.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/kms_keymaster.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/keymaster.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/encrypter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/crypto_utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/crypto/decrypter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto copying build/lib/swift/common/middleware/cname_lookup.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/recon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/middleware/ratelimit.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware copying build/lib/swift/common/exceptions.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/linkat.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/internal_client.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/bufferedhttp.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/request_helpers.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/db.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/builder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/composite_builder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/ring/ring.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring copying build/lib/swift/common/splice.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/http_protocol.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/manager.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/base_storage_server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/recon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/memcached.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common copying build/lib/swift/common/http.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/reaper.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/backend.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/utils.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account copying build/lib/swift/account/replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/mem_diskfile.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers copying build/lib/swift/obj/watchers/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers copying build/lib/swift/obj/watchers/dark_data.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers copying build/lib/swift/obj/ssync_receiver.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/reconstructor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/mem_server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/ssync_sender.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/updater.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/auditor.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/diskfile.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/expirer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/server.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj copying build/lib/swift/obj/replicator.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/shard-info.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/drive_full_checker.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/relinker.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/container_deleter.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/__init__.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/form_signature.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/recon_cron.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/reload.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/info.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/manage_shard_ranges.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/ringcomposer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/ringbuilder.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/dispersion_report.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/recon.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli copying build/lib/swift/cli/ring_builder_analyzer.py -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/base.py to base.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/account.py to account.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/container.py to container.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/info.py to info.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/obj.py to obj.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/reconciler.py to reconciler.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/updater.py to updater.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/auditor.py to auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/backend.py to backend.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync_store.py to sync_store.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/replicator.py to replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sharder.py to sharder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync.py to sync.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/registry.py to registry.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/container_sync_realms.py to container_sync_realms.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_auditor.py to db_auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/error_limiter.py to error_limiter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/daemon.py to daemon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/direct_client.py to direct_client.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/constraints.py to constraints.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/libc.py to libc.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/timestamp.py to timestamp.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/ipaddrs.py to ipaddrs.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/digest.py to digest.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/swob.py to swob.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_replicator.py to db_replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/header_key_dict.py to header_key_dict.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/wsgi.py to wsgi.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/storage_policy.py to storage_policy.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/keystoneauth.py to keystoneauth.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/xprofile.py to xprofile.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/memcache.py to memcache.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/read_only.py to read_only.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/list_endpoints.py to list_endpoints.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/listing_formats.py to listing_formats.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/proxy_logging.py to proxy_logging.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/copy.py to copy.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_quotas.py to container_quotas.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/symlink.py to symlink.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/acl.py to acl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/staticweb.py to staticweb.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crossdomain.py to crossdomain.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3response.py to s3response.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/subresource.py to subresource.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_utils.py to acl_utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_handlers.py to acl_handlers.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/etree.py to etree.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/logging.py to logging.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/bucket.py to bucket.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/base.py to base.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/acl.py to acl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/tagging.py to tagging.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/s3_acl.py to s3_acl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/service.py to service.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_delete.py to multi_delete.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/location.py to location.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/obj.py to obj.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/object_lock.py to object_lock.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_upload.py to multi_upload.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/versioning.py to versioning.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3api.py to s3api.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/exception.py to exception.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3token.py to s3token.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/utils.py to utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3request.py to s3request.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/slo.py to slo.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempauth.py to tempauth.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_sync.py to container_sync.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/domain_remap.py to domain_remap.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/backend_ratelimit.py to backend_ratelimit.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/html_viewer.py to html_viewer.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/exceptions.py to exceptions.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/profile_model.py to profile_model.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/etag_quoter.py to etag_quoter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/formpost.py to formpost.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/bulk.py to bulk.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/gatekeeper.py to gatekeeper.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempurl.py to tempurl.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/account_quotas.py to account_quotas.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/name_check.py to name_check.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/legacy.py to legacy.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/object_versioning.py to object_versioning.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/dlo.py to dlo.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/catch_errors.py to catch_errors.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/healthcheck.py to healthcheck.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kmip_keymaster.py to kmip_keymaster.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kms_keymaster.py to kms_keymaster.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/keymaster.py to keymaster.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/encrypter.py to encrypter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/crypto_utils.py to crypto_utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/decrypter.py to decrypter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/cname_lookup.py to cname_lookup.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/recon.py to recon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/ratelimit.py to ratelimit.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/exceptions.py to exceptions.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/linkat.py to linkat.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/internal_client.py to internal_client.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/bufferedhttp.py to bufferedhttp.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/request_helpers.py to request_helpers.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db.py to db.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/builder.py to builder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/composite_builder.py to composite_builder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/utils.py to utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/ring.py to ring.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/splice.py to splice.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http_protocol.py to http_protocol.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/manager.py to manager.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/base_storage_server.py to base_storage_server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/recon.py to recon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/memcached.py to memcached.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http.py to http.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/reaper.py to reaper.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/auditor.py to auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/backend.py to backend.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/utils.py to utils.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/replicator.py to replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_diskfile.py to mem_diskfile.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/dark_data.py to dark_data.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_receiver.py to ssync_receiver.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/reconstructor.py to reconstructor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_server.py to mem_server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_sender.py to ssync_sender.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/updater.py to updater.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/auditor.py to auditor.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/diskfile.py to diskfile.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/expirer.py to expirer.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/server.py to server.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/replicator.py to replicator.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/shard-info.py to shard-info.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/drive_full_checker.py to drive_full_checker.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/relinker.py to relinker.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/container_deleter.py to container_deleter.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/__init__.py to __init__.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/form_signature.py to form_signature.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon_cron.py to recon_cron.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/reload.py to reload.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/info.py to info.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/manage_shard_ranges.py to manage_shard_ranges.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringcomposer.py to ringcomposer.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringbuilder.py to ringbuilder.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/dispersion_report.py to dispersion_report.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon.py to recon.cpython-312.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ring_builder_analyzer.py to ring_builder_analyzer.cpython-312.pyc running install_egg_info Copying swift.egg-info to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift-2.33.0.egg-info Skipping SOURCES.txt running install_scripts creating /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-sharder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-reconstructor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-ring-composer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-sync -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-config -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-recon -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-reconciler-enqueue -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-orphans -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-dispersion-populate -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-ring-builder-analyzer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-reaper -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-oldies -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-form-signature -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-ring-builder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-relinker -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-drive-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-recon-cron -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-init -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-get-nodes -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-dispersion-report -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-account-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-container-reconciler -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-object-expirer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.12/swift-proxy-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sharder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-reconstructor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-composer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sync to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-config to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-reconciler-enqueue to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-orphans to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-populate to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder-analyzer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-reaper to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-oldies to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-form-signature to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-relinker to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-drive-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon-cron to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-init to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-get-nodes to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-report to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-reconciler to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-expirer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-proxy-server to 755 Installing swift-container-deleter script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-drive-full-checker script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-manage-shard-ranges script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-reload script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin + pwd + python3.11 setup.py install --install-layout=deb --root /build/reproducible-path/swift-2.33.0/debian/tmp /usr/lib/python3/dist-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) /usr/lib/python3/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) /usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running install running build running build_py running egg_info writing swift.egg-info/PKG-INFO writing dependency_links to swift.egg-info/dependency_links.txt writing entry points to swift.egg-info/entry_points.txt writing requirements to swift.egg-info/requires.txt writing top-level names to swift.egg-info/top_level.txt [pbr] Reusing existing SOURCES.txt /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.common.middleware.s3api.schema' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.common.middleware.s3api.schema' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.common.middleware.s3api.schema' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.common.middleware.s3api.schema' to be distributed and are already explicitly excluding 'swift.common.middleware.s3api.schema' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.de.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.de.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.de.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.de.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.de.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.en_GB.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.en_GB.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.en_GB.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.en_GB.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.en_GB.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.es.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.es.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.es.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.es.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.es.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.fr.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.fr.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.fr.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.fr.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.fr.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.it.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.it.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.it.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.it.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.it.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ja.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ja.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ja.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ja.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ja.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ko_KR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ko_KR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ko_KR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ko_KR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ko_KR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.pt_BR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.pt_BR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.pt_BR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.pt_BR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.pt_BR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.ru.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.ru.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.ru.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.ru.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.ru.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.tr_TR.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.tr_TR.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.tr_TR.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.tr_TR.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.tr_TR.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_CN.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_CN.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_CN.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_CN.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_CN.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:204: _Warning: Package 'swift.locale.zh_TW.LC_MESSAGES' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'swift.locale.zh_TW.LC_MESSAGES' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'swift.locale.zh_TW.LC_MESSAGES' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'swift.locale.zh_TW.LC_MESSAGES' to be distributed and are already explicitly excluding 'swift.locale.zh_TW.LC_MESSAGES' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) running build_scripts running install_lib byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/base.py to base.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/account.py to account.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/container.py to container.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/info.py to info.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/controllers/obj.py to obj.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/proxy/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/reconciler.py to reconciler.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/updater.py to updater.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/auditor.py to auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/backend.py to backend.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync_store.py to sync_store.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/replicator.py to replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sharder.py to sharder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/container/sync.py to sync.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/registry.py to registry.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/container_sync_realms.py to container_sync_realms.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_auditor.py to db_auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/error_limiter.py to error_limiter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/daemon.py to daemon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/direct_client.py to direct_client.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/constraints.py to constraints.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/libc.py to libc.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/timestamp.py to timestamp.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/utils/ipaddrs.py to ipaddrs.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/digest.py to digest.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/swob.py to swob.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db_replicator.py to db_replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/header_key_dict.py to header_key_dict.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/wsgi.py to wsgi.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/storage_policy.py to storage_policy.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/keystoneauth.py to keystoneauth.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/xprofile.py to xprofile.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/memcache.py to memcache.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/read_only.py to read_only.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/list_endpoints.py to list_endpoints.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/listing_formats.py to listing_formats.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/proxy_logging.py to proxy_logging.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/copy.py to copy.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_quotas.py to container_quotas.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/symlink.py to symlink.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/acl.py to acl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/staticweb.py to staticweb.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crossdomain.py to crossdomain.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3response.py to s3response.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/subresource.py to subresource.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_utils.py to acl_utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/acl_handlers.py to acl_handlers.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/etree.py to etree.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/logging.py to logging.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/bucket.py to bucket.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/base.py to base.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/acl.py to acl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/tagging.py to tagging.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/s3_acl.py to s3_acl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/service.py to service.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_delete.py to multi_delete.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/location.py to location.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/obj.py to obj.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/object_lock.py to object_lock.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/multi_upload.py to multi_upload.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/controllers/versioning.py to versioning.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3api.py to s3api.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/exception.py to exception.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3token.py to s3token.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/utils.py to utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/s3api/s3request.py to s3request.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/slo.py to slo.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempauth.py to tempauth.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/container_sync.py to container_sync.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/domain_remap.py to domain_remap.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/backend_ratelimit.py to backend_ratelimit.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/html_viewer.py to html_viewer.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/exceptions.py to exceptions.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/x_profile/profile_model.py to profile_model.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/etag_quoter.py to etag_quoter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/formpost.py to formpost.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/bulk.py to bulk.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/gatekeeper.py to gatekeeper.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/tempurl.py to tempurl.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/account_quotas.py to account_quotas.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/name_check.py to name_check.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/legacy.py to legacy.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/versioned_writes/object_versioning.py to object_versioning.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/dlo.py to dlo.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/catch_errors.py to catch_errors.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/healthcheck.py to healthcheck.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kmip_keymaster.py to kmip_keymaster.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/kms_keymaster.py to kms_keymaster.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/keymaster.py to keymaster.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/encrypter.py to encrypter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/crypto_utils.py to crypto_utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/crypto/decrypter.py to decrypter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/cname_lookup.py to cname_lookup.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/recon.py to recon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/middleware/ratelimit.py to ratelimit.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/exceptions.py to exceptions.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/linkat.py to linkat.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/internal_client.py to internal_client.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/bufferedhttp.py to bufferedhttp.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/request_helpers.py to request_helpers.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/db.py to db.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/builder.py to builder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/composite_builder.py to composite_builder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/utils.py to utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/ring/ring.py to ring.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/splice.py to splice.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http_protocol.py to http_protocol.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/manager.py to manager.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/base_storage_server.py to base_storage_server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/recon.py to recon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/memcached.py to memcached.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/common/http.py to http.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/reaper.py to reaper.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/auditor.py to auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/backend.py to backend.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/utils.py to utils.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/account/replicator.py to replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_diskfile.py to mem_diskfile.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/watchers/dark_data.py to dark_data.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_receiver.py to ssync_receiver.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/reconstructor.py to reconstructor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/mem_server.py to mem_server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/ssync_sender.py to ssync_sender.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/updater.py to updater.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/auditor.py to auditor.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/diskfile.py to diskfile.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/expirer.py to expirer.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/server.py to server.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/obj/replicator.py to replicator.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/shard-info.py to shard-info.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/drive_full_checker.py to drive_full_checker.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/relinker.py to relinker.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/container_deleter.py to container_deleter.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/__init__.py to __init__.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/form_signature.py to form_signature.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon_cron.py to recon_cron.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/reload.py to reload.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/info.py to info.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/manage_shard_ranges.py to manage_shard_ranges.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringcomposer.py to ringcomposer.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ringbuilder.py to ringbuilder.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/dispersion_report.py to dispersion_report.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/recon.py to recon.cpython-311.pyc byte-compiling /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift/cli/ring_builder_analyzer.py to ring_builder_analyzer.cpython-311.pyc running install_egg_info removing '/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift-2.33.0.egg-info' (and everything under it) Copying swift.egg-info to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages/swift-2.33.0.egg-info Skipping SOURCES.txt running install_scripts copying build/scripts-3.11/swift-container-sharder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-reconstructor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-ring-composer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-sync -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-config -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-recon -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-reconciler-enqueue -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-orphans -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-dispersion-populate -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-auditor -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-info -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-ring-builder-analyzer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-reaper -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-oldies -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-form-signature -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-ring-builder -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-relinker -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-drive-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-recon-cron -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-init -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-replicator -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-get-nodes -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-dispersion-report -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-account-audit -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-updater -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-container-reconciler -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-object-expirer -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin copying build/scripts-3.11/swift-proxy-server -> /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sharder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-reconstructor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-composer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-sync to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-config to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-reconciler-enqueue to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-server to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-orphans to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-populate to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-auditor to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-info to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder-analyzer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-reaper to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-oldies to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-form-signature to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-ring-builder to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-relinker to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-drive-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-recon-cron to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-init to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-replicator to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-get-nodes to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-dispersion-report to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-account-audit to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-updater to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-container-reconciler to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-object-expirer to 755 changing mode of /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin/swift-proxy-server to 755 Installing swift-container-deleter script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-drive-full-checker script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-manage-shard-ranges script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin Installing swift-reload script to /build/reproducible-path/swift-2.33.0/debian/tmp/usr/bin + pwd + rm -rf /build/reproducible-path/swift-2.33.0/debian/python*/usr/lib/python*/dist-packages/*.pth + pwd + rm -rf /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python*/dist-packages/*.pth rm debian/tmp/usr/bin/swift-init pkgos-dh_auto_test --no-py2 --test-path test/unit 'test\.unit\.(?!common\.test_utils\.TestUtils\.test_get_logger_sysloghandler_plumbing|common\.middleware\.test_cname_lookup\.TestCNAMELookup.*|common\.test_db\.TestDatabaseBroker\.test_get.*|container\.test_sync\.TestContainerSync\.test_init.*|common\.test_utils\.TestPunchHoleReally\.test_punch_a_hole.*|common\.test_utils\.Test_LibcWrapper\.test_argument_plumbing.*|common\.test_utils\.TestUtils\.test_load_pkg_resource_importlib.*|common\.test_db_replicator\.TestHandoffsOnly.*|common\.ring\.test_builder\.TestGetRequiredOverload\.test_perfect_four_zone_four_replica_bad_placement|common\.test_wsgi\.TestWSGI.*|obj\.test_expirer\.TestObjectExpirer\.test_delete_actual_object|common\.test_memcached\.TestMemcached\.test_get_conns_hostname6|common\.test_memcached\.TestMemcached\.test_get_conns_v6|common\.test_memcached\.TestMemcached\.test_get_conns_v6_default|obj\.test_reconstructor\.TestWorkerReconstructor\.test_run_forever_recon_aggregation|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_with_duplicate_but_sufficient_frag_indexes|proxy\.test_mem_server\.TestProxyServer\.test_statsd_prefix|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_read_timeout|proxy\.controllers\.test_base\.TestGetOrHeadHandler\.test_disconnected_logging|obj\.test_expirer\.TestObjectExpirer\.test_failed_delete_continues_on|obj\.test_replicator\.TestObjectReplicator\.test_update)' + PKGOS_TEST_PARALLEL=yes + PKGOS_TEST_SERIAL=no + PYTHONS=disabled + PYTHON3S=disabled + TEST_PARALLEL_OPT=--parallel + TEST_SERIAL_OPT= + echo WARNING: --no-py2 is deprecated, and always on. WARNING: --no-py2 is deprecated, and always on. + shift + py3versions -vr + PYTHON3S=3.12 3.11 + [ yes = no ] + [ no = yes ] + [ 3.12 = disabled ] + echo 3.12 + cut -d. -f1 + PYMAJOR=3 + echo ===> Testing with python (python3) ===> Testing with python (python3) + pwd + [ -d /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages ] + [ -z ] + pwd + echo Implicitly adding PYTHONPATH=/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages Implicitly adding PYTHONPATH=/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages + pwd + export PYTHONPATH=/build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages + [ -e .stestr.conf ] + [ -x /usr/bin/python3-stestr ] + STESTR=stestr + rm -rf .stestr + PYTHON=python3.12 stestr run --parallel --subunit --test-path test/unit test\.unit\.(?!common\.test_utils\.TestUtils\.test_get_logger_sysloghandler_plumbing|common\.middleware\.test_cname_lookup\.TestCNAMELookup.*|common\.test_db\.TestDatabaseBroker\.test_get.*|container\.test_sync\.TestContainerSync\.test_init.*|common\.test_utils\.TestPunchHoleReally\.test_punch_a_hole.*|common\.test_utils\.Test_LibcWrapper\.test_argument_plumbing.*|common\.test_utils\.TestUtils\.test_load_pkg_resource_importlib.*|common\.test_db_replicator\.TestHandoffsOnly.*|common\.ring\.test_builder\.TestGetRequiredOverload\.test_perfect_four_zone_four_replica_bad_placement|common\.test_wsgi\.TestWSGI.*|obj\.test_expirer\.TestObjectExpirer\.test_delete_actual_object|common\.test_memcached\.TestMemcached\.test_get_conns_hostname6|common\.test_memcached\.TestMemcached\.test_get_conns_v6|common\.test_memcached\.TestMemcached\.test_get_conns_v6_default|obj\.test_reconstructor\.TestWorkerReconstructor\.test_run_forever_recon_aggregation|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_with_duplicate_but_sufficient_frag_indexes|proxy\.test_mem_server\.TestProxyServer\.test_statsd_prefix|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_read_timeout|proxy\.controllers\.test_base\.TestGetOrHeadHandler\.test_disconnected_logging|obj\.test_expirer\.TestObjectExpirer\.test_failed_delete_continues_on|obj\.test_replicator\.TestObjectReplicator\.test_update) + subunit2pyunit /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_reload.TestValidateManagerPid.test_good test.unit.cli.test_reload.TestValidateManagerPid.test_good ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_python test.unit.cli.test_reload.TestValidateManagerPid.test_non_python ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_server test.unit.cli.test_reload.TestValidateManagerPid.test_non_server ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_open_error test.unit.cli.test_reload.TestValidateManagerPid.test_open_error ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_worker test.unit.cli.test_reload.TestValidateManagerPid.test_worker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_info test.unit.account.test_backend.TestCommonAccountBroker.test_get_info ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects ... ok Finished. Marked 0 objects for deletion. test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 199, in HEAD return self._handle_acl(app, 'DELETE') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 141, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_response test.unit.account.test_utils.TestAccountUtils.test_account_listing_response ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data ... ok test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_syncs test.unit.cli.test_info.TestCliInfo.test_print_db_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_info test.unit.cli.test_info.TestCliInfo.test_print_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat ... ok test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{}' test DEBUG: Error on keystone reply: 201 b'{}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'[]' test DEBUG: Error on keystone reply: 201 b'[]' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'null' test DEBUG: Error on keystone reply: 201 b'null' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'"foo"' test DEBUG: Error on keystone reply: 201 b'"foo"' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'1' test DEBUG: Error on keystone reply: 201 b'1' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'true' test DEBUG: Error on keystone reply: 201 b'true' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, rejecting request with error: 400 Bad Request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none ... ok test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails ... ok test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status ... ok test.unit.common.test_db.TestChexor.test_chexor test.unit.common.test_db.TestChexor.test_chexor ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.test_db.TestChexor.test_invalid_old_hash test.unit.common.test_db.TestChexor.test_invalid_old_hash ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware ... ok test.unit.common.test_db.TestChexor.test_no_name test.unit.common.test_db.TestChexor.test_no_name ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory ... ok test.unit.common.test_db.TestChexor.test_normal_case test.unit.common.test_db.TestChexor.test_normal_case ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config ... ok test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats ... ok test.unit.common.test_db.TestDatabaseBroker.test_commit_puts test.unit.common.test_db.TestDatabaseBroker.test_commit_puts ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail ... ok test.unit.common.test_db.TestDatabaseBroker.test_delete_db test.unit.common.test_db.TestDatabaseBroker.test_delete_db ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.test_db.TestDatabaseBroker.test_initialize test.unit.common.test_db.TestDatabaseBroker.test_initialize ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.test_db.TestDatabaseBroker.test_lock test.unit.common.test_db.TestDatabaseBroker.test_lock ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats ... ok test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata test.unit.common.test_db.TestDatabaseBroker.test_metadata ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.test_db.TestDatabaseBroker.test_newid test.unit.common.test_db.TestDatabaseBroker.test_newid ... ok test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0391 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0377 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0005 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0268 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0129 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0290 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0200 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0122 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0378 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0128 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0002 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0138 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0226 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0272 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0317 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0474 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0026 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0235 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0166 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0174 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0024 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0266 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0279 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0229 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0163 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0027 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5119 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5348 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0003 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0207 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0026 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0107 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0149 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0637 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.1262 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0229 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0126 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0143 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0181 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0284 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0382 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0291 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0301 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0050 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0055 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0059 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0360 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0030 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0190 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0219 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0208 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0165 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0148 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0241 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0059 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0211 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0189 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0263 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0217 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0191 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0054 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0185 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0024 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0367 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.4457 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0356 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0625 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0440 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0034 "-" 3610486 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging ... ok test.unit.account.test_server.TestAccountController.test_DELETE_empty test.unit.account.test_server.TestAccountController.test_DELETE_empty ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read ... ok test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_empty test.unit.account.test_server.TestAccountController.test_DELETE_not_empty ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_found test.unit.account.test_server.TestAccountController.test_DELETE_not_found ... ok test.unit.account.test_server.TestAccountController.test_DELETE_now_empty test.unit.account.test_server.TestAccountController.test_DELETE_now_empty ... ok test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, rejecting request with error: 401 Unauthorized proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test WARNING: The following digest algorithms are configured but not supported: md5, not-a-valid-digest test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. Quarantined /tmp/tmp49811ami/dev/dbs/par/pre/db to /tmp/tmp49811ami/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmp49811ami/dev/dbs/par/pre/db to /tmp/tmp49811ami/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmp49811ami/dev/dbs/par/pre/db to /tmp/tmp49811ami/dev/quarantined/tests/db due to corrupted database Quarantined /tmp/tmp49811ami/dev/dbs/par/pre/db to /tmp/tmp49811ami/dev/quarantined/tests/db due to disk error while accessing database test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_json test.unit.account.test_server.TestAccountController.test_GET_accept_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts ... ok test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors ... ok test.unit.common.test_db.TestDatabaseBroker.test_put_record test.unit.common.test_db.TestDatabaseBroker.test_put_record ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_xml test.unit.account.test_server.TestAccountController.test_GET_accept_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_json test.unit.account.test_server.TestAccountController.test_GET_empty_account_json ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_invalid_accept test.unit.account.test_server.TestAccountController.test_GET_invalid_accept ... ok test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status ... ok Quarantined /tmp/tmptds5mz6e/dev/dbs/par/pre/db to /tmp/tmptds5mz6e/dev/quarantined/containers/db due to missing row in container_stat table test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_log_requests_false test.unit.account.test_server.TestAccountController.test_GET_log_requests_false ... ok test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format ... ok test.unit.common.test_db.TestDatabaseBroker.test_skip_commits test.unit.common.test_db.TestDatabaseBroker.test_skip_commits ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid ... ok Quarantined /tmp/tmprcg29fhi/dev/dbs/par/pre/db to /tmp/tmprcg29fhi/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring test.unit.cli.test_ringbuilder.TestCommands.test_create_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default test.unit.cli.test_ringbuilder.TestCommands.test_default ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output test.unit.cli.test_ringbuilder.TestCommands.test_default_output ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata ... ok test.unit.common.test_exceptions.TestExceptions.test_client_exception test.unit.common.test_exceptions.TestExceptions.test_client_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_exception test.unit.common.test_exceptions.TestExceptions.test_replication_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout ... ok test.unit.common.test_utils.TestContextPool.test_close test.unit.common.test_utils.TestContextPool.test_close ... ok test.unit.common.test_utils.TestContextPool.test_context_manager test.unit.common.test_utils.TestContextPool.test_context_manager ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove ... ok test ERROR: Error decrypting container listing: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: get_keys(): unknown key id: unknown_key test ERROR: Error decrypting container listing: unknown_key test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility ... ok test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0383 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0038 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0301 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0186 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0146 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0260 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0180 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0137 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0150 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0124 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0070 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0041 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0031 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0032 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0033 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0030 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0029 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0028 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0193 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.2574 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0020 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0003 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0418 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0018 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0142 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0236 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0126 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0120 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0170 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0179 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0279 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0306 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0354 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0271 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0151 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0296 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0029 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0029 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0301 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0024 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0471 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0213 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0212 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0193 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0315 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0298 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0197 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0418 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0204 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0143 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0204 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0141 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0031 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0030 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0213 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0039 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0245 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0228 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0212 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0193 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0232 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0153 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0157 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0149 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0144 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0264 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0148 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0133 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0047 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0040 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0320 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3610486 - test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'}test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json ... ok test.unit.cli.test_ringbuilder.TestCommands.test_find_parts test.unit.cli.test_ringbuilder.TestCommands.test_find_parts ... ok test.unit.account.test_reaper.TestReaper.test_basic_reap_account test.unit.account.test_reaper.TestReaper.test_basic_reap_account ... ok test.unit.account.test_reaper.TestReaper.test_creation test.unit.account.test_reaper.TestReaper.test_creation ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml ... ok test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token ... ok test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output ... ok test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0205 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0112 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0008 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0316 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0034 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0412 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0241 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0076 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0296 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:44 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.1942 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.7126 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0276 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0039 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0127 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0185 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:45 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0276 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0278 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0057 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0415 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0181 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0013 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0003 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0160 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0453 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0244 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0056 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0566 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0309 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0026 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0266 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0235 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0024 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0186 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0020 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0182 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0348 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0252 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0282 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0177 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0121 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0318 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0334 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0250 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.4933 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0414 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0468 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0024 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0027 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0024 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0459 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0020 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0275 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0253 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0198 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0262 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0149 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0060 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0332 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0128 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0253 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0146 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0173 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0161 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.2503 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0089 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0291 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0021 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0139 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:47 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0154 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0150 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0214 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0125 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0027 "-" 3610586 - test.unit.account.test_server.TestAccountController.test_GET_log_requests_true test.unit.account.test_server.TestAccountController.test_GET_log_requests_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format ... ok test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account ... ok test.unit.account.test_reaper.TestReaper.test_reap_account test.unit.account.test_reaper.TestReaper.test_reap_account ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_json test.unit.account.test_server.TestAccountController.test_GET_not_found_json ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_no_container test.unit.account.test_reaper.TestReaper.test_reap_account_no_container ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding ... ok test.unit.account.test_reaper.TestReaper.test_reap_container test.unit.account.test_reaper.TestReaper.test_reap_container ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_plain test.unit.account.test_server.TestAccountController.test_GET_not_found_plain ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_xml test.unit.account.test_server.TestAccountController.test_GET_not_found_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_over_limit test.unit.account.test_server.TestAccountController.test_GET_over_limit ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:full_control OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration ... ok test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users ... ok test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.account.test_replicator.TestReplicatorSync.test_local_ids test.unit.account.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim ... ok test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended ... ok Ring Builder file does not exist: /tmp/tmpk51ffp8t/tmp44liq_0z Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.2:6200R127.0.0.2:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.1:6200R127.0.0.1:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:6200R[3::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Ring Builder file does not exist: /tmp/tmp46vik1gd/tmpd6rf23sy Partition power increase in progress. You need to finish the increase first before adding devices. Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sda9_"" with 3.14159265359 weight got id 4 Device 4 already uses 127.0.0.1:6200/sda9. The on-disk ring builder is unchanged. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. /tmp/tmpg333sprk/tmpj3y47auz, build version 4, id ffcf5d2eeaf54ff2b5bd42e6c5b87aa9 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmpg333sprk/tmpj3y47auz.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output_id_assigned Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 Device d4r1z2-127.0.0.5:6004R127.0.0.5:6004/sda5_"" with 100.0 weight got id 4 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d0r2z1-127.0.0.6:6005R127.0.0.6:6005/sdb6_"" with 100.0 weight got id 0 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_sorted_output Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. /build/reproducible-path/swift-2.33.0/test/unit/cli/test_ipv6_output Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 61 1 51 1 Partition Matches 11 1 12 1 16 1 23 1 26 1 27 1 31 1 33 1 34 1 38 1 44 1 47 1 53 1 57 1 59 1 60 1 1 1 2 1 4 1 5 1 8 1 14 1 17 1 20 1 22 1 32 1 35 1 37 1 39 1 42 1 45 1 50 1 3 1 6 1 7 1 9 1 13 1 15 1 18 1 19 1 24 1 25 1 28 1 30 1 51 1 55 1 56 1 63 1 Partition Matches 1 1 4 1 10 1 12 1 13 1 21 1 23 1 28 1 32 1 33 1 34 1 39 1 42 1 48 1 52 1 53 1 3 1 7 1 11 1 15 1 17 1 19 1 24 1 29 1 30 1 40 1 43 1 45 1 46 1 54 1 59 1 61 1 14 1 18 1 22 1 26 1 27 1 35 1 36 1 41 1 44 1 49 1 51 1 55 1 56 1 58 1 60 1 63 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 38 1 56 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 57 1 24 1 Specified builder file "/tmp/tmpvcydvshs/tmpqhwtg4zx" is not rebalanced yet. Please rebalance first. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled ... ok test INFO: Disabling account_sdb test INFO: Disabling container_sdb test INFO: Disabling object_sdb test INFO: Enabling account_sdb test INFO: Enabling container_sdb test INFO: Enabling object_sdb fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_01 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_02 fake-swift ERROR: Got status 404 listing container /AUTH_test/c fake-swift ERROR: Too many bytes for /v1/AUTH_test/mancon/manifest; truncating in /v1/AUTH_test/c/seg_03 with 5 bytes left fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, max LO GET time of 86400s exceeded fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had a21075a36eeddd084e17611a238c7101, but object MD5 was actually 3902ee7e149eb8313a34757e89e21af6 fake-swift ERROR: Bad response length for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/c/seg_01 test DEBUG: User tenant_id has system reader authorizing test DEBUG: tenant mismatch: AUTH_tenant_id2 != tenant_id test DEBUG: allow user with role(s) swiftoperator,admin as account admin test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - First:%201%0ASecond:%202 0.0001 - - 1717346924.481376886 1717346924.481480598 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 500 - - - - - - - - 0.0000 - - 1717346924.483746767 1717346924.483761072 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.485100269 1717346924.485170126 - test INFO: GET /v1/a/c 205 205 test INFO: GET /v1/a/c 209 205 test INFO: GET /v1/a/c 210 205 test INFO: 1.2.3.4 1.2.3.4 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.497335434 1717346924.497409582 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717346924.499208212 1717346924.499244452 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717346924.500945330 1717346924.500962973 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.502219200 1717346924.502279520 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.503942490 1717346924.504048824 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 15 - - - 0.0007 - - 1717346924.503908396 1717346924.504643679 - test INFO: - - 26/Apr/1970/17/46/41 GET / HTTP/1.0 200 - - - - 8 - - - 1.0000 - - 10000000.000000000 10000001.000000000 - test INFO: GET /v1/a/c 503 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c True 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c parsing%20ok 503 test INFO: GET /v1/a/c 500 500 test INFO: GET /v1/a/c 500 568 test INFO: GET /v1/a/c 499 503 test INFO: GET /v1/a/c 499 503 test INFO: GET / 500 200 test INFO: 2001:db8:85a3:8d3:1319:8a2e:370:7348 2001:db8:85a3:8d3:1319:8a2e:370:7348 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.547662020 1717346924.547730446 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.551503420 1717346924.551572561 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - b05bf940-0464-4c... - 8 - - - 0.0001 - - 1717346924.552609682 1717346924.552670240 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.553764820 1717346924.553823233 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - b05bf940... - 8 - - - 0.0001 - - 1717346924.554900169 1717346924.554966927 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717346924.555993080 1717346924.556044579 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717346924.557092428 1717346924.557153225 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.558371305 1717346924.558427811 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - ... - 8 - - - 0.0001 - - 1717346924.559797525 1717346924.559860229 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717346924.561634779 1717346924.561701298 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717346924.562823057 1717346924.562876940 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one%2Cand%20two 1717346924.564571142 1717346924.564646959 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.566472054 1717346924.566541672 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.567745209 1717346924.567797899 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one 1717346924.569404364 1717346924.569458723 - test INFO: template which can be edited in config: HTTP/1.0 / GET {SMD5}c65475e457fea0951fbb9ec9596b2177 - 1.0000 26/Apr/1970/17/46/40 10000001.000000000 0.5 example.com test INFO: HTTP/1.0 /bucket/path/to/key GET AUTH_test bucket path/to/key test INFO: - - 02/Jun/2024/16/48/44 GET /%3Fx%3D3 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.576114655 1717346924.576222181 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim ... ok test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_replicator.TestReplicatorSync.test_sync test.unit.account.test_replicator.TestReplicatorSync.test_sync ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0206 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0019 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0161 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0259 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0170 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0168 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0271 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0216 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0158 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0144 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0063 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0051 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0034 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0036 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0032 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0030 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0032 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0437 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0138 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0336 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0019 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0442 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0024 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0270 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0244 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0383 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0171 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0160 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0146 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0154 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0205 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0174 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0260 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0215 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0258 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0037 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0284 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0023 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0281 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0172 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0299 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0169 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0146 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0225 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0206 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0153 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0228 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0337 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0328 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0390 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0039 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0053 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0339 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0032 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0371 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0179 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0225 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0256 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0140 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0149 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0191 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0214 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0247 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0162 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0163 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0227 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0046 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0130 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0408 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0024 "-" 3610586 - test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_json test.unit.account.test_server.TestAccountController.test_GET_with_containers_json ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:write OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator ... ok test INFO: - - - [02/Jun/2024:16:48:48 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0199 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0278 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0027 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0284 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0301 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0331 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0083 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0440 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0313 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0196 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0188 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0030 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0240 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0049 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0309 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0368 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0058 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0192 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0180 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0300 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0470 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0017 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0203 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0286 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0180 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0216 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0006 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0005 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0008 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0321 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0373 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0047 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0244 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0187 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0156 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0036 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0147 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0217 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0283 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0198 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0213 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0047 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0236 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0298 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0190 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0186 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0270 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0033 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0227 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0093 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0508 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0298 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0019 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0257 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0196 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0381 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0146 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0123 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0405 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0036 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0034 "-" 3610486 - test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 370, in reap_container headers, objects = direct_get_container( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 171, in fake_direct_get_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.10.10.1:6202/sda3 test-reaper ERROR: ERROR: invalid storage policy index: 2 test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing ... ok test.unit.cli.test_info.TestPrintObj.test_missing_etag test.unit.cli.test_info.TestPrintObj.test_missing_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body ... ok test.unit.account.test_reaper.TestReaper.test_reap_delay test.unit.account.test_reaper.TestReaper.test_reap_delay ... ok test.unit.account.test_reaper.TestReaper.test_reap_device test.unit.account.test_reaper.TestReaper.test_reap_device ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled ... ok test.unit.account.test_server.TestAccountController.test_HEAD_empty_account test.unit.account.test_server.TestAccountController.test_HEAD_empty_account ... ok test.unit.account.test_backend.TestAccountBroker.test_chexor test.unit.account.test_backend.TestAccountBroker.test_chexor ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create ... ok test.unit.cli.test_info.TestPrintObj.test_print_obj_valid test.unit.cli.test_info.TestPrintObj.test_print_obj_valid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_creation test.unit.account.test_backend.TestAccountBroker.test_creation ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping ... ok test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.account.test_backend.TestAccountBroker.test_delete_container test.unit.account.test_backend.TestAccountBroker.test_delete_container ... ok test.unit.account.test_backend.TestAccountBroker.test_delete_db_status test.unit.account.test_backend.TestAccountBroker.test_delete_db_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered ... ok test.unit.cli.test_recon.TestScout.test_scout_http_error test.unit.cli.test_recon.TestScout.test_scout_http_error ... ok test.unit.account.test_reaper.TestReaper.test_reap_object test.unit.account.test_reaper.TestReaper.test_reap_object ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_fail test.unit.account.test_reaper.TestReaper.test_reap_object_fail ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format ... ok test.unit.cli.test_recon.TestScout.test_scout_ok test.unit.cli.test_recon.TestScout.test_scout_ok ... ok test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline ... ok test.unit.account.test_backend.TestAccountBroker.test_empty test.unit.account.test_backend.TestAccountBroker.test_empty ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_object_timeout ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_HEAD_not_found test.unit.account.test_server.TestAccountController.test_HEAD_not_found ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok ... ok test.unit.account.test_backend.TestAccountBroker.test_exception test.unit.account.test_backend.TestAccountBroker.test_exception ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.account.test_server.TestAccountController.test_HEAD_with_containers test.unit.account.test_server.TestAccountController.test_HEAD_with_containers ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info test.unit.account.test_backend.TestAccountBroker.test_get_info ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_OPTIONS test.unit.account.test_server.TestAccountController.test_OPTIONS ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme ... ok test.unit.account.test_reaper.TestReaper.test_reset_stats test.unit.account.test_reaper.TestReaper.test_reset_stats ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout ... ok test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path ... ok test.unit.account.test_reaper.TestReaper.test_run_forever test.unit.account.test_reaper.TestReaper.test_run_forever ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller ... ok test.unit.account.test_backend.TestAccountBroker.test_initialize_fail test.unit.account.test_backend.TestAccountBroker.test_initialize_fail ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path ... ok test.unit.account.test_reaper.TestReaper.test_run_once test.unit.account.test_reaper.TestReaper.test_run_once ... ok test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error ... ok test.unit.account.test_server.TestAccountController.test_POST_insufficient_space test.unit.account.test_server.TestAccountController.test_POST_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_default ... ok test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted ... ok test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented ... ok test.unit.cli.test_recon.TestHelpers.test_size_suffix test.unit.cli.test_recon.TestHelpers.test_size_suffix ... ok test.unit.cli.test_recon.TestScout.test_scout_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_socket_timeout ... ok test.unit.account.test_server.TestAccountController.test_POST_invalid_partition test.unit.account.test_server.TestAccountController.test_POST_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples ... ok test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD ... ok test.unit.account.test_server.TestAccountController.test_PUT test.unit.account.test_server.TestAccountController.test_PUT ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict ... ok test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found ... ok test.unit.cli.test_recon.TestScout.test_scout_url_error test.unit.cli.test_recon.TestScout.test_scout_url_error ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_json test.unit.common.middleware.test_xprofile.TestStats2.test_to_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items test.unit.account.test_backend.TestAccountBroker.test_merge_items ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace ... ok test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions ... ok test.unit.common.ring.test_utils.TestUtils.test_dispersion_report test.unit.common.ring.test_utils.TestUtils.test_dispersion_report ... ok test.unit.common.ring.test_utils.TestUtils.test_is_local_device test.unit.common.ring.test_utils.TestUtils.test_is_local_device ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_close_error test.unit.obj.test_diskfile.TestECDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_newid test.unit.account.test_backend.TestAccountBroker.test_newid ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account ... ok test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container ... ok test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names ... ok test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found ... ok test.unit.account.test_backend.TestAccountBroker.test_put_container test.unit.account.test_backend.TestAccountBroker.test_put_container ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_backend.TestAccountBroker.test_reclaim test.unit.account.test_backend.TestAccountBroker.test_reclaim ... ok test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit test.unit.obj.test_diskfile.TestECDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_args test.unit.common.ring.test_utils.TestUtils.test_parse_args ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_value test.unit.common.ring.test_utils.TestUtils.test_parse_search_value ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put ... ok test.unit.common.ring.test_utils.TestUtils.test_replication_defaults test.unit.common.ring.test_utils.TestUtils.test_replication_defaults ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error ... ok test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args test.unit.common.ring.test_utils.TestUtils.test_validate_args ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts ... ok test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon test.unit.common.middleware.test_tempauth.TestAuth.test_anon ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete ... ok test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac test.unit.common.test_digest.TestDigestUtils.test_get_hmac ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type ... ok test.unit.common.test_splice.TestSplice.test_available test.unit.common.test_splice.TestSplice.test_available ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_splice.TestSplice.test_errno test.unit.common.test_splice.TestSplice.test_errno ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.cli.test_recon.TestRecon.test_async_check test.unit.cli.test_recon.TestRecon.test_async_check ... ok test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag ... ok test.unit.common.test_splice.TestSplice.test_fileno test.unit.common.test_splice.TestSplice.test_fileno ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers ... ok test.unit.cli.test_recon.TestRecon.test_drive_audit_check test.unit.cli.test_recon.TestRecon.test_drive_audit_check ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config ... ok test.unit.common.test_splice.TestSplice.test_flags test.unit.common.test_splice.TestSplice.test_flags ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_gen_stats test.unit.cli.test_recon.TestRecon.test_gen_stats ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_get_error_ringnames test.unit.cli.test_recon.TestRecon.test_get_error_ringnames ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers ... ok test.unit.common.test_splice.TestSplice.test_flags_list test.unit.common.test_splice.TestSplice.test_flags_list ... ok test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe ... ok test INFO: Begin account audit "once" mode test ERROR: Audit Failed for /tmp/tmpval_kswi/sda/accounts/0/0/0/test.db: The total container_count for the account a (3) does not match the sum of container_count across policies (2) test INFO: Account audit "once" mode completed: 0.00s test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details ... ok test.unit.cli.test_recon.TestRecon.test_get_hosts test.unit.cli.test_recon.TestRecon.test_get_hosts ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers ... ok test.unit.cli.test_recon.TestRecon.test_get_ring_names test.unit.cli.test_recon.TestRecon.test_get_ring_names ... ok test.unit.cli.test_recon.TestRecon.test_get_ringmd5 test.unit.cli.test_recon.TestRecon.test_get_ringmd5 ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers ... ok test.unit.common.test_splice.TestSplice.test_unavailable test.unit.common.test_splice.TestSplice.test_unavailable ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_splice.TestSplice.test_unavailable_in_libc test.unit.common.test_splice.TestSplice.test_unavailable_in_libc ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr ... ok test.unit.cli.test_recon.TestRecon.test_ptime test.unit.cli.test_recon.TestRecon.test_ptime ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest ... ok test.unit.cli.test_recon.TestRecon.test_quarantine_check test.unit.cli.test_recon.TestRecon.test_quarantine_check ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path ... ok test.unit.cli.test_recon.TestRecon.test_umount_check test.unit.cli.test_recon.TestRecon.test_umount_check ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_reload.TestMain.test_check_failed test.unit.cli.test_reload.TestMain.test_check_failed ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length ... ok test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures ... ok test.unit.cli.test_reload.TestMain.test_good test.unit.cli.test_reload.TestMain.test_good ... ok test.unit.cli.test_reload.TestMain.test_needs_pid test.unit.cli.test_reload.TestMain.test_needs_pid ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success ... ok test.unit.common.test_utils.TestDistributeEvenly.test_just_one test.unit.common.test_utils.TestDistributeEvenly.test_just_one ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section ... ok test.unit.cli.test_reload.TestMain.test_timeout test.unit.cli.test_reload.TestMain.test_timeout ... ok test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_delete test.unit.obj.test_diskfile.TestECDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE ... ok test.unit.common.test_utils.TestDistributeEvenly.test_uneven test.unit.common.test_utils.TestDistributeEvenly.test_uneven ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges ... ok test.unit.common.test_utils.TestPairs.test_pairs test.unit.common.test_utils.TestPairs.test_pairs ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges ... ok test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied ... ok test.unit.common.test_utils.TestStringAlong.test_happy test.unit.common.test_utils.TestStringAlong.test_happy ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list ... ok test.unit.common.test_utils.TestStringAlong.test_unhappy test.unit.common.test_utils.TestStringAlong.test_unhappy ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_utils.TestUTC.test_tzname test.unit.common.test_utils.TestUTC.test_tzname ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET ... ok test.unit.obj.test_server.TestZeroCopy.test_GET test.unit.obj.test_server.TestZeroCopy.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated ... ok test.unit.obj.test_server.TestZeroCopy.test_GET_big test.unit.obj.test_server.TestZeroCopy.test_GET_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list ... ok test.unit.obj.test_server.TestZeroCopy.test_quarantine test.unit.obj.test_server.TestZeroCopy.test_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json ... ok test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_options_call test.unit.common.middleware.test_tempauth.TestAuth.test_options_call ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root ... ok test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap ... ok test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed ... ok test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation ... ok test.unit.common.middleware.test_acl.TestACL.test_clean_acl test.unit.common.middleware.test_acl.TestACL.test_clean_acl ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded ... ok test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_acl.TestACL.test_parse_acl test.unit.common.middleware.test_acl.TestACL.test_parse_acl ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init ... ok test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences ... ok test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors ... ok test.unit.common.middleware.test_bulk.TestConfig.test_defaults test.unit.common.middleware.test_bulk.TestConfig.test_defaults ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value ... ok test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'X-Amz-Acl': 'public-read', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Grant AllUsers READ permission on the bucket /bucket test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker ... ok test.unit.cli.test_recon.TestReconCommands.test_disk_usage test.unit.cli.test_recon.TestReconCommands.test_disk_usage ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_sync test.unit.obj.test_ssync.TestSsyncReplication.test_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers ... ok test.unit.cli.test_recon.TestReconCommands.test_load_check test.unit.cli.test_recon.TestReconCommands.test_load_check ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries ... ok test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions ... ok test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured ... ok test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check ... ok test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_encrypter.py", line 913, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' Something useful Also useful Boring, but not secret test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations ... ok test.unit.cli.test_recon.TestReconCommands.test_replication_check test.unit.cli.test_recon.TestReconCommands.test_replication_check ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy ... ok test.unit.cli.test_recon.TestReconCommands.test_server_type_check test.unit.cli.test_recon.TestReconCommands.test_server_type_check ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range test.unit.common.middleware.test_helpers.TestFakeSwift.test_range ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker ... ok test.unit.cli.test_recon.TestReconCommands.test_sharding_check test.unit.cli.test_recon.TestReconCommands.test_sharding_check ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestReconCommands.test_time_check test.unit.cli.test_recon.TestReconCommands.test_time_check ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check test.unit.cli.test_recon.TestReconCommands.test_version_check ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check_differs test.unit.cli.test_recon.TestReconCommands.test_version_check_differs ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 ... ok test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_conf_file test.unit.cli.test_relinker.TestRelinker.test_conf_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings ... ok test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access ... ok test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups ... ok test.unit.cli.test_relinker.TestRelinker.test_rehashing test.unit.cli.test_relinker.TestRelinker.test_rehashing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments test.unit.obj.test_diskfile.TestECDiskFile.test_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_file test.unit.cli.test_relinker.TestRelinker.test_relink_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_db_auditor.TestAuditor.test_database_auditor test.unit.common.test_db_auditor.TestAuditor.test_database_auditor ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.ring.test_ring.TestRing.test_creation test.unit.common.ring.test_ring.TestRing.test_creation ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes test.unit.common.ring.test_ring.TestRing.test_get_more_nodes ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica ... ok test.unit.common.ring.test_ring.TestRing.test_get_nodes test.unit.common.ring.test_ring.TestRing.test_get_nodes ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_ring.TestRing.test_get_part test.unit.common.ring.test_ring.TestRing.test_get_part ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_ring.TestRing.test_get_part_nodes test.unit.common.ring.test_ring.TestRing.test_get_part_nodes ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders ... ok test.unit.common.ring.test_ring.TestRing.test_has_changed test.unit.common.ring.test_ring.TestRing.test_has_changed ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges ... ok test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_ring.TestRing.test_reload test.unit.common.ring.test_ring.TestRing.test_reload ... ok test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names ... ok test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache ... ok test.unit.common.ring.test_ring.TestRing.test_reload_without_replication test.unit.common.ring.test_ring.TestRing.test_reload_without_replication ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id ... ok test.unit.common.ring.test_ring.TestRing.test_replica_count test.unit.common.ring.test_ring.TestRing.test_replica_count ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon ... ok test.unit.cli.test_relinker.TestRelinker.test_state_file test.unit.cli.test_relinker.TestRelinker.test_state_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders ... ok test.unit.cli.test_relinker.TestRelinker.test_workers_children test.unit.cli.test_relinker.TestRelinker.test_workers_children ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_daemon.TestWorkerDaemon.test_stubs test.unit.common.test_daemon.TestWorkerDaemon.test_stubs ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ ... ok test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked ... ok test.unit.cli.test_relinker.TestRelinker.test_workers_parent test.unit.cli.test_relinker.TestRelinker.test_workers_parent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy ... ok test.unit.common.test_swob.TestTransferEncoding.test_is_chunked test.unit.common.test_swob.TestTransferEncoding.test_is_chunked ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestTpoolSize.test_default_config test.unit.obj.test_server.TestTpoolSize.test_default_config ... ok test.unit.obj.test_server.TestTpoolSize.test_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting ... ok test.unit.obj.test_updater.TestObjectUpdater.test_conf_params test.unit.obj.test_updater.TestObjectUpdater.test_conf_params ... ok test.unit.obj.test_updater.TestObjectUpdater.test_creation test.unit.obj.test_updater.TestObjectUpdater.test_creation ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case ... ok test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_update test.unit.common.middleware.s3api.test_utils.TestConfig.test_update ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: public None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: authenticated None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users ... ok test ERROR: Bad key for 'container': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: get_keys(): from callback: boom: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_crypto_utils.py", line 95, in callback raise Exception('boom') Exception: boom test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Missing key for 'container': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'container' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Did not get a keys dict: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ TypeError: list indices must be integers or slices, not str test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names ... ok test.unit.common.test_utils.ResellerConfReader.test_defaults test.unit.common.test_utils.ResellerConfReader.test_defaults ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers ... ok test.unit.common.test_utils.ResellerConfReader.test_same_as_default test.unit.common.test_utils.ResellerConfReader.test_same_as_default ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf ... ok test.unit.common.test_utils.ResellerConfReader.test_stray_comma test.unit.common.test_utils.ResellerConfReader.test_stray_comma ... ok test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time ... ok test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00091 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpqnfk7grc/sda/accounts/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00906 seconds (110.40135/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00060 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00100 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00038 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sda/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'c6b49115-2558-4600-be73-947fc8ee8437-sda', '1717346926.23209', '1717346926.22699', '0', '') test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpzjuqtjj_/sda/accounts/0/cca/7e85f39cfbd92b74b0b21a19f4188cca/7e85f39cfbd92b74b0b21a19f4188cca.db REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'f76c7758-cc77-4665-b485-fb29ae493966-sda', '1717346926.28794', '1717346926.27271', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('f76c7758-cc77-4665-b485-fb29ae493966-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'f76c7758-cc77-4665-b485-fb29ae493966-sda', '1717346926.28794', '1717346926.27271', '0', '') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('f76c7758-cc77-4665-b485-fb29ae493966-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12740 seconds (7.84904/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (1, '252ba9d1ccce1ced761b3636ba932bb3', '882febae-7772-47fe-8940-4537c8faf404-sda', '1717346926.46598', '1717346926.45938', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, rsync_then_merge, ('882febae-7772-47fe-8940-4537c8faf404-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (11, '7567441c6bac8c8c526b2d9237a98c8c', 'cf6b7663-3d4f-4ad7-b013-1b071f36d968-sda', '1717346926.58772', '1717346926.5782', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_items, ([{'ROWID': 1, 'name': '/a/c_0', 'put_timestamp': '1717346926.62796', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c_1', 'put_timestamp': '1717346926.62884', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c_2', 'put_timestamp': '1717346926.62986', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c_3', 'put_timestamp': '1717346926.63058', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c_4', 'put_timestamp': '1717346926.63126', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c_5', 'put_timestamp': '1717346926.63247', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c_6', 'put_timestamp': '1717346926.63314', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c_7', 'put_timestamp': '1717346926.63348', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c_8', 'put_timestamp': '1717346926.63392', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c_9', 'put_timestamp': '1717346926.63458', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c_missing', 'put_timestamp': '1717346926.63799', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}], 'cf6b7663-3d4f-4ad7-b013-1b071f36d968-sda') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_syncs, ([],) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files ... ok test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0148 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0153 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0213 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0142 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0218 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0161 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0192 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0042 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0166 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0232 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0060 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0375 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0038 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0346 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0176 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0041 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0293 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0212 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0340 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0076 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0335 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:49 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0022 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0181 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0568 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0093 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0400 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0403 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0005 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0006 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0006 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0009 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0345 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0116 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0320 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0016 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0280 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0090 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0310 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0183 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0046 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0232 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0253 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0280 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0347 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0114 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0226 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0313 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0282 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0104 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0516 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0041 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0252 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0359 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0216 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0021 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0384 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0273 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0524 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0120 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0180 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0215 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0596 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0054 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0035 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3610586 - test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestAccountController.test_PUT_auto_create test.unit.account.test_server.TestAccountController.test_PUT_auto_create ... ok test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_not_found test.unit.account.test_server.TestAccountController.test_PUT_not_found ... ok test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestAccountController.test__call__raise_timeout test.unit.account.test_server.TestAccountController.test__call__raise_timeout ... ok test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD ... ok test.unit.account.test_server.TestAccountController.test_correct_allowed_method test.unit.account.test_server.TestAccountController.test_correct_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies ... ok test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.0.0.0:1000/sda: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 490, in reap_object direct_delete_object( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 160, in fake_direct_delete_object raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.0.0.0:1000/sda test-reaper ERROR: Timeout Exception with 10.0.0.1:1001/sdb test-reaper ERROR: Timeout Exception with 10.0.0.2:1002/sdc test-reaper ERROR: Timeout Exception with 10.0.0.3:1003/sdd test-reaper DEBUG: Begin devices pass: thisdeviceisbad test-reaper ERROR: Exception in top-level account reaper loop: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 135, in run_once for device in os.listdir(self.devices): ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'thisdeviceisbad' test-reaper INFO: Devices pass completed: 0.00s proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpn80fh1rs/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpn80fh1rs/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpn80fh1rs/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpn80fh1rs/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.1436 "-" 3610512 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0228 "-" 3610512 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0338 "-" 3610512 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0147 "-" 3610512 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx08a097d7ab994b02bc19c-00665ca26e" "proxy-server 3610512" 0.0024 "-" 3610512 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:45423/sdb1/1/a/c" "tx08a097d7ab994b02bc19c-00665ca26e" "container-server 3610512" 0.0028 "-" 3610512 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:42719/sda1/1/a/c" "tx08a097d7ab994b02bc19c-00665ca26e" "container-server 3610512" 0.0023 "-" 3610512 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx08a097d7ab994b02bc19c-00665ca26e" "proxy-server 3610512" 0.0537 "-" 3610512 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx08a097d7ab994b02bc19c-00665ca26e" "proxy-server 3610512" 0.0746 "-" 3610512 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/46 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx08a097d7ab994b02bc19c-00665ca26e - 0.0906 - - 1717346926.198283195 1717346926.288912058 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx39c25033b734447fa9871-00665ca26e" "proxy-server 3610512" 0.0030 "-" 3610512 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:45423/sdb1/1/a1/c1" "tx39c25033b734447fa9871-00665ca26e" "container-server 3610512" 0.0028 "-" 3610512 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:42719/sda1/1/a1/c1" "tx39c25033b734447fa9871-00665ca26e" "container-server 3610512" 0.0041 "-" 3610512 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx39c25033b734447fa9871-00665ca26e" "proxy-server 3610512" 0.0438 "-" 3610512 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx39c25033b734447fa9871-00665ca26e" "proxy-server 3610512" 0.0826 "-" 3610512 0 test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container ... ok test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 HEAD /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 POST /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 DELETE /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 OPTIONS /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/42 GET - HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET / HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /bad HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /baddy/mc_badderson HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1 HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1/ HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.677516699 1717346924.677592993 - test INFO: - - 02/Jun/2024/16/48/44 GET /foo HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.678241253 1717346924.678313732 - test INFO: - - 02/Jun/2024/16/48/44 GET /foo/bar HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.678969860 1717346924.679049730 - test INFO: - - 02/Jun/2024/16/48/44 GET /v1 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.679658413 1717346924.679735899 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 17 - - - 0.0001 SOS - 1717346924.685263872 1717346924.685394764 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - - - - - 0.0001 - - 1717346924.687434912 1717346924.687494755 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 3 - - - 0.0007 - - 1717346924.689375401 1717346924.690073490 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 12 - - - 0.0001 - - 1717346924.691970825 1717346924.692026138 - test INFO: - - 02/Jun/2024/16/48/44 PUT /v1/a/c/o HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717346924.702424288 1717346924.702466249 1 test INFO: - - 02/Jun/2024/16/48/44 GET /v1/a/c HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.703776360 1717346924.703872442 1 test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.705307961 1717346924.705383778 - test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/48/44 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.707194567 1717346924.707267523 - test INFO: - - 02/Jun/2024/16/48/44 GET /v1/something HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717346924.708545208 1717346924.708626986 - test INFO: - - 02/Jun/2024/16/48/44 GET / HTTP/1.0 499 - - - - 4 - - - 0.0001 - - 1717346924.710856676 1717346924.710940599 - test INFO: - - 02/Jun/2024/16/48/44 POST /v1/a/c HTTP/1.0 200 - - - 11 8 - - Host:%20localhost:80 0.0000 - - 1717346924.712281704 1717346924.712314129 - test INFO: - - 02/Jun/2024/16/48/44 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717346924.713839293 1717346924.713880777 0 test INFO: - - 02/Jun/2024/16/48/44 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717346924.715392590 1717346924.715430975 -1 test INFO: - - 02/Jun/2024/16/48/44 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717346924.716934681 1717346924.716965199 - test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00049 seconds (0.00000/s) test INFO: Removed 0 dbs test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout ... ok test.unit.common.test_internal_client.TestInternalClient.test_container_exists test.unit.common.test_internal_client.TestInternalClient.test_container_exists ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_account test.unit.common.test_internal_client.TestInternalClient.test_create_account ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container test.unit.common.test_internal_client.TestInternalClient.test_create_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_account test.unit.common.test_internal_client.TestInternalClient.test_delete_account ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_container test.unit.common.test_internal_client.TestInternalClient.test_delete_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_object test.unit.common.test_internal_client.TestInternalClient.test_delete_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info test.unit.common.test_internal_client.TestInternalClient.test_get_account_info ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_object test.unit.common.test_internal_client.TestInternalClient.test_get_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_init test.unit.common.test_internal_client.TestInternalClient.test_init ... ok test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_containers test.unit.common.test_internal_client.TestInternalClient.test_iter_containers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items test.unit.common.test_internal_client.TestInternalClient.test_iter_items ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_objects test.unit.common.test_internal_client.TestInternalClient.test_iter_objects ... ok test.unit.common.test_internal_client.TestInternalClient.test_load_from_config test.unit.common.test_internal_client.TestInternalClient.test_load_from_config ... ok test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path test.unit.common.test_internal_client.TestInternalClient.test_make_path ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object test.unit.common.test_internal_client.TestInternalClient.test_upload_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing ... ok test.unit.common.test_manager.TestManagerModule.test_command_wrapper test.unit.common.test_manager.TestManagerModule.test_command_wrapper ... ok test.unit.common.test_manager.TestManagerModule.test_exc test.unit.common.test_manager.TestManagerModule.test_exc ... ok test.unit.common.test_manager.TestManagerModule.test_format_server_name test.unit.common.test_manager.TestManagerModule.test_format_server_name ... ok test.unit.common.test_manager.TestManagerModule.test_safe_kill test.unit.common.test_manager.TestManagerModule.test_safe_kill ... ok test.unit.common.test_manager.TestManagerModule.test_servers test.unit.common.test_manager.TestManagerModule.test_servers ... ok test.unit.common.test_manager.TestManagerModule.test_setup_env test.unit.common.test_manager.TestManagerModule.test_setup_env ... ok test.unit.common.test_manager.TestManagerModule.test_verify_server test.unit.common.test_manager.TestManagerModule.test_verify_server ... ok test.unit.common.test_manager.TestManagerModule.test_watch_server_pids test.unit.common.test_manager.TestManagerModule.test_watch_server_pids ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order ... ok test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0299 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0165 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0472 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0108 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0297 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0033 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0863 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0637 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0038 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0548 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0418 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:50 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0065 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.0500 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0006 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0672 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0087 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0020 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.1557 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.2741 "-" 3610486 - test DEBUG: - - - [02/Jun/2024:16:48:51 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0006 "-" 3610486 - test DEBUG: - - - [02/Jun/2024:16:48:51 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0007 "-" 3610486 - test DEBUG: - - - [02/Jun/2024:16:48:51 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0091 "-" 3610486 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:48:51 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0236 "-" 3610486 - test DEBUG: - - - [02/Jun/2024:16:48:51 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0006 "-" 3610486 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:48:52 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5805 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0031 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0059 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5414 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0001 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0003 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0470 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0067 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0508 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0591 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0851 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0154 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0348 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0118 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0031 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0075 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0416 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0222 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0283 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0220 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0036 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0033 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0029 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0030 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0045 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0245 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0021 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0063 "-" 3610486 2 test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0397 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0037 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0511 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0070 "-" 3610486 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test_digest_utils WARNING: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 test WARNING: boom proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to get commit status of PUT to /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1750, in await_response self.resp = self.conn.getexpect() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 854, in getexpect expect_status = self._status.get_expect_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 757, in get_expect_status raise expect_status Exception: kaboom! (txn: tx8b042b6067db4360b923f-00665ca26e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to get commit status of PUT to /v1/a/c/o: Timeout (Nones) (txn: tx9a87d12fd1ed45eea4129-00665ca26e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success ... ok test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Cipher must be AES_CTR_256 test DEBUG: No decryption is necessary because of override test DEBUG: No decryption is necessary because of override test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_decrypter.py", line 902, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: Error decrypting header X-Object-Sysmeta-Crypto-Etag: Missing crypto meta in value fgBd78GL6kACmfmuAxTz9B7eJCFTfjxotcr7trw/5Qo= test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting object: Bad crypto meta: Missing 'cipher' test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Key must be length 32 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting object: Missing 'body_key' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test ERROR: Error retrieving recon data: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 228, in raise_Exception raise Exception Exception test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test ERROR: Error parsing recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 225, in raise_ValueError raise ValueError ValueError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version ... ok test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception ... ok test.unit.common.test_db.TestDatabaseConnectionError.test_str test.unit.common.test_db.TestDatabaseConnectionError.test_str ... ok test.unit.common.test_splice.TestTee.test_available test.unit.common.test_splice.TestTee.test_available ... ok test.unit.common.test_splice.TestTee.test_errno test.unit.common.test_splice.TestTee.test_errno ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'Content-Md5': 'Je0by0I7C3IA9IX8X/ccjg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '25ed1bcb423b0b7200f485fc5ff71c8e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET ... ok test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record ... ok test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test ERROR: ERROR: Unable to access foo/bar: permission_denied test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpdas_hlrv/devices/sda1 (pid: 3610687) test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpdas_hlrv/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346940.00000 test INFO: Object update sweep completed on /tmp/tmpdas_hlrv/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp9i422g_h/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmp9i422g_h/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346940.00000 test INFO: Object update sweep completed on /tmp/tmp9i422g_h/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpbiiwvxl9/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpbiiwvxl9/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_new test DEBUG: Update sent for /.shards_a/c_shard_new/o /tmp/tmpbiiwvxl9/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 test INFO: Object update sweep completed on /tmp/tmpbiiwvxl9/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpvc5r5rc8/devices/sda1 (pid: 3610687) test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpvc5r5rc8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 test INFO: Object update sweep completed on /tmp/tmpvc5r5rc8/devices/sda1 in 0.15s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.16s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpvc5r5rc8/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpvc5r5rc8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_1 test DEBUG: Update sent for /.shards_a/c_shard_1/o /tmp/tmpvc5r5rc8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 test INFO: Object update sweep completed on /tmp/tmpvc5r5rc8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpo4gvxikj/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_older/o /tmp/tmpo4gvxikj/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_new test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_new/o /tmp/tmpo4gvxikj/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_newer test INFO: Object update sweep completed on /tmp/tmpo4gvxikj/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpo4gvxikj/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /.shards_a/c_shard_newer/o /tmp/tmpo4gvxikj/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 test INFO: Object update sweep completed on /tmp/tmpo4gvxikj/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpidb2dql3/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges ... ok =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 5 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== Reloaded swift-proxy-server test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/48/52 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0018 - - 1717346932.882871389 1717346932.884641171 0 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/48/52 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0099 - - 1717346932.886048794 1717346932.895975590 1 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpxury6x7o/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpxury6x7o/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpg9s1mrzn/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test INFO: Since Wed Dec 31 12:00:00 1969: container audits: 0 passed audit, 1 failed audit test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass ... ok test INFO: Begin account audit "once" mode test INFO: Account audit "once" mode completed: 0.00s [2024-06-02 16:49:01] Checking auditor stats =============================================================================== test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY jojo', (txn: tx3b7303534de94284a12ff-00665ca281) proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXYjojo a b c d e', (txn: tx219a6908b1ea44aba2c17-00665ca281) proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY a b c d e', (txn: tx4a94b683588c4d77841f8-00665ca281) proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'GET /someurl HTTP/1.0\r\n', (txn: test-bad-req-trans-id) proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:49:05] "GET /someurl HTTP/1.1" 200 160 0.000098 proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:49:05] "GET /otherurl HTTP/1.1" 200 179 0.000059 proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:49:05] "GET /someurl HTTP/1.0" 200 186 0.000064 proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:49:05] "GET /someurl HTTP/1.0" 200 179 0.000093 proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils ... ok test.unit.common.test_utils.TestCooperativeIterator.test_close test.unit.common.test_utils.TestCooperativeIterator.test_close ... ok test.unit.common.test_utils.TestCooperativeIterator.test_init test.unit.common.test_utils.TestCooperativeIterator.test_init ... ok test.unit.common.test_utils.TestCooperativeIterator.test_iter test.unit.common.test_utils.TestCooperativeIterator.test_iter ... ok test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps ... ok test.unit.common.test_utils.TestCooperativeIterator.test_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_sleeps ... ok test.unit.common.test_utils.TestWatchdog.test_start_stop test.unit.common.test_utils.TestWatchdog.test_start_stop ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_expire test.unit.common.test_utils.TestWatchdog.test_timeout_expire ... ok test.unit.container.test_server.TestContainerController.test_DELETE test.unit.container.test_server.TestContainerController.test_DELETE ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race ... ok test.unit.container.test_server.TestContainerController.test_DELETE_account_update test.unit.container.test_server.TestContainerController.test_DELETE_account_update ... ok test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_DELETE_not_found test.unit.container.test_server.TestContainerController.test_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_object test.unit.container.test_server.TestContainerController.test_DELETE_object ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page ... ok test.unit.common.test_db.TestExampleBroker.test_commit_pending test.unit.common.test_db.TestExampleBroker.test_commit_pending ... ok test.unit.common.test_db.TestExampleBroker.test_delete_db test.unit.common.test_db.TestExampleBroker.test_delete_db ... ok test.unit.common.test_db.TestExampleBroker.test_get_info test.unit.common.test_db.TestExampleBroker.test_get_info ... ok test.unit.common.test_db.TestExampleBroker.test_get_max_row test.unit.common.test_db.TestExampleBroker.test_get_max_row ... ok test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata ... ok test.unit.common.test_db.TestExampleBroker.test_get_syncs test.unit.common.test_db.TestExampleBroker.test_get_syncs ... ok test.unit.common.test_db.TestExampleBroker.test_maybe_get test.unit.common.test_db.TestExampleBroker.test_maybe_get ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_put_timestamp test.unit.common.test_db.TestExampleBroker.test_put_timestamp ... ok test.unit.common.test_db.TestExampleBroker.test_status_changed_at test.unit.common.test_db.TestExampleBroker.test_status_changed_at ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_closes test.unit.common.test_utils.TestReiterate.test_reiterate_closes ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple ... ok test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 ... ok test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set ... ok test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/48/58 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0024 - - 1717346938.080258608 1717346938.082704306 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/48/58 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0066 - - 1717346938.088868380 1717346938.095473289 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - deferring rejection downstream test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, deferring rejection based on error: 400 Bad Request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, deferring rejection based on error: 401 Unauthorized test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile123 Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile3610505 Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile456 30 function calls in 0.005 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.003 0.001 0.005 0.001 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile123 Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile3610505 Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile456 30 function calls in 0.005 seconds Ordered by: call count ncalls tottime percall cumtime percall filename:lineno(function) 6 0.003 0.001 0.005 0.001 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :0(setprofile) 0 0.000 0.000 profile:0(profiler) Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile123 Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile3610505 Sun Jun 2 04:48:58 2024 /tmp/tmpkzhcrfj4/unittest.profile456 30 function calls in 0.005 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.003 0.001 0.005 0.001 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render ... ok test.unit.common.ring.test_ring.TestRingData.test_attrs test.unit.common.ring.test_ring.TestRingData.test_attrs ... ok test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization ... ok test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data ... ok test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization ... ok test.unit.common.ring.test_ring.TestRingData.test_load_closes_file test.unit.common.ring.test_ring.TestRingData.test_load_closes_file ... ok test.unit.common.ring.test_ring.TestRingData.test_permissions test.unit.common.ring.test_ring.TestRingData.test_permissions ... ok test.unit.common.ring.test_ring.TestRingData.test_replica_count test.unit.common.ring.test_ring.TestRingData.test_replica_count ... ok test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default test.unit.common.test_memcached.TestLoadMemcache.test_conf_default ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst test.unit.common.test_utils.TestEventletRateLimiter.test_burst ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_init test.unit.common.test_utils.TestEventletRateLimiter.test_init ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit ... ok test.unit.common.test_utils.TestShardRange.test_constants test.unit.common.test_utils.TestShardRange.test_constants ... ok test.unit.common.test_utils.TestShardRange.test_copy test.unit.common.test_utils.TestShardRange.test_copy ... ok test.unit.common.test_utils.TestShardRange.test_deleted_setter test.unit.common.test_utils.TestShardRange.test_deleted_setter ... ok test.unit.common.test_utils.TestShardRange.test_epoch_setter test.unit.common.test_utils.TestShardRange.test_epoch_setter ... ok test.unit.common.test_utils.TestShardRange.test_find_ancestors test.unit.common.test_utils.TestShardRange.test_find_ancestors ... ok test.unit.common.test_utils.TestShardRange.test_find_root test.unit.common.test_utils.TestShardRange.test_find_root ... ok test.unit.common.test_utils.TestShardRange.test_increment_meta test.unit.common.test_utils.TestShardRange.test_increment_meta ... ok test.unit.common.test_utils.TestShardRange.test_is_child_of test.unit.common.test_utils.TestShardRange.test_is_child_of ... ok test.unit.common.test_utils.TestShardRange.test_make_path test.unit.common.test_utils.TestShardRange.test_make_path ... ok test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_min_max_bounds test.unit.common.test_utils.TestShardRange.test_min_max_bounds ... ok test.unit.common.test_utils.TestShardRange.test_name test.unit.common.test_utils.TestShardRange.test_name ... ok test.unit.common.test_utils.TestShardRange.test_name_validation test.unit.common.test_utils.TestShardRange.test_name_validation ... ok test.unit.common.test_utils.TestShardRange.test_repr test.unit.common.test_utils.TestShardRange.test_repr ... ok test.unit.common.test_utils.TestShardRange.test_resolve_state test.unit.common.test_utils.TestShardRange.test_resolve_state ... ok test.unit.common.test_utils.TestShardRange.test_row_count test.unit.common.test_utils.TestShardRange.test_row_count ... ok test.unit.common.test_utils.TestShardRange.test_set_deleted test.unit.common.test_utils.TestShardRange.test_set_deleted ... ok test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation ... ok test.unit.common.test_utils.TestShardRange.test_sort_key test.unit.common.test_utils.TestShardRange.test_sort_key ... ok test.unit.common.test_utils.TestShardRange.test_sort_key_order test.unit.common.test_utils.TestShardRange.test_sort_key_order ... ok test.unit.common.test_utils.TestShardRange.test_state_setter test.unit.common.test_utils.TestShardRange.test_state_setter ... ok test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_timestamp_setter test.unit.common.test_utils.TestShardRange.test_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_to_from_dict test.unit.common.test_utils.TestShardRange.test_to_from_dict ... ok test.unit.common.test_utils.TestShardRange.test_update_meta test.unit.common.test_utils.TestShardRange.test_update_meta ... ok test.unit.common.test_utils.TestShardRange.test_update_state test.unit.common.test_utils.TestShardRange.test_update_state ... ok test.unit.common.test_utils.TestShardRange.test_update_tombstones test.unit.common.test_utils.TestShardRange.test_update_tombstones ... ok test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding ... ok proxy-server WARNING: ERROR Client read timeout (Nones) (txn: tx4854d4c048184fae827ae-00665ca26e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: tx198da8fa59614609b9a7d-00665ca272) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3173, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 6776, in read raise Exception('exception message') Exception: exception message (txn: txe26821e4866441948c9ba-00665ca272) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: txeb18bafdadc8498081536-00665ca272) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.13:1013/sdn (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server ERROR: Not enough object servers ack'ed (got 0) (txn: tx7a29a31b2aa24479a6a10-00665ca272) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.13:1013/sdn (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txa7dad703cbe4444097115-00665ca272) proxy-server ERROR: Not enough object servers ack'ed (got 3) (txn: txa7dad703cbe4444097115-00665ca272) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx219ba43fea5a4ad2bf331-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx219ba43fea5a4ad2bf331-00665ca272) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx219ba43fea5a4ad2bf331-00665ca272) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding ... ok test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces ... ok test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT ... ok test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects ... ok test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_2 test INFO: Object update sweep completed on /tmp/tmpidb2dql3/devices/sda1 in 1.82s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 1.82s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpidb2dql3/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_2/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to None test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_3 test INFO: Object update sweep completed on /tmp/tmpidb2dql3/devices/sda1 in 0.03s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.03s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpidb2dql3/devices/sda1 (pid: 3610687) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_3/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 to None test INFO: Object update sweep completed on /tmp/tmpidb2dql3/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpidb2dql3/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmpidb2dql3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346939.00000 test INFO: Object update sweep completed on /tmp/tmpidb2dql3/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpusv9oprw/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /.shards_a/shard_c/o /tmp/tmpusv9oprw/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346943.00000 test INFO: Object update sweep completed on /tmp/tmpusv9oprw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpusv9oprw/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmpusv9oprw/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346944.00000 test INFO: Object update sweep completed on /tmp/tmpusv9oprw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpusv9oprw/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmpusv9oprw/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346945.00000 test INFO: Object update sweep completed on /tmp/tmpusv9oprw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpusv9oprw/devices/sda1 (pid: 3610687) test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine ... ok test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep ... ok test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0666 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.3871 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:51 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.3727 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:52 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 1.2311 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0112 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0073 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0106 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0907 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0641 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0459 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0017 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0008 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.0545 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0739 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0112 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0105 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0782 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:53 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0346 "-" 3610586 - test DEBUG: - - - [02/Jun/2024:16:48:54 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0006 "-" 3610586 - test DEBUG: - - - [02/Jun/2024:16:48:54 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0006 "-" 3610586 - test DEBUG: - - - [02/Jun/2024:16:48:54 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0008 "-" 3610586 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:48:54 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0128 "-" 3610586 - test DEBUG: - - - [02/Jun/2024:16:48:54 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0006 "-" 3610586 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0598 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0306 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0573 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0057 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0431 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0436 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0431 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0099 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0490 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0032 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0076 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0036 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0812 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0095 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0627 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0452 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0047 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0153 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0038 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0039 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0656 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0041 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0037 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0036 "-" 3610586 1 test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0289 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0066 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0583 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610586 - test.unit.account.test_server.TestAccountController.test_empty_policy_stats test.unit.account.test_server.TestAccountController.test_empty_policy_stats ... ok test.unit.account.test_server.TestAccountController.test_init test.unit.account.test_server.TestAccountController.test_init ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestAccountController.test_list_allowed_methods test.unit.account.test_server.TestAccountController.test_list_allowed_methods ... ok test.unit.account.test_server.TestAccountController.test_log_line_format test.unit.account.test_server.TestAccountController.test_log_line_format ... ok test.unit.account.test_server.TestAccountController.test_markers_with_reserved test.unit.account.test_server.TestAccountController.test_markers_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestAccountController.test_not_allowed_method test.unit.account.test_server.TestAccountController.test_not_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_null_markers test.unit.account.test_server.TestAccountController.test_null_markers ... ok test.unit.account.test_server.TestAccountController.test_params_format test.unit.account.test_server.TestAccountController.test_params_format ... ok test.unit.account.test_server.TestAccountController.test_params_utf8 test.unit.account.test_server.TestAccountController.test_params_utf8 ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_non_default test.unit.account.test_server.TestAccountController.test_policy_stats_non_default ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_prefix_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestAccountController.test_serv_reserv test.unit.account.test_server.TestAccountController.test_serv_reserv ... ok test.unit.account.test_server.TestAccountController.test_through_call test.unit.account.test_server.TestAccountController.test_through_call ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path test.unit.account.test_server.TestAccountController.test_through_call_invalid_path ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor test.unit.common.test_splice.TestTee.test_fileno test.unit.common.test_splice.TestTee.test_fileno ... ok test.unit.common.test_splice.TestTee.test_flags_list test.unit.common.test_splice.TestTee.test_flags_list ... ok test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe ... ok test.unit.common.test_splice.TestTee.test_unavailable test.unit.common.test_splice.TestTee.test_unavailable ... ok test.unit.common.test_splice.TestTee.test_unavailable_in_libc test.unit.common.test_splice.TestTee.test_unavailable_in_libc ... /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same ... ok test.unit.common.test_utils.TestSpliterator.test_big_input_string test.unit.common.test_utils.TestSpliterator.test_big_input_string ... ok test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries ... ok test.unit.common.test_utils.TestSpliterator.test_closing test.unit.common.test_utils.TestSpliterator.test_closing ... ok test.unit.common.test_utils.TestSpliterator.test_no_empty_strings test.unit.common.test_utils.TestSpliterator.test_no_empty_strings ... ok test.unit.common.test_utils.TestSpliterator.test_overlap test.unit.common.test_utils.TestSpliterator.test_overlap ... ok test.unit.common.test_utils.TestSpliterator.test_running_out test.unit.common.test_utils.TestSpliterator.test_running_out ... ok test.unit.common.test_utils.TestSpliterator.test_string test.unit.common.test_utils.TestSpliterator.test_string ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str test.unit.common.test_wsgi.TestPipelineWrapper.test_str ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter ... ok test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock ... ok test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges ... ok proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/46 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx39c25033b734447fa9871-00665ca26e - 0.0986 - - 1717346926.291774273 1717346926.390393257 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdec06687ff5f462aa9cce-00665ca26e" "proxy-server 3610512" 0.0297 "-" 3610512 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:42719/sda1/2/a/c1" "txdec06687ff5f462aa9cce-00665ca26e" "container-server 3610512" 0.0019 "-" 3610512 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:45423/sdb1/2/a/c1" "txdec06687ff5f462aa9cce-00665ca26e" "container-server 3610512" 0.0135 "-" 3610512 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txdec06687ff5f462aa9cce-00665ca26e" "proxy-server 3610512" 0.0478 "-" 3610512 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txdec06687ff5f462aa9cce-00665ca26e" "proxy-server 3610512" 0.0872 "-" 3610512 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/46 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txdec06687ff5f462aa9cce-00665ca26e - 0.1294 - - 1717346926.392838240 1717346926.522217751 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx8aa70a16af0e4d2ba1e16-00665ca26e" "proxy-server 3610512" 0.0216 "-" 3610512 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:42719/sda1/1/a/c2" "tx8aa70a16af0e4d2ba1e16-00665ca26e" "container-server 3610512" 0.0154 "-" 3610512 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:45423/sdb1/1/a/c2" "tx8aa70a16af0e4d2ba1e16-00665ca26e" "container-server 3610512" 0.0020 "-" 3610512 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx8aa70a16af0e4d2ba1e16-00665ca26e" "proxy-server 3610512" 0.0953 "-" 3610512 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:46 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx8aa70a16af0e4d2ba1e16-00665ca26e" "proxy-server 3610512" 0.0491 "-" 3610512 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/46 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx8aa70a16af0e4d2ba1e16-00665ca26e - 0.1351 - - 1717346926.524624825 1717346926.659686089 2 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET ... ok test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument ... ok test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe ... ok test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments ... ok test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0003 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0015 "-" 3610486 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0223 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0020 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0284 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0193 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0031 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0033 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0166 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0021 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0202 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0257 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0423 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0100 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0530 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/c_zero" 201 - "-" "-" "-" 0.0067 "-" 3610486 0 test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0498 "-" 3610486 1 test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0185 "-" 3610486 2 test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0456 "-" 3610486 3 test INFO: - - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0383 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0430 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0024 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0367 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0266 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0327 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0188 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0051 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0042 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0485 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0057 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0073 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0006 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0001 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0002 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0001 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:54 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0285 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0078 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0190 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0083 "-" 3610486 3 test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0257 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0138 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0392 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0037 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0288 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0044 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0312 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0106 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0787 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0517 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0313 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0428 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0130 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0577 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0023 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0528 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0512 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0594 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0486 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0102 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0360 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0138 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0352 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0411 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0620 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0117 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0085 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0036 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3610486 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run test.unit.obj.test_replicator.TestObjectReplicator.test_run ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once test.unit.obj.test_replicator.TestObjectReplicator.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync ... ok test DEBUG: Update sent for /a/c/o /tmp/tmpusv9oprw/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.00000 test INFO: Object update sweep completed on /tmp/tmpusv9oprw/devices/sda1 in 1.17s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 1.17s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpusv9oprw/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmpusv9oprw/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346947.00000 test INFO: Object update sweep completed on /tmp/tmpusv9oprw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpusv9oprw/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmpusv9oprw/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346948.00000 test INFO: Object update sweep completed on /tmp/tmpusv9oprw/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7qc9d5tv/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmp7qc9d5tv/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346945.00000 test INFO: Object update sweep completed on /tmp/tmp7qc9d5tv/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7qc9d5tv/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmp7qc9d5tv/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346947.00000 test INFO: Object update sweep completed on /tmp/tmp7qc9d5tv/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp80p1ng1y/devices/sda1 (pid: 3610687) test ERROR: ERROR Pickle problem, quarantining /tmp/tmp80p1ng1y/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346944.00000: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/updater.py", line 431, in _load_update return pickle.load(open(update_path, 'rb')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _pickle.UnpicklingError: invalid load key, '\xff'. test INFO: Object update sweep completed on /tmp/tmp80p1ng1y/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpnps4n3ug/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c1/o04 /tmp/tmpnps4n3ug/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717346944.73404 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmpnps4n3ug/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717346944.73685 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx89fa050221024d60bed33-00665ca274) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx89fa050221024d60bed33-00665ca274) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx89fa050221024d60bed33-00665ca274) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Expect: 100-continue on /a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3038, in _make_putter return MIMEPutter.connect( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2001, in connect raise FooterNotSupported() swift.common.exceptions.FooterNotSupported (txn: txf211fce119f34cd9a2602-00665ca274) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txaf587aaf8aae4c2ca6ea0-00665ca275) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txaf587aaf8aae4c2ca6ea0-00665ca275) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: txaf587aaf8aae4c2ca6ea0-00665ca275) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx423d3bad66bd4aa19c459-00665ca275) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx423d3bad66bd4aa19c459-00665ca275) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: tx423d3bad66bd4aa19c459-00665ca275) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx423d3bad66bd4aa19c459-00665ca275) proxy-server ERROR: Object PUT returning 503 for [201, 503, 201, 201, 201, 201, 201, 201, 201, 201, 201, 503, 503, 503] (txn: tx423d3bad66bd4aa19c459-00665ca275) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id test.unit.common.ring.test_builder.TestRingBuilder.test_id ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_init test.unit.common.ring.test_builder.TestRingBuilder.test_init ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_load test.unit.common.ring.test_builder.TestRingBuilder.test_load ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload test.unit.common.ring.test_builder.TestRingBuilder.test_overload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save test.unit.common.ring.test_builder.TestRingBuilder.test_save ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save_load test.unit.common.ring.test_builder.TestRingBuilder.test_save_load ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate test.unit.common.ring.test_builder.TestRingBuilder.test_validate ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_eq test.unit.common.test_utils.TestNamespaceBoundList.test_eq ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_parse test.unit.common.test_utils.TestNamespaceBoundList.test_parse ... ok test.unit.container.test_updater.TestContainerUpdater.test_conf_params test.unit.container.test_updater.TestContainerUpdater.test_conf_params ... ok test.unit.container.test_updater.TestContainerUpdater.test_creation test.unit.container.test_updater.TestContainerUpdater.test_creation ... ok test.unit.container.test_updater.TestContainerUpdater.test_error_in_process test.unit.container.test_updater.TestContainerUpdater.test_error_in_process ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container ... ok :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :0(getcwd) 1 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :0(getcwd) 1 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :0(getcwd) 1 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :0(getcwd) 1 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile456 Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile3610505 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile456 Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile3610505 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile123 Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile456 Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile3610505 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile3610505 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :0(getcwd) 3 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile3610505 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :0(getcwd) 3 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:48:59 2024 /tmp/tmp19jvq2el/unittest.profile3610505 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :0(getcwd) 3 0.000 0.000 0.000 0.000 :1() 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) test-replicator INFO: [worker 1/2 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 1/2 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/2 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 2/2 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/3 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 2/3 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 3/3 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 3/3 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3610505] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3610505] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=8804] Running object replicator in script mode. test-replicator DEBUG: [worker 1/3 pid=8804] debug message test-replicator INFO: [worker 1/3 pid=8804] info message test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim ... ok test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b fake INFO: - - 02/Jun/2024/16/48/46 GET /v1/a/c/o%3Ftemp_url_sig%3Ddeadbeefdeadbeef...%26temp_url_expires%3D1717347926 HTTP/1.0 404 - - - - - - - - 0.0005 - - 1717346926.314526558 1717346926.314979792 - test ERROR: Problem finding shard upper in '/tmp/tmpfk28tpgo/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpfk28tpgo/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmp05sv0q48/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmp05sv0q48/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call test.unit.obj.test_ssync_sender.TestSender.test_call ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect test.unit.obj.test_ssync_sender.TestSender.test_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect test.unit.obj.test_ssync_sender.TestSender.test_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all test.unit.obj.test_ssync_sender.TestSender.test_readline_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete test.unit.obj.test_ssync_sender.TestSender.test_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post test.unit.obj.test_ssync_sender.TestSender.test_send_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put test.unit.obj.test_ssync_sender.TestSender.test_send_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_post test.unit.obj.test_ssync_sender.TestSender.test_updates_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put test.unit.obj.test_ssync_sender.TestSender.test_updates_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test INFO: - - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0119 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0005 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0026 "-" 3610586 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0688 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0177 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0503 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0620 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0103 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0362 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0020 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0403 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0246 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0181 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:55 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0092 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0274 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0122 "-" 3610586 0 test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0221 "-" 3610586 1 test INFO: - - - [02/Jun/2024:16:48:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0278 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0396 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0059 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0362 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0325 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0418 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0032 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0050 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0218 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0251 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0038 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0027 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0004 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0002 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0395 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0116 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0113 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0034 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0572 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0033 "-" 3610586 1 test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0738 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0731 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0570 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0073 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0411 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0139 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0398 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0312 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0543 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0374 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0036 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0380 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0022 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0366 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0479 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0278 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0360 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0033 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0523 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0022 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0339 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0918 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0361 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0242 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0028 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0027 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0008 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0001 "-" 3610586 - test.unit.account.test_server.TestAccountController.test_utf8_metadata test.unit.account.test_server.TestAccountController.test_utf8_metadata ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata ... ok test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE ... ok test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1nymku96/sda/fake/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00470 seconds (212.96916/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00052 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00037 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00030 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-ic DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) fake-swift INFO: - - 02/Jun/2024/16/48/46 GET /cont/obj HTTP/1.0 200 - some_agent - - 19 - - - 0.0004 - - 1717346926.459256411 1717346926.459638596 - fake-swift INFO: - - 02/Jun/2024/16/48/46 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717346926.462023258 1717346926.462232590 - fake-swift INFO: - - 02/Jun/2024/16/48/46 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717346926.463234186 1717346926.463466167 - fake-swift INFO: - - 02/Jun/2024/16/48/46 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717346926.464182854 1717346926.464405298 - test DEBUG: Before request test DEBUG: Inside of request (txn: foo) test DEBUG: After request fake-swift INFO: - - 02/Jun/2024/16/48/47 DELETE /container HTTP/1.0 409 - some_agent - - 19 - - - 0.0002 - - 1717346927.001310110 1717346927.001540661 - WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify memory limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? test ERROR: Problem finding shard upper in '/tmp/tmpdqy9rjol/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpdqy9rjol/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmp32c0rva_/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmp32c0rva_/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging ... ok test.unit.container.test_sync.TestContainerSync.test_FileLikeIter test.unit.container.test_sync.TestContainerSync.test_FileLikeIter ... ok test.unit.container.test_sync.TestContainerSync.test_container_first_loop test.unit.container.test_sync.TestContainerSync.test_container_first_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_report test.unit.container.test_sync.TestContainerSync.test_container_report ... ok test.unit.container.test_sync.TestContainerSync.test_container_second_loop test.unit.container.test_sync.TestContainerSync.test_container_second_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_stop_at test.unit.container.test_sync.TestContainerSync.test_container_stop_at ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms ... ok test ERROR: Failed to unlink '/tmp/tmpfafvt1hg/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpfafvt1hg/containers/part/suffix/hash/container_1717346973.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpfafvt1hg/containers/part/suffix/hash/container_1717346973.00000.db': ['/tmp/tmpfafvt1hg/containers/part/suffix/hash/container.db', '/tmp/tmpfafvt1hg/containers/part/suffix/hash/container_1717346974.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpfafvt1hg/containers/part/suffix/hash/container_1717346974.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0004 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0001 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0681 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0472 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0375 "-" 3610486 - test INFO: - - - [02/Jun/2024:16:48:56 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0255 "-" 3610486 - test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test ERROR: ERROR Could not get container info /tmp/tmpg9s1mrzn/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test DEBUG: Audit passed for test DEBUG: Audit passed for test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 3.72s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpw6mypu_x/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test.unit.common.test_db_auditor.TestAuditor.test_run_forever test.unit.common.test_db_auditor.TestAuditor.test_run_forever ... ok test.unit.common.test_db_auditor.TestAuditor.test_run_once test.unit.common.test_db_auditor.TestAuditor.test_run_once ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: nojunk None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Fri, 01 Apr 2014 12:00:00 GMT', 'X-Amz-Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000'} test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000'} test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000'} test DEBUG: checking permission: bucket None POST {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000'} test DEBUG: bucket None POST test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Base DELETE returning 503 for (204, 204, 404) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3e6e64045b754b9d8ab62-00665ca28c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3e6e64045b754b9d8ab62-00665ca28c) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3e6e64045b754b9d8ab62-00665ca28c) test ERROR: Container HEAD returning 503 for [] (txn: tx3e6e64045b754b9d8ab62-00665ca28c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info test.unit.common.middleware.test_container_sync.TestContainerSync.test_info ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info test.unit.proxy.controllers.test_base.TestFuncs.test_get_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same ... ok test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers ... ok test.unit.common.test_swob.TestResponse.test_507 test.unit.common.test_swob.TestResponse.test_507 ... ok test.unit.common.test_swob.TestResponse.test_app_iter test.unit.common.test_swob.TestResponse.test_app_iter ... ok test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk ... ok test.unit.common.test_swob.TestResponse.test_call_preserves_closeability test.unit.common.test_swob.TestResponse.test_call_preserves_closeability ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary ... ok test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called ... ok test.unit.common.test_swob.TestResponse.test_charset test.unit.common.test_swob.TestResponse.test_charset ... ok test.unit.common.test_swob.TestResponse.test_charset_content_type test.unit.common.test_swob.TestResponse.test_charset_content_type ... ok test.unit.common.test_swob.TestResponse.test_content_type test.unit.common.test_swob.TestResponse.test_content_type ... ok test.unit.common.test_swob.TestResponse.test_empty_body test.unit.common.test_swob.TestResponse.test_empty_body ... ok test.unit.common.test_swob.TestResponse.test_etag test.unit.common.test_swob.TestResponse.test_etag ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end ... ok test.unit.common.test_swob.TestResponse.test_host_url_default test.unit.common.test_swob.TestResponse.test_host_url_default ... ok test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_https test.unit.common.test_swob.TestResponse.test_host_url_host_https ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_override test.unit.common.test_swob.TestResponse.test_host_url_host_override ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_port_override test.unit.common.test_swob.TestResponse.test_host_url_host_port_override ... ok test.unit.common.test_swob.TestResponse.test_host_url_https test.unit.common.test_swob.TestResponse.test_host_url_https ... ok test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched ... ok test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite test.unit.common.test_swob.TestResponse.test_location_no_rewrite ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite test.unit.common.test_swob.TestResponse.test_location_rewrite ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable ... ok test.unit.common.test_swob.TestResponse.test_multi_range_body test.unit.common.test_swob.TestResponse.test_multi_range_body ... ok test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest ... ok test.unit.common.test_swob.TestResponse.test_multi_response_iter test.unit.common.test_swob.TestResponse.test_multi_response_iter ... ok test.unit.common.test_swob.TestResponse.test_properties test.unit.common.test_swob.TestResponse.test_properties ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error /tmp/tmpxtdy1bqk/tmpgpuwfoky, build version 4, id 187239810b8e4e559f65e3ccd0cc75bc 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmpxtdy1bqk/tmpgpuwfoky.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 Reassigned 192 (300.00%) partitions. Balance is now 5.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance test.unit.cli.test_ringbuilder.TestCommands.test_rebalance ... ok test.unit.common.test_swob.TestResponse.test_range_body test.unit.common.test_swob.TestResponse.test_range_body ... ok test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range ... ok test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter ... ok test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter ... ok test.unit.common.test_swob.TestResponse.test_unicode_body test.unit.common.test_swob.TestResponse.test_unicode_body ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_increment test.unit.common.test_error_limiter.TestErrorLimiter.test_increment ... ok test.unit.common.test_swob.TestStatusMap.test_status_map test.unit.common.test_swob.TestStatusMap.test_status_map ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_pending test.unit.common.test_utils.TestGreenAsyncPile.test_pending ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestGetAuth.test_invalid_version test.unit.common.test_internal_client.TestGetAuth.test_invalid_version ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_internal_client.TestGetAuth.test_ok test.unit.common.test_internal_client.TestGetAuth.test_ok ... ok test.unit.common.test_manager.TestManager.test_create test.unit.common.test_manager.TestManager.test_create ... ok test.unit.common.test_manager.TestManager.test_default_strict test.unit.common.test_manager.TestManager.test_default_strict ... ok test.unit.common.test_manager.TestManager.test_force_reload test.unit.common.test_manager.TestManager.test_force_reload ... ok test.unit.common.test_manager.TestManager.test_get_command test.unit.common.test_manager.TestManager.test_get_command ... ok test.unit.common.test_manager.TestManager.test_iter test.unit.common.test_manager.TestManager.test_iter ... ok test.unit.common.test_manager.TestManager.test_list_commands test.unit.common.test_manager.TestManager.test_list_commands ... ok test.unit.common.test_manager.TestManager.test_no_daemon test.unit.common.test_manager.TestManager.test_no_daemon ... ok test.unit.common.test_manager.TestManager.test_no_wait test.unit.common.test_manager.TestManager.test_no_wait ... ok test.unit.common.test_manager.TestManager.test_once test.unit.common.test_manager.TestManager.test_once ... ok test.unit.common.test_manager.TestManager.test_reload test.unit.common.test_manager.TestManager.test_reload ... ok test.unit.common.test_manager.TestManager.test_restart test.unit.common.test_manager.TestManager.test_restart ... ok test.unit.common.test_manager.TestManager.test_run_command test.unit.common.test_manager.TestManager.test_run_command ... ok test.unit.common.test_manager.TestManager.test_shutdown test.unit.common.test_manager.TestManager.test_shutdown ... ok test.unit.common.test_manager.TestManager.test_start test.unit.common.test_manager.TestManager.test_start ... ok test.unit.common.test_manager.TestManager.test_status test.unit.common.test_manager.TestManager.test_status ... ok test.unit.common.test_manager.TestManager.test_stop test.unit.common.test_manager.TestManager.test_stop ... ok test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_info test.unit.container.test_backend.TestCommonContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges ... ok test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts ... ok test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted ... ok test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart ... ok cont1 DEBUG: Could not load '/tmp/tmpb3blqlz_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpb3blqlz_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpb3blqlz_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpb3blqlz_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0370 "-" 3610682 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0383 "-" 3610682 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0470 "-" 3610682 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0463 "-" 3610682 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx991301f2184c48329917d-00665ca276" "proxy-server 3610682" 0.0029 "-" 3610682 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:59575/sda1/1/a/c" "tx991301f2184c48329917d-00665ca276" "container-server 3610682" 0.0091 "-" 3610682 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:42297/sdb1/1/a/c" "tx991301f2184c48329917d-00665ca276" "container-server 3610682" 0.0126 "-" 3610682 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx991301f2184c48329917d-00665ca276" "proxy-server 3610682" 0.1237 "-" 3610682 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx991301f2184c48329917d-00665ca276" "proxy-server 3610682" 0.0704 "-" 3610682 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/54 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx991301f2184c48329917d-00665ca276 - 0.1444 - - 1717346934.660695314 1717346934.805118799 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx3f747dfde154486a98068-00665ca276" "proxy-server 3610682" 0.0109 "-" 3610682 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:59575/sda1/1/a1/c1" "tx3f747dfde154486a98068-00665ca276" "container-server 3610682" 0.0159 "-" 3610682 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:54 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:42297/sdb1/1/a1/c1" "tx3f747dfde154486a98068-00665ca276" "container-server 3610682" 0.0142 "-" 3610682 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx3f747dfde154486a98068-00665ca276" "proxy-server 3610682" 0.1629 "-" 3610682 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx3f747dfde154486a98068-00665ca276" "proxy-server 3610682" 0.0960 "-" 3610682 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/55 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx3f747dfde154486a98068-00665ca276 - 0.1883 - - 1717346934.816555023 1717346935.004851818 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txfd47cf07c1f6475c99600-00665ca277" "proxy-server 3610682" 0.0722 "-" 3610682 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59575/sda1/2/a/c1" "txfd47cf07c1f6475c99600-00665ca277" "container-server 3610682" 0.0228 "-" 3610682 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:42297/sdb1/2/a/c1" "txfd47cf07c1f6475c99600-00665ca277" "container-server 3610682" 0.0024 "-" 3610682 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txfd47cf07c1f6475c99600-00665ca277" "proxy-server 3610682" 0.0932 "-" 3610682 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txfd47cf07c1f6475c99600-00665ca277" "proxy-server 3610682" 0.0677 "-" 3610682 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/55 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txfd47cf07c1f6475c99600-00665ca277 - 0.1978 - - 1717346935.016208172 1717346935.213984728 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf9a689859d48493389de3-00665ca277" "proxy-server 3610682" 0.0272 "-" 3610682 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:42297/sdb1/1/a/c2" "txf9a689859d48493389de3-00665ca277" "container-server 3610682" 0.4276 "-" 3610682 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:59575/sda1/1/a/c2" "txf9a689859d48493389de3-00665ca277" "container-server 3610682" 0.0263 "-" 3610682 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf9a689859d48493389de3-00665ca277" "proxy-server 3610682" 0.5674 "-" 3610682 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:48:55 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf9a689859d48493389de3-00665ca277" "proxy-server 3610682" 0.5311 "-" 3610682 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/48/55 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txf9a689859d48493389de3-00665ca277 - 0.6291 - - 1717346935.217531681 1717346935.846659660 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads ... ok test INFO: Object update sweep completed on /tmp/tmpnps4n3ug/devices/sda1 in 0.20s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 1.13s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp6hs4a469/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c1/o03 /tmp/tmp6hs4a469/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717346946.35047 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp6hs4a469/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717346946.34853 test DEBUG: Update sent for /a/c1/o01 /tmp/tmp6hs4a469/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717346946.35003 test INFO: Object update sweep completed on /tmp/tmp6hs4a469/devices/sda1 in 0.10s seconds:, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.10s, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpo5jlgqi6/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c1/o04 /tmp/tmpo5jlgqi6/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717346946.46036 test DEBUG: Update sent for /a/c1/o03 /tmp/tmpo5jlgqi6/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717346946.46018 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpo5jlgqi6/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717346946.46001 test DEBUG: Update sent for /a/c1/o00 /tmp/tmpo5jlgqi6/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717346946.45965 test INFO: Object update sweep completed on /tmp/tmpo5jlgqi6/devices/sda1 in 0.01s seconds:, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpijfx0y_3/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c1/o00 /tmp/tmpijfx0y_3/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717346946.47832 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpijfx0y_3/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717346946.47868 test INFO: Object update sweep completed on /tmp/tmpijfx0y_3/devices/sda1 in 0.11s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.11s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp78vyjn4p/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c1/o05 /tmp/tmp78vyjn4p/devices/sda1/async_pending/dc4/017ff741f591e7220e0836a0c4fcedc4-1717346946.61103 test DEBUG: Update sent for /a/c1/o03 /tmp/tmp78vyjn4p/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717346946.60829 test DEBUG: Update sent for /a/c1/o01 /tmp/tmp78vyjn4p/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717346946.60780 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp78vyjn4p/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717346946.60747 test DEBUG: Update sent for /a/c1/o02 /tmp/tmp78vyjn4p/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717346946.60798 test DEBUG: Update sent for /a/c1/o08 /tmp/tmp78vyjn4p/devices/sda1/async_pending/5b8/4e594cf882aac2f7f8f2f4cdade285b8-1717346946.61179 test DEBUG: Update sent for /a/c1/o06 /tmp/tmp78vyjn4p/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717346946.61123 test DEBUG: Update sent for /a/c1/o07 /tmp/tmp78vyjn4p/devices/sda1/async_pending/fc8/26ac197975ac1169d403eeb7ae8bbfc8-1717346946.61140 test DEBUG: Update sent for /a/c1/o09 /tmp/tmp78vyjn4p/devices/sda1/async_pending/bee/db77f283b4b1bb5e5b6ce501ca337bee-1717346946.61261 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmp78vyjn4p/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717346946.61339 test DEBUG: Update sent for /a/c1/o04 /tmp/tmp78vyjn4p/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717346946.60847 test INFO: Object update sweep completed on /tmp/tmp78vyjn4p/devices/sda1 in 0.21s seconds:, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.21s, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_9o7xi91/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c1/o01 /tmp/tmp_9o7xi91/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717346946.83365 test DEBUG: Update sent for /a/c1/o06 /tmp/tmp_9o7xi91/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717346946.83630 test DEBUG: Update sent for /a/c1/o04 /tmp/tmp_9o7xi91/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717346946.83592 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp_9o7xi91/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717346946.83335 test DEBUG: Update sent for /a/c1/o02 /tmp/tmp_9o7xi91/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717346946.83527 test INFO: Object update sweep completed on /tmp/tmp_9o7xi91/devices/sda1 in 0.01s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test.unit.obj.test_updater.TestObjectUpdater.test_run_once test.unit.obj.test_updater.TestObjectUpdater.test_run_once ... ok test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container ... ok test-replicator WARNING: [worker 1/3 pid=8804] warning message test-replicator ERROR: [worker 1/3 pid=8804] error message test-replicator INFO: [worker 1/3 pid=8804] Object replication complete (once). (0.00 minutes) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: Setting object count to 1032 (txn: tx9fa05404bc5a48eebf811-00665ca28d) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: Setting object count to 48 (txn: tx9d52c4b1eec3451189085-00665ca28d) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: Setting object count to 1064 (txn: txfd95bbef53e4478f8c9c7-00665ca28d) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx8e53021bdbfd4de9babdb-00665ca28d) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txc0a204d10301450aa6f0e-00665ca28d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc0a204d10301450aa6f0e-00665ca28d) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory ... ok test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' - rejecting request test INFO: Begin container update single threaded sweep test ERROR: Error processing container /tmp/tmpr4e3thr3/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db: Boom!: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/updater.py", line 225, in container_sweep self.process_container(dbfile) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Boom! test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: ERROR: Failed to get paths to drive partitions: permission_denied test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpf1m7sso4/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpf1m7sso4/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.03s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpf1m7sso4/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test.unit.container.test_updater.TestContainerUpdater.test_run_once test.unit.container.test_updater.TestContainerUpdater.test_run_once ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout ... ok DEBUG: New builder; performing initial balance New builder; performing initial balance DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 768 new parts and 0 removed parts from replica-count change 768 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 190 (Last start was 0) Gather start is 190 (Last start was 0) DEBUG: Gathered 768 parts Gathered 768 parts DEBUG: Placed 10/0 onto dev r1z1-10.1.1.1/sdd Placed 10/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 10/1 onto dev r1z1-10.1.1.1/sda Placed 10/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 10/2 onto dev r1z1-10.1.1.1/sdb Placed 10/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 120/0 onto dev r1z1-10.1.1.1/sdc Placed 120/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 120/1 onto dev r1z1-10.1.1.1/sdd Placed 120/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 120/2 onto dev r1z1-10.1.1.1/sda Placed 120/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 196/0 onto dev r1z1-10.1.1.1/sdb Placed 196/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 196/1 onto dev r1z1-10.1.1.1/sdc Placed 196/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 196/2 onto dev r1z1-10.1.1.1/sdd Placed 196/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 118/0 onto dev r1z1-10.1.1.1/sda Placed 118/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 118/1 onto dev r1z1-10.1.1.1/sdb Placed 118/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 118/2 onto dev r1z1-10.1.1.1/sdc Placed 118/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 158/0 onto dev r1z1-10.1.1.1/sdd Placed 158/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 158/1 onto dev r1z1-10.1.1.1/sda Placed 158/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 158/2 onto dev r1z1-10.1.1.1/sdb Placed 158/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 112/0 onto dev r1z1-10.1.1.1/sdc Placed 112/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 112/1 onto dev r1z1-10.1.1.1/sdd Placed 112/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 112/2 onto dev r1z1-10.1.1.1/sda Placed 112/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 177/0 onto dev r1z1-10.1.1.1/sdb Placed 177/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 177/1 onto dev r1z1-10.1.1.1/sdc Placed 177/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 177/2 onto dev r1z1-10.1.1.1/sdd Placed 177/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 98/0 onto dev r1z1-10.1.1.1/sda Placed 98/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 98/1 onto dev r1z1-10.1.1.1/sdb Placed 98/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 98/2 onto dev r1z1-10.1.1.1/sdc Placed 98/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 3/0 onto dev r1z1-10.1.1.1/sdd Placed 3/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 3/1 onto dev r1z1-10.1.1.1/sda Placed 3/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 3/2 onto dev r1z1-10.1.1.1/sdb Placed 3/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 135/0 onto dev r1z1-10.1.1.1/sdc Placed 135/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 135/1 onto dev r1z1-10.1.1.1/sdd Placed 135/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 135/2 onto dev r1z1-10.1.1.1/sda Placed 135/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 72/0 onto dev r1z1-10.1.1.1/sdb Placed 72/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 72/1 onto dev r1z1-10.1.1.1/sdc Placed 72/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 72/2 onto dev r1z1-10.1.1.1/sdd Placed 72/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 28/0 onto dev r1z1-10.1.1.1/sda Placed 28/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 28/1 onto dev r1z1-10.1.1.1/sdb Placed 28/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 28/2 onto dev r1z1-10.1.1.1/sdc Placed 28/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 193/0 onto dev r1z1-10.1.1.1/sdd Placed 193/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 193/1 onto dev r1z1-10.1.1.1/sda Placed 193/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 193/2 onto dev r1z1-10.1.1.1/sdb Placed 193/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 238/0 onto dev r1z1-10.1.1.1/sdc Placed 238/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 238/1 onto dev r1z1-10.1.1.1/sdd Placed 238/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 238/2 onto dev r1z1-10.1.1.1/sda Placed 238/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 245/0 onto dev r1z1-10.1.1.1/sdb Placed 245/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 245/1 onto dev r1z1-10.1.1.1/sdc Placed 245/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 245/2 onto dev r1z1-10.1.1.1/sdd Placed 245/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 145/0 onto dev r1z1-10.1.1.1/sda Placed 145/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 145/1 onto dev r1z1-10.1.1.1/sdb Placed 145/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 145/2 onto dev r1z1-10.1.1.1/sdc Placed 145/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 223/0 onto dev r1z1-10.1.1.1/sdd Placed 223/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 223/1 onto dev r1z1-10.1.1.1/sda Placed 223/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 223/2 onto dev r1z1-10.1.1.1/sdb Placed 223/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 212/0 onto dev r1z1-10.1.1.1/sdc Placed 212/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 212/1 onto dev r1z1-10.1.1.1/sdd Placed 212/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 212/2 onto dev r1z1-10.1.1.1/sda Placed 212/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 104/0 onto dev r1z1-10.1.1.1/sdb Placed 104/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 104/1 onto dev r1z1-10.1.1.1/sdc Placed 104/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 104/2 onto dev r1z1-10.1.1.1/sdd Placed 104/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 56/0 onto dev r1z1-10.1.1.1/sda Placed 56/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 56/1 onto dev r1z1-10.1.1.1/sdb Placed 56/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 56/2 onto dev r1z1-10.1.1.1/sdc Placed 56/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 123/0 onto dev r1z1-10.1.1.1/sdd Placed 123/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 123/1 onto dev r1z1-10.1.1.1/sda Placed 123/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 123/2 onto dev r1z1-10.1.1.1/sdb Placed 123/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 68/0 onto dev r1z1-10.1.1.1/sdc Placed 68/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 68/1 onto dev r1z1-10.1.1.1/sdd Placed 68/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 68/2 onto dev r1z1-10.1.1.1/sda Placed 68/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 25/0 onto dev r1z1-10.1.1.1/sdb Placed 25/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 25/1 onto dev r1z1-10.1.1.1/sdc Placed 25/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 25/2 onto dev r1z1-10.1.1.1/sdd Placed 25/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 42/0 onto dev r1z1-10.1.1.1/sda Placed 42/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 42/1 onto dev r1z1-10.1.1.1/sdb Placed 42/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 42/2 onto dev r1z1-10.1.1.1/sdc Placed 42/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 189/0 onto dev r1z1-10.1.1.1/sdd Placed 189/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 189/1 onto dev r1z1-10.1.1.1/sda Placed 189/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 189/2 onto dev r1z1-10.1.1.1/sdb Placed 189/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 2/0 onto dev r1z1-10.1.1.1/sdc Placed 2/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 2/1 onto dev r1z1-10.1.1.1/sdd Placed 2/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 2/2 onto dev r1z1-10.1.1.1/sda Placed 2/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 178/0 onto dev r1z1-10.1.1.1/sdb Placed 178/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 178/1 onto dev r1z1-10.1.1.1/sdc Placed 178/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 178/2 onto dev r1z1-10.1.1.1/sdd Placed 178/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 53/0 onto dev r1z1-10.1.1.1/sda Placed 53/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 53/1 onto dev r1z1-10.1.1.1/sdb Placed 53/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 53/2 onto dev r1z1-10.1.1.1/sdc Placed 53/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 75/0 onto dev r1z1-10.1.1.1/sdd Placed 75/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 75/1 onto dev r1z1-10.1.1.1/sda Placed 75/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 75/2 onto dev r1z1-10.1.1.1/sdb Placed 75/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 45/0 onto dev r1z1-10.1.1.1/sdc Placed 45/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 45/1 onto dev r1z1-10.1.1.1/sdd Placed 45/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 45/2 onto dev r1z1-10.1.1.1/sda Placed 45/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 208/0 onto dev r1z1-10.1.1.1/sdb Placed 208/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 208/1 onto dev r1z1-10.1.1.1/sdc Placed 208/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 208/2 onto dev r1z1-10.1.1.1/sdd Placed 208/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 99/0 onto dev r1z1-10.1.1.1/sda Placed 99/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 99/1 onto dev r1z1-10.1.1.1/sdb Placed 99/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 99/2 onto dev r1z1-10.1.1.1/sdc Placed 99/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 81/0 onto dev r1z1-10.1.1.1/sdd Placed 81/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 81/1 onto dev r1z1-10.1.1.1/sda Placed 81/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 81/2 onto dev r1z1-10.1.1.1/sdb Placed 81/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 185/0 onto dev r1z1-10.1.1.1/sdc Placed 185/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 185/1 onto dev r1z1-10.1.1.1/sdd Placed 185/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 185/2 onto dev r1z1-10.1.1.1/sda Placed 185/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 164/0 onto dev r1z1-10.1.1.1/sdb Placed 164/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 164/1 onto dev r1z1-10.1.1.1/sdc Placed 164/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 164/2 onto dev r1z1-10.1.1.1/sdd Placed 164/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 80/0 onto dev r1z1-10.1.1.1/sda Placed 80/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 80/1 onto dev r1z1-10.1.1.1/sdb Placed 80/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 80/2 onto dev r1z1-10.1.1.1/sdc Placed 80/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 4/0 onto dev r1z1-10.1.1.1/sdd Placed 4/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 4/1 onto dev r1z1-10.1.1.1/sda Placed 4/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 4/2 onto dev r1z1-10.1.1.1/sdb Placed 4/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 170/0 onto dev r1z1-10.1.1.1/sdc Placed 170/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 170/1 onto dev r1z1-10.1.1.1/sdd Placed 170/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 170/2 onto dev r1z1-10.1.1.1/sda Placed 170/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 128/0 onto dev r1z1-10.1.1.1/sdb Placed 128/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 128/1 onto dev r1z1-10.1.1.1/sdc Placed 128/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 128/2 onto dev r1z1-10.1.1.1/sdd Placed 128/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 141/0 onto dev r1z1-10.1.1.1/sda Placed 141/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 141/1 onto dev r1z1-10.1.1.1/sdb Placed 141/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 141/2 onto dev r1z1-10.1.1.1/sdc Placed 141/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 182/0 onto dev r1z1-10.1.1.1/sdd Placed 182/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 182/1 onto dev r1z1-10.1.1.1/sda Placed 182/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 182/2 onto dev r1z1-10.1.1.1/sdb Placed 182/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 90/0 onto dev r1z1-10.1.1.1/sdc Placed 90/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 90/1 onto dev r1z1-10.1.1.1/sdd Placed 90/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 90/2 onto dev r1z1-10.1.1.1/sda Placed 90/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 0/0 onto dev r1z1-10.1.1.1/sdb Placed 0/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 0/1 onto dev r1z1-10.1.1.1/sdc Placed 0/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 0/2 onto dev r1z1-10.1.1.1/sdd Placed 0/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 83/0 onto dev r1z1-10.1.1.1/sda Placed 83/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 83/1 onto dev r1z1-10.1.1.1/sdb Placed 83/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 83/2 onto dev r1z1-10.1.1.1/sdc Placed 83/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 197/0 onto dev r1z1-10.1.1.1/sdd Placed 197/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 197/1 onto dev r1z1-10.1.1.1/sda Placed 197/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 197/2 onto dev r1z1-10.1.1.1/sdb Placed 197/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 138/0 onto dev r1z1-10.1.1.1/sdc Placed 138/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 138/1 onto dev r1z1-10.1.1.1/sdd Placed 138/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 138/2 onto dev r1z1-10.1.1.1/sda Placed 138/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 209/0 onto dev r1z1-10.1.1.1/sdb Placed 209/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 209/1 onto dev r1z1-10.1.1.1/sdc Placed 209/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 209/2 onto dev r1z1-10.1.1.1/sdd Placed 209/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 206/0 onto dev r1z1-10.1.1.1/sda Placed 206/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 206/1 onto dev r1z1-10.1.1.1/sdb Placed 206/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 206/2 onto dev r1z1-10.1.1.1/sdc Placed 206/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 255/0 onto dev r1z1-10.1.1.1/sdd Placed 255/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 255/1 onto dev r1z1-10.1.1.1/sda Placed 255/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 255/2 onto dev r1z1-10.1.1.1/sdb Placed 255/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 132/0 onto dev r1z1-10.1.1.1/sdc Placed 132/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 132/1 onto dev r1z1-10.1.1.1/sdd Placed 132/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 132/2 onto dev r1z1-10.1.1.1/sda Placed 132/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 70/0 onto dev r1z1-10.1.1.1/sdb Placed 70/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 70/1 onto dev r1z1-10.1.1.1/sdc Placed 70/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 70/2 onto dev r1z1-10.1.1.1/sdd Placed 70/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 77/0 onto dev r1z1-10.1.1.1/sda Placed 77/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 77/1 onto dev r1z1-10.1.1.1/sdb Placed 77/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 77/2 onto dev r1z1-10.1.1.1/sdc Placed 77/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 156/0 onto dev r1z1-10.1.1.1/sdd Placed 156/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 156/1 onto dev r1z1-10.1.1.1/sda Placed 156/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 156/2 onto dev r1z1-10.1.1.1/sdb Placed 156/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 252/0 onto dev r1z1-10.1.1.1/sdc Placed 252/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 252/1 onto dev r1z1-10.1.1.1/sdd Placed 252/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 252/2 onto dev r1z1-10.1.1.1/sda Placed 252/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 13/0 onto dev r1z1-10.1.1.1/sdb Placed 13/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 13/1 onto dev r1z1-10.1.1.1/sdc Placed 13/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 13/2 onto dev r1z1-10.1.1.1/sdd Placed 13/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 249/0 onto dev r1z1-10.1.1.1/sda Placed 249/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 249/1 onto dev r1z1-10.1.1.1/sdb Placed 249/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 249/2 onto dev r1z1-10.1.1.1/sdc Placed 249/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 38/0 onto dev r1z1-10.1.1.1/sdd Placed 38/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 38/1 onto dev r1z1-10.1.1.1/sda Placed 38/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 38/2 onto dev r1z1-10.1.1.1/sdb Placed 38/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 67/0 onto dev r1z1-10.1.1.1/sdc Placed 67/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 67/1 onto dev r1z1-10.1.1.1/sdd Placed 67/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 67/2 onto dev r1z1-10.1.1.1/sda Placed 67/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 9/0 onto dev r1z1-10.1.1.1/sdb Placed 9/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 9/1 onto dev r1z1-10.1.1.1/sdc Placed 9/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 9/2 onto dev r1z1-10.1.1.1/sdd Placed 9/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 97/0 onto dev r1z1-10.1.1.1/sda Placed 97/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 97/1 onto dev r1z1-10.1.1.1/sdb Placed 97/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 97/2 onto dev r1z1-10.1.1.1/sdc Placed 97/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 36/0 onto dev r1z1-10.1.1.1/sdd Placed 36/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 36/1 onto dev r1z1-10.1.1.1/sda Placed 36/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 36/2 onto dev r1z1-10.1.1.1/sdb Placed 36/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 200/0 onto dev r1z1-10.1.1.1/sdc Placed 200/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 200/1 onto dev r1z1-10.1.1.1/sdd Placed 200/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 200/2 onto dev r1z1-10.1.1.1/sda Placed 200/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 31/0 onto dev r1z1-10.1.1.1/sdb Placed 31/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 31/1 onto dev r1z1-10.1.1.1/sdc Placed 31/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 31/2 onto dev r1z1-10.1.1.1/sdd Placed 31/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 100/0 onto dev r1z1-10.1.1.1/sda Placed 100/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 100/1 onto dev r1z1-10.1.1.1/sdb Placed 100/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 100/2 onto dev r1z1-10.1.1.1/sdc Placed 100/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 251/0 onto dev r1z1-10.1.1.1/sdd Placed 251/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 251/1 onto dev r1z1-10.1.1.1/sda Placed 251/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 251/2 onto dev r1z1-10.1.1.1/sdb Placed 251/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 180/0 onto dev r1z1-10.1.1.1/sdc Placed 180/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 180/1 onto dev r1z1-10.1.1.1/sdd Placed 180/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 180/2 onto dev r1z1-10.1.1.1/sda Placed 180/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 20/0 onto dev r1z1-10.1.1.1/sdb Placed 20/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 20/1 onto dev r1z1-10.1.1.1/sdc Placed 20/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 20/2 onto dev r1z1-10.1.1.1/sdd Placed 20/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 29/0 onto dev r1z1-10.1.1.1/sda Placed 29/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 29/1 onto dev r1z1-10.1.1.1/sdb Placed 29/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 29/2 onto dev r1z1-10.1.1.1/sdc Placed 29/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 143/0 onto dev r1z1-10.1.1.1/sdd Placed 143/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 143/1 onto dev r1z1-10.1.1.1/sda Placed 143/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 143/2 onto dev r1z1-10.1.1.1/sdb Placed 143/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 74/0 onto dev r1z1-10.1.1.1/sdc Placed 74/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 74/1 onto dev r1z1-10.1.1.1/sdd Placed 74/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 74/2 onto dev r1z1-10.1.1.1/sda Placed 74/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 244/0 onto dev r1z1-10.1.1.1/sdb Placed 244/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 244/1 onto dev r1z1-10.1.1.1/sdc Placed 244/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 244/2 onto dev r1z1-10.1.1.1/sdd Placed 244/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 190/0 onto dev r1z1-10.1.1.1/sda Placed 190/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 190/1 onto dev r1z1-10.1.1.1/sdb Placed 190/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 190/2 onto dev r1z1-10.1.1.1/sdc Placed 190/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 55/0 onto dev r1z1-10.1.1.1/sdd Placed 55/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 55/1 onto dev r1z1-10.1.1.1/sda Placed 55/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 55/2 onto dev r1z1-10.1.1.1/sdb Placed 55/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 105/0 onto dev r1z1-10.1.1.1/sdc Placed 105/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 105/1 onto dev r1z1-10.1.1.1/sdd Placed 105/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 105/2 onto dev r1z1-10.1.1.1/sda Placed 105/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 228/0 onto dev r1z1-10.1.1.1/sdb Placed 228/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 228/1 onto dev r1z1-10.1.1.1/sdc Placed 228/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 228/2 onto dev r1z1-10.1.1.1/sdd Placed 228/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 225/0 onto dev r1z1-10.1.1.1/sda Placed 225/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 225/1 onto dev r1z1-10.1.1.1/sdb Placed 225/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 225/2 onto dev r1z1-10.1.1.1/sdc Placed 225/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 149/0 onto dev r1z1-10.1.1.1/sdd Placed 149/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 149/1 onto dev r1z1-10.1.1.1/sda Placed 149/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 149/2 onto dev r1z1-10.1.1.1/sdb Placed 149/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 32/0 onto dev r1z1-10.1.1.1/sdc Placed 32/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 32/1 onto dev r1z1-10.1.1.1/sdd Placed 32/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 32/2 onto dev r1z1-10.1.1.1/sda Placed 32/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 181/0 onto dev r1z1-10.1.1.1/sdb Placed 181/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 181/1 onto dev r1z1-10.1.1.1/sdc Placed 181/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 181/2 onto dev r1z1-10.1.1.1/sdd Placed 181/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 207/0 onto dev r1z1-10.1.1.1/sda Placed 207/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 207/1 onto dev r1z1-10.1.1.1/sdb Placed 207/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 207/2 onto dev r1z1-10.1.1.1/sdc Placed 207/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 210/0 onto dev r1z1-10.1.1.1/sdd Placed 210/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 210/1 onto dev r1z1-10.1.1.1/sda Placed 210/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 210/2 onto dev r1z1-10.1.1.1/sdb Placed 210/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 26/0 onto dev r1z1-10.1.1.1/sdc Placed 26/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 26/1 onto dev r1z1-10.1.1.1/sdd Placed 26/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 26/2 onto dev r1z1-10.1.1.1/sda Placed 26/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 49/0 onto dev r1z1-10.1.1.1/sdb Placed 49/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 49/1 onto dev r1z1-10.1.1.1/sdc Placed 49/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 49/2 onto dev r1z1-10.1.1.1/sdd Placed 49/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 224/0 onto dev r1z1-10.1.1.1/sda Placed 224/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 224/1 onto dev r1z1-10.1.1.1/sdb Placed 224/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 224/2 onto dev r1z1-10.1.1.1/sdc Placed 224/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 30/0 onto dev r1z1-10.1.1.1/sdd Placed 30/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 30/1 onto dev r1z1-10.1.1.1/sda Placed 30/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 30/2 onto dev r1z1-10.1.1.1/sdb Placed 30/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 246/0 onto dev r1z1-10.1.1.1/sdc Placed 246/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 246/1 onto dev r1z1-10.1.1.1/sdd Placed 246/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 246/2 onto dev r1z1-10.1.1.1/sda Placed 246/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 7/0 onto dev r1z1-10.1.1.1/sdb Placed 7/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 7/1 onto dev r1z1-10.1.1.1/sdc Placed 7/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 7/2 onto dev r1z1-10.1.1.1/sdd Placed 7/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 73/0 onto dev r1z1-10.1.1.1/sda Placed 73/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 73/1 onto dev r1z1-10.1.1.1/sdb Placed 73/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 73/2 onto dev r1z1-10.1.1.1/sdc Placed 73/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 119/0 onto dev r1z1-10.1.1.1/sdd Placed 119/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 119/1 onto dev r1z1-10.1.1.1/sda Placed 119/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 119/2 onto dev r1z1-10.1.1.1/sdb Placed 119/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 57/0 onto dev r1z1-10.1.1.1/sdc Placed 57/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 57/1 onto dev r1z1-10.1.1.1/sdd Placed 57/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 57/2 onto dev r1z1-10.1.1.1/sda Placed 57/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 84/0 onto dev r1z1-10.1.1.1/sdb Placed 84/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 84/1 onto dev r1z1-10.1.1.1/sdc Placed 84/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 84/2 onto dev r1z1-10.1.1.1/sdd Placed 84/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 102/0 onto dev r1z1-10.1.1.1/sda Placed 102/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 102/1 onto dev r1z1-10.1.1.1/sdb Placed 102/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 102/2 onto dev r1z1-10.1.1.1/sdc Placed 102/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 169/0 onto dev r1z1-10.1.1.1/sdd Placed 169/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 169/1 onto dev r1z1-10.1.1.1/sda Placed 169/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 169/2 onto dev r1z1-10.1.1.1/sdb Placed 169/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 86/0 onto dev r1z1-10.1.1.1/sdc Placed 86/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 86/1 onto dev r1z1-10.1.1.1/sdd Placed 86/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 86/2 onto dev r1z1-10.1.1.1/sda Placed 86/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 114/0 onto dev r1z1-10.1.1.1/sdb Placed 114/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 114/1 onto dev r1z1-10.1.1.1/sdc Placed 114/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 114/2 onto dev r1z1-10.1.1.1/sdd Placed 114/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 108/0 onto dev r1z1-10.1.1.1/sda Placed 108/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 108/1 onto dev r1z1-10.1.1.1/sdb Placed 108/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 108/2 onto dev r1z1-10.1.1.1/sdc Placed 108/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 203/0 onto dev r1z1-10.1.1.1/sdd Placed 203/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 203/1 onto dev r1z1-10.1.1.1/sda Placed 203/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 203/2 onto dev r1z1-10.1.1.1/sdb Placed 203/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 11/0 onto dev r1z1-10.1.1.1/sdc Placed 11/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 11/1 onto dev r1z1-10.1.1.1/sdd Placed 11/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 11/2 onto dev r1z1-10.1.1.1/sda Placed 11/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 60/0 onto dev r1z1-10.1.1.1/sdb Placed 60/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 60/1 onto dev r1z1-10.1.1.1/sdc Placed 60/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 60/2 onto dev r1z1-10.1.1.1/sdd Placed 60/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 23/0 onto dev r1z1-10.1.1.1/sda Placed 23/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 23/1 onto dev r1z1-10.1.1.1/sdb Placed 23/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 23/2 onto dev r1z1-10.1.1.1/sdc Placed 23/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 126/0 onto dev r1z1-10.1.1.1/sdd Placed 126/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 126/1 onto dev r1z1-10.1.1.1/sda Placed 126/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 126/2 onto dev r1z1-10.1.1.1/sdb Placed 126/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 22/0 onto dev r1z1-10.1.1.1/sdc Placed 22/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 22/1 onto dev r1z1-10.1.1.1/sdd Placed 22/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 22/2 onto dev r1z1-10.1.1.1/sda Placed 22/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 115/0 onto dev r1z1-10.1.1.1/sdb Placed 115/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 115/1 onto dev r1z1-10.1.1.1/sdc Placed 115/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 115/2 onto dev r1z1-10.1.1.1/sdd Placed 115/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 8/0 onto dev r1z1-10.1.1.1/sda Placed 8/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 8/1 onto dev r1z1-10.1.1.1/sdb Placed 8/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 8/2 onto dev r1z1-10.1.1.1/sdc Placed 8/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 234/0 onto dev r1z1-10.1.1.1/sdd Placed 234/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 234/1 onto dev r1z1-10.1.1.1/sda Placed 234/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 234/2 onto dev r1z1-10.1.1.1/sdb Placed 234/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 230/0 onto dev r1z1-10.1.1.1/sdc Placed 230/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 230/1 onto dev r1z1-10.1.1.1/sdd Placed 230/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 230/2 onto dev r1z1-10.1.1.1/sda Placed 230/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 142/0 onto dev r1z1-10.1.1.1/sdb Placed 142/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 142/1 onto dev r1z1-10.1.1.1/sdc Placed 142/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 142/2 onto dev r1z1-10.1.1.1/sdd Placed 142/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 187/0 onto dev r1z1-10.1.1.1/sda Placed 187/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 187/1 onto dev r1z1-10.1.1.1/sdb Placed 187/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 187/2 onto dev r1z1-10.1.1.1/sdc Placed 187/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 198/0 onto dev r1z1-10.1.1.1/sdd Placed 198/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 198/1 onto dev r1z1-10.1.1.1/sda Placed 198/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 198/2 onto dev r1z1-10.1.1.1/sdb Placed 198/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 78/0 onto dev r1z1-10.1.1.1/sdc Placed 78/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 78/1 onto dev r1z1-10.1.1.1/sdd Placed 78/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 78/2 onto dev r1z1-10.1.1.1/sda Placed 78/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 101/0 onto dev r1z1-10.1.1.1/sdb Placed 101/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 101/1 onto dev r1z1-10.1.1.1/sdc Placed 101/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 101/2 onto dev r1z1-10.1.1.1/sdd Placed 101/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 236/0 onto dev r1z1-10.1.1.1/sda Placed 236/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 236/1 onto dev r1z1-10.1.1.1/sdb Placed 236/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 236/2 onto dev r1z1-10.1.1.1/sdc Placed 236/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 151/0 onto dev r1z1-10.1.1.1/sdd Placed 151/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 151/1 onto dev r1z1-10.1.1.1/sda Placed 151/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 151/2 onto dev r1z1-10.1.1.1/sdb Placed 151/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 254/0 onto dev r1z1-10.1.1.1/sdc Placed 254/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 254/1 onto dev r1z1-10.1.1.1/sdd Placed 254/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 254/2 onto dev r1z1-10.1.1.1/sda Placed 254/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 175/0 onto dev r1z1-10.1.1.1/sdb Placed 175/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 175/1 onto dev r1z1-10.1.1.1/sdc Placed 175/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 175/2 onto dev r1z1-10.1.1.1/sdd Placed 175/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 52/0 onto dev r1z1-10.1.1.1/sda Placed 52/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 52/1 onto dev r1z1-10.1.1.1/sdb Placed 52/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 52/2 onto dev r1z1-10.1.1.1/sdc Placed 52/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 163/0 onto dev r1z1-10.1.1.1/sdd Placed 163/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 163/1 onto dev r1z1-10.1.1.1/sda Placed 163/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 163/2 onto dev r1z1-10.1.1.1/sdb Placed 163/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 50/0 onto dev r1z1-10.1.1.1/sdc Placed 50/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 50/1 onto dev r1z1-10.1.1.1/sdd Placed 50/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 50/2 onto dev r1z1-10.1.1.1/sda Placed 50/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 92/0 onto dev r1z1-10.1.1.1/sdb Placed 92/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 92/1 onto dev r1z1-10.1.1.1/sdc Placed 92/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 92/2 onto dev r1z1-10.1.1.1/sdd Placed 92/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 48/0 onto dev r1z1-10.1.1.1/sda Placed 48/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 48/1 onto dev r1z1-10.1.1.1/sdb Placed 48/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 48/2 onto dev r1z1-10.1.1.1/sdc Placed 48/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 184/0 onto dev r1z1-10.1.1.1/sdd Placed 184/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 184/1 onto dev r1z1-10.1.1.1/sda Placed 184/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 184/2 onto dev r1z1-10.1.1.1/sdb Placed 184/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 199/0 onto dev r1z1-10.1.1.1/sdc Placed 199/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 199/1 onto dev r1z1-10.1.1.1/sdd Placed 199/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 199/2 onto dev r1z1-10.1.1.1/sda Placed 199/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 226/0 onto dev r1z1-10.1.1.1/sdb Placed 226/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 226/1 onto dev r1z1-10.1.1.1/sdc Placed 226/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 226/2 onto dev r1z1-10.1.1.1/sdd Placed 226/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 248/0 onto dev r1z1-10.1.1.1/sda Placed 248/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 248/1 onto dev r1z1-10.1.1.1/sdb Placed 248/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 248/2 onto dev r1z1-10.1.1.1/sdc Placed 248/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 47/0 onto dev r1z1-10.1.1.1/sdd Placed 47/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 47/1 onto dev r1z1-10.1.1.1/sda Placed 47/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 47/2 onto dev r1z1-10.1.1.1/sdb Placed 47/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 121/0 onto dev r1z1-10.1.1.1/sdc Placed 121/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 121/1 onto dev r1z1-10.1.1.1/sdd Placed 121/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 121/2 onto dev r1z1-10.1.1.1/sda Placed 121/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 64/0 onto dev r1z1-10.1.1.1/sdb Placed 64/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 64/1 onto dev r1z1-10.1.1.1/sdc Placed 64/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 64/2 onto dev r1z1-10.1.1.1/sdd Placed 64/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 106/0 onto dev r1z1-10.1.1.1/sda Placed 106/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 106/1 onto dev r1z1-10.1.1.1/sdb Placed 106/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 106/2 onto dev r1z1-10.1.1.1/sdc Placed 106/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 40/0 onto dev r1z1-10.1.1.1/sdd Placed 40/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 40/1 onto dev r1z1-10.1.1.1/sda Placed 40/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 40/2 onto dev r1z1-10.1.1.1/sdb Placed 40/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 204/0 onto dev r1z1-10.1.1.1/sdc Placed 204/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 204/1 onto dev r1z1-10.1.1.1/sdd Placed 204/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 204/2 onto dev r1z1-10.1.1.1/sda Placed 204/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 103/0 onto dev r1z1-10.1.1.1/sdb Placed 103/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 103/1 onto dev r1z1-10.1.1.1/sdc Placed 103/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 103/2 onto dev r1z1-10.1.1.1/sdd Placed 103/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 215/0 onto dev r1z1-10.1.1.1/sda Placed 215/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 215/1 onto dev r1z1-10.1.1.1/sdb Placed 215/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 215/2 onto dev r1z1-10.1.1.1/sdc Placed 215/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 37/0 onto dev r1z1-10.1.1.1/sdd Placed 37/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 37/1 onto dev r1z1-10.1.1.1/sda Placed 37/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 37/2 onto dev r1z1-10.1.1.1/sdb Placed 37/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 232/0 onto dev r1z1-10.1.1.1/sdc Placed 232/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 232/1 onto dev r1z1-10.1.1.1/sdd Placed 232/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 232/2 onto dev r1z1-10.1.1.1/sda Placed 232/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 183/0 onto dev r1z1-10.1.1.1/sdb Placed 183/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 183/1 onto dev r1z1-10.1.1.1/sdc Placed 183/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 183/2 onto dev r1z1-10.1.1.1/sdd Placed 183/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 89/0 onto dev r1z1-10.1.1.1/sda Placed 89/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 89/1 onto dev r1z1-10.1.1.1/sdb Placed 89/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 89/2 onto dev r1z1-10.1.1.1/sdc Placed 89/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 173/0 onto dev r1z1-10.1.1.1/sdd Placed 173/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 173/1 onto dev r1z1-10.1.1.1/sda Placed 173/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 173/2 onto dev r1z1-10.1.1.1/sdb Placed 173/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 34/0 onto dev r1z1-10.1.1.1/sdc Placed 34/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 34/1 onto dev r1z1-10.1.1.1/sdd Placed 34/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 34/2 onto dev r1z1-10.1.1.1/sda Placed 34/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 136/0 onto dev r1z1-10.1.1.1/sdb Placed 136/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 136/1 onto dev r1z1-10.1.1.1/sdc Placed 136/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 136/2 onto dev r1z1-10.1.1.1/sdd Placed 136/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 139/0 onto dev r1z1-10.1.1.1/sda Placed 139/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 139/1 onto dev r1z1-10.1.1.1/sdb Placed 139/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 139/2 onto dev r1z1-10.1.1.1/sdc Placed 139/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 227/0 onto dev r1z1-10.1.1.1/sdd Placed 227/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 227/1 onto dev r1z1-10.1.1.1/sda Placed 227/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 227/2 onto dev r1z1-10.1.1.1/sdb Placed 227/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 6/0 onto dev r1z1-10.1.1.1/sdc Placed 6/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 6/1 onto dev r1z1-10.1.1.1/sdd Placed 6/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 6/2 onto dev r1z1-10.1.1.1/sda Placed 6/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 71/0 onto dev r1z1-10.1.1.1/sdb Placed 71/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 71/1 onto dev r1z1-10.1.1.1/sdc Placed 71/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 71/2 onto dev r1z1-10.1.1.1/sdd Placed 71/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 191/0 onto dev r1z1-10.1.1.1/sda Placed 191/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 191/1 onto dev r1z1-10.1.1.1/sdb Placed 191/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 191/2 onto dev r1z1-10.1.1.1/sdc Placed 191/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 222/0 onto dev r1z1-10.1.1.1/sdd Placed 222/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 222/1 onto dev r1z1-10.1.1.1/sda Placed 222/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 222/2 onto dev r1z1-10.1.1.1/sdb Placed 222/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 213/0 onto dev r1z1-10.1.1.1/sdc Placed 213/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 213/1 onto dev r1z1-10.1.1.1/sdd Placed 213/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 213/2 onto dev r1z1-10.1.1.1/sda Placed 213/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 194/0 onto dev r1z1-10.1.1.1/sdb Placed 194/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 194/1 onto dev r1z1-10.1.1.1/sdc Placed 194/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 194/2 onto dev r1z1-10.1.1.1/sdd Placed 194/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 19/0 onto dev r1z1-10.1.1.1/sda Placed 19/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 19/1 onto dev r1z1-10.1.1.1/sdb Placed 19/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 19/2 onto dev r1z1-10.1.1.1/sdc Placed 19/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 66/0 onto dev r1z1-10.1.1.1/sdd Placed 66/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 66/1 onto dev r1z1-10.1.1.1/sda Placed 66/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 66/2 onto dev r1z1-10.1.1.1/sdb Placed 66/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 217/0 onto dev r1z1-10.1.1.1/sdc Placed 217/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 217/1 onto dev r1z1-10.1.1.1/sdd Placed 217/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 217/2 onto dev r1z1-10.1.1.1/sda Placed 217/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 146/0 onto dev r1z1-10.1.1.1/sdb Placed 146/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 146/1 onto dev r1z1-10.1.1.1/sdc Placed 146/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 146/2 onto dev r1z1-10.1.1.1/sdd Placed 146/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 152/0 onto dev r1z1-10.1.1.1/sda Placed 152/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 152/1 onto dev r1z1-10.1.1.1/sdb Placed 152/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 152/2 onto dev r1z1-10.1.1.1/sdc Placed 152/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 33/0 onto dev r1z1-10.1.1.1/sdd Placed 33/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 33/1 onto dev r1z1-10.1.1.1/sda Placed 33/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 33/2 onto dev r1z1-10.1.1.1/sdb Placed 33/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 41/0 onto dev r1z1-10.1.1.1/sdc Placed 41/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 41/1 onto dev r1z1-10.1.1.1/sdd Placed 41/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 41/2 onto dev r1z1-10.1.1.1/sda Placed 41/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 17/0 onto dev r1z1-10.1.1.1/sdb Placed 17/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 17/1 onto dev r1z1-10.1.1.1/sdc Placed 17/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 17/2 onto dev r1z1-10.1.1.1/sdd Placed 17/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 186/0 onto dev r1z1-10.1.1.1/sda Placed 186/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 186/1 onto dev r1z1-10.1.1.1/sdb Placed 186/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 186/2 onto dev r1z1-10.1.1.1/sdc Placed 186/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 76/0 onto dev r1z1-10.1.1.1/sdd Placed 76/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 76/1 onto dev r1z1-10.1.1.1/sda Placed 76/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 76/2 onto dev r1z1-10.1.1.1/sdb Placed 76/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 159/0 onto dev r1z1-10.1.1.1/sdc Placed 159/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 159/1 onto dev r1z1-10.1.1.1/sdd Placed 159/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 159/2 onto dev r1z1-10.1.1.1/sda Placed 159/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 241/0 onto dev r1z1-10.1.1.1/sdb Placed 241/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 241/1 onto dev r1z1-10.1.1.1/sdc Placed 241/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 241/2 onto dev r1z1-10.1.1.1/sdd Placed 241/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 116/0 onto dev r1z1-10.1.1.1/sda Placed 116/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 116/1 onto dev r1z1-10.1.1.1/sdb Placed 116/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 116/2 onto dev r1z1-10.1.1.1/sdc Placed 116/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 174/0 onto dev r1z1-10.1.1.1/sdd Placed 174/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 174/1 onto dev r1z1-10.1.1.1/sda Placed 174/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 174/2 onto dev r1z1-10.1.1.1/sdb Placed 174/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 65/0 onto dev r1z1-10.1.1.1/sdc Placed 65/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 65/1 onto dev r1z1-10.1.1.1/sdd Placed 65/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 65/2 onto dev r1z1-10.1.1.1/sda Placed 65/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 229/0 onto dev r1z1-10.1.1.1/sdb Placed 229/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 229/1 onto dev r1z1-10.1.1.1/sdc Placed 229/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 229/2 onto dev r1z1-10.1.1.1/sdd Placed 229/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 167/0 onto dev r1z1-10.1.1.1/sda Placed 167/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 167/1 onto dev r1z1-10.1.1.1/sdb Placed 167/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 167/2 onto dev r1z1-10.1.1.1/sdc Placed 167/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 1/0 onto dev r1z1-10.1.1.1/sdd Placed 1/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 1/1 onto dev r1z1-10.1.1.1/sda Placed 1/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 1/2 onto dev r1z1-10.1.1.1/sdb Placed 1/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 21/0 onto dev r1z1-10.1.1.1/sdc Placed 21/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 21/1 onto dev r1z1-10.1.1.1/sdd Placed 21/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 21/2 onto dev r1z1-10.1.1.1/sda Placed 21/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 130/0 onto dev r1z1-10.1.1.1/sdb Placed 130/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 130/1 onto dev r1z1-10.1.1.1/sdc Placed 130/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 130/2 onto dev r1z1-10.1.1.1/sdd Placed 130/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 171/0 onto dev r1z1-10.1.1.1/sda Placed 171/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 171/1 onto dev r1z1-10.1.1.1/sdb Placed 171/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 171/2 onto dev r1z1-10.1.1.1/sdc Placed 171/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 214/0 onto dev r1z1-10.1.1.1/sdd Placed 214/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 214/1 onto dev r1z1-10.1.1.1/sda Placed 214/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 214/2 onto dev r1z1-10.1.1.1/sdb Placed 214/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 61/0 onto dev r1z1-10.1.1.1/sdc Placed 61/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 61/1 onto dev r1z1-10.1.1.1/sdd Placed 61/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 61/2 onto dev r1z1-10.1.1.1/sda Placed 61/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 122/0 onto dev r1z1-10.1.1.1/sdb Placed 122/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 122/1 onto dev r1z1-10.1.1.1/sdc Placed 122/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 122/2 onto dev r1z1-10.1.1.1/sdd Placed 122/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 218/0 onto dev r1z1-10.1.1.1/sda Placed 218/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 218/1 onto dev r1z1-10.1.1.1/sdb Placed 218/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 218/2 onto dev r1z1-10.1.1.1/sdc Placed 218/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 63/0 onto dev r1z1-10.1.1.1/sdd Placed 63/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 63/1 onto dev r1z1-10.1.1.1/sda Placed 63/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 63/2 onto dev r1z1-10.1.1.1/sdb Placed 63/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 109/0 onto dev r1z1-10.1.1.1/sdc Placed 109/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 109/1 onto dev r1z1-10.1.1.1/sdd Placed 109/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 109/2 onto dev r1z1-10.1.1.1/sda Placed 109/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 95/0 onto dev r1z1-10.1.1.1/sdb Placed 95/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 95/1 onto dev r1z1-10.1.1.1/sdc Placed 95/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 95/2 onto dev r1z1-10.1.1.1/sdd Placed 95/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 168/0 onto dev r1z1-10.1.1.1/sda Placed 168/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 168/1 onto dev r1z1-10.1.1.1/sdb Placed 168/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 168/2 onto dev r1z1-10.1.1.1/sdc Placed 168/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 140/0 onto dev r1z1-10.1.1.1/sdd Placed 140/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 140/1 onto dev r1z1-10.1.1.1/sda Placed 140/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 140/2 onto dev r1z1-10.1.1.1/sdb Placed 140/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 144/0 onto dev r1z1-10.1.1.1/sdc Placed 144/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 144/1 onto dev r1z1-10.1.1.1/sdd Placed 144/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 144/2 onto dev r1z1-10.1.1.1/sda Placed 144/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 219/0 onto dev r1z1-10.1.1.1/sdb Placed 219/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 219/1 onto dev r1z1-10.1.1.1/sdc Placed 219/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 219/2 onto dev r1z1-10.1.1.1/sdd Placed 219/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 205/0 onto dev r1z1-10.1.1.1/sda Placed 205/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 205/1 onto dev r1z1-10.1.1.1/sdb Placed 205/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 205/2 onto dev r1z1-10.1.1.1/sdc Placed 205/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 93/0 onto dev r1z1-10.1.1.1/sdd Placed 93/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 93/1 onto dev r1z1-10.1.1.1/sda Placed 93/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 93/2 onto dev r1z1-10.1.1.1/sdb Placed 93/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 240/0 onto dev r1z1-10.1.1.1/sdc Placed 240/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 240/1 onto dev r1z1-10.1.1.1/sdd Placed 240/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 240/2 onto dev r1z1-10.1.1.1/sda Placed 240/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 243/0 onto dev r1z1-10.1.1.1/sdb Placed 243/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 243/1 onto dev r1z1-10.1.1.1/sdc Placed 243/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 243/2 onto dev r1z1-10.1.1.1/sdd Placed 243/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 124/0 onto dev r1z1-10.1.1.1/sda Placed 124/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 124/1 onto dev r1z1-10.1.1.1/sdb Placed 124/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 124/2 onto dev r1z1-10.1.1.1/sdc Placed 124/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 125/0 onto dev r1z1-10.1.1.1/sdd Placed 125/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 125/1 onto dev r1z1-10.1.1.1/sda Placed 125/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 125/2 onto dev r1z1-10.1.1.1/sdb Placed 125/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 148/0 onto dev r1z1-10.1.1.1/sdc Placed 148/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 148/1 onto dev r1z1-10.1.1.1/sdd Placed 148/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 148/2 onto dev r1z1-10.1.1.1/sda Placed 148/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 131/0 onto dev r1z1-10.1.1.1/sdb Placed 131/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 131/1 onto dev r1z1-10.1.1.1/sdc Placed 131/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 131/2 onto dev r1z1-10.1.1.1/sdd Placed 131/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 239/0 onto dev r1z1-10.1.1.1/sda Placed 239/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 239/1 onto dev r1z1-10.1.1.1/sdb Placed 239/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 239/2 onto dev r1z1-10.1.1.1/sdc Placed 239/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 150/0 onto dev r1z1-10.1.1.1/sdd Placed 150/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 150/1 onto dev r1z1-10.1.1.1/sda Placed 150/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 150/2 onto dev r1z1-10.1.1.1/sdb Placed 150/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 153/0 onto dev r1z1-10.1.1.1/sdc Placed 153/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 153/1 onto dev r1z1-10.1.1.1/sdd Placed 153/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 153/2 onto dev r1z1-10.1.1.1/sda Placed 153/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 137/0 onto dev r1z1-10.1.1.1/sdb Placed 137/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 137/1 onto dev r1z1-10.1.1.1/sdc Placed 137/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 137/2 onto dev r1z1-10.1.1.1/sdd Placed 137/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 195/0 onto dev r1z1-10.1.1.1/sda Placed 195/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 195/1 onto dev r1z1-10.1.1.1/sdb Placed 195/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 195/2 onto dev r1z1-10.1.1.1/sdc Placed 195/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 231/0 onto dev r1z1-10.1.1.1/sdd Placed 231/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 231/1 onto dev r1z1-10.1.1.1/sda Placed 231/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 231/2 onto dev r1z1-10.1.1.1/sdb Placed 231/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 107/0 onto dev r1z1-10.1.1.1/sdc Placed 107/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 107/1 onto dev r1z1-10.1.1.1/sdd Placed 107/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 107/2 onto dev r1z1-10.1.1.1/sda Placed 107/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 220/0 onto dev r1z1-10.1.1.1/sdb Placed 220/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 220/1 onto dev r1z1-10.1.1.1/sdc Placed 220/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 220/2 onto dev r1z1-10.1.1.1/sdd Placed 220/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 134/0 onto dev r1z1-10.1.1.1/sda Placed 134/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 134/1 onto dev r1z1-10.1.1.1/sdb Placed 134/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 134/2 onto dev r1z1-10.1.1.1/sdc Placed 134/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 15/0 onto dev r1z1-10.1.1.1/sdd Placed 15/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 15/1 onto dev r1z1-10.1.1.1/sda Placed 15/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 15/2 onto dev r1z1-10.1.1.1/sdb Placed 15/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 111/0 onto dev r1z1-10.1.1.1/sdc Placed 111/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 111/1 onto dev r1z1-10.1.1.1/sdd Placed 111/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 111/2 onto dev r1z1-10.1.1.1/sda Placed 111/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 162/0 onto dev r1z1-10.1.1.1/sdb Placed 162/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 162/1 onto dev r1z1-10.1.1.1/sdc Placed 162/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 162/2 onto dev r1z1-10.1.1.1/sdd Placed 162/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 87/0 onto dev r1z1-10.1.1.1/sda Placed 87/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 87/1 onto dev r1z1-10.1.1.1/sdb Placed 87/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 87/2 onto dev r1z1-10.1.1.1/sdc Placed 87/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 58/0 onto dev r1z1-10.1.1.1/sdd Placed 58/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 58/1 onto dev r1z1-10.1.1.1/sda Placed 58/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 58/2 onto dev r1z1-10.1.1.1/sdb Placed 58/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 147/0 onto dev r1z1-10.1.1.1/sdc Placed 147/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 147/1 onto dev r1z1-10.1.1.1/sdd Placed 147/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 147/2 onto dev r1z1-10.1.1.1/sda Placed 147/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 253/0 onto dev r1z1-10.1.1.1/sdb Placed 253/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 253/1 onto dev r1z1-10.1.1.1/sdc Placed 253/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 253/2 onto dev r1z1-10.1.1.1/sdd Placed 253/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 27/0 onto dev r1z1-10.1.1.1/sda Placed 27/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 27/1 onto dev r1z1-10.1.1.1/sdb Placed 27/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 27/2 onto dev r1z1-10.1.1.1/sdc Placed 27/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 14/0 onto dev r1z1-10.1.1.1/sdd Placed 14/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 14/1 onto dev r1z1-10.1.1.1/sda Placed 14/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 14/2 onto dev r1z1-10.1.1.1/sdb Placed 14/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 91/0 onto dev r1z1-10.1.1.1/sdc Placed 91/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 91/1 onto dev r1z1-10.1.1.1/sdd Placed 91/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 91/2 onto dev r1z1-10.1.1.1/sda Placed 91/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 233/0 onto dev r1z1-10.1.1.1/sdb Placed 233/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 233/1 onto dev r1z1-10.1.1.1/sdc Placed 233/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 233/2 onto dev r1z1-10.1.1.1/sdd Placed 233/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 157/0 onto dev r1z1-10.1.1.1/sda Placed 157/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 157/1 onto dev r1z1-10.1.1.1/sdb Placed 157/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 157/2 onto dev r1z1-10.1.1.1/sdc Placed 157/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 127/0 onto dev r1z1-10.1.1.1/sdd Placed 127/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 127/1 onto dev r1z1-10.1.1.1/sda Placed 127/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 127/2 onto dev r1z1-10.1.1.1/sdb Placed 127/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 46/0 onto dev r1z1-10.1.1.1/sdc Placed 46/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 46/1 onto dev r1z1-10.1.1.1/sdd Placed 46/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 46/2 onto dev r1z1-10.1.1.1/sda Placed 46/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 250/0 onto dev r1z1-10.1.1.1/sdb Placed 250/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 250/1 onto dev r1z1-10.1.1.1/sdc Placed 250/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 250/2 onto dev r1z1-10.1.1.1/sdd Placed 250/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 39/0 onto dev r1z1-10.1.1.1/sda Placed 39/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 39/1 onto dev r1z1-10.1.1.1/sdb Placed 39/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 39/2 onto dev r1z1-10.1.1.1/sdc Placed 39/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 166/0 onto dev r1z1-10.1.1.1/sdd Placed 166/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 166/1 onto dev r1z1-10.1.1.1/sda Placed 166/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 166/2 onto dev r1z1-10.1.1.1/sdb Placed 166/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 82/0 onto dev r1z1-10.1.1.1/sdc Placed 82/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 82/1 onto dev r1z1-10.1.1.1/sdd Placed 82/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 82/2 onto dev r1z1-10.1.1.1/sda Placed 82/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 247/0 onto dev r1z1-10.1.1.1/sdb Placed 247/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 247/1 onto dev r1z1-10.1.1.1/sdc Placed 247/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 247/2 onto dev r1z1-10.1.1.1/sdd Placed 247/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 211/0 onto dev r1z1-10.1.1.1/sda Placed 211/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 211/1 onto dev r1z1-10.1.1.1/sdb Placed 211/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 211/2 onto dev r1z1-10.1.1.1/sdc Placed 211/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 94/0 onto dev r1z1-10.1.1.1/sdd Placed 94/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 94/1 onto dev r1z1-10.1.1.1/sda Placed 94/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 94/2 onto dev r1z1-10.1.1.1/sdb Placed 94/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 43/0 onto dev r1z1-10.1.1.1/sdc Placed 43/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 43/1 onto dev r1z1-10.1.1.1/sdd Placed 43/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 43/2 onto dev r1z1-10.1.1.1/sda Placed 43/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 221/0 onto dev r1z1-10.1.1.1/sdb Placed 221/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 221/1 onto dev r1z1-10.1.1.1/sdc Placed 221/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 221/2 onto dev r1z1-10.1.1.1/sdd Placed 221/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 69/0 onto dev r1z1-10.1.1.1/sda Placed 69/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 69/1 onto dev r1z1-10.1.1.1/sdb Placed 69/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 69/2 onto dev r1z1-10.1.1.1/sdc Placed 69/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 179/0 onto dev r1z1-10.1.1.1/sdd Placed 179/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 179/1 onto dev r1z1-10.1.1.1/sda Placed 179/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 179/2 onto dev r1z1-10.1.1.1/sdb Placed 179/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 18/0 onto dev r1z1-10.1.1.1/sdc Placed 18/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 18/1 onto dev r1z1-10.1.1.1/sdd Placed 18/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 18/2 onto dev r1z1-10.1.1.1/sda Placed 18/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 88/0 onto dev r1z1-10.1.1.1/sdb Placed 88/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 88/1 onto dev r1z1-10.1.1.1/sdc Placed 88/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 88/2 onto dev r1z1-10.1.1.1/sdd Placed 88/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 5/0 onto dev r1z1-10.1.1.1/sda Placed 5/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 5/1 onto dev r1z1-10.1.1.1/sdb Placed 5/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 5/2 onto dev r1z1-10.1.1.1/sdc Placed 5/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 44/0 onto dev r1z1-10.1.1.1/sdd Placed 44/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 44/1 onto dev r1z1-10.1.1.1/sda Placed 44/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 44/2 onto dev r1z1-10.1.1.1/sdb Placed 44/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 16/0 onto dev r1z1-10.1.1.1/sdc Placed 16/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 16/1 onto dev r1z1-10.1.1.1/sdd Placed 16/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 16/2 onto dev r1z1-10.1.1.1/sda Placed 16/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 133/0 onto dev r1z1-10.1.1.1/sdb Placed 133/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 133/1 onto dev r1z1-10.1.1.1/sdc Placed 133/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 133/2 onto dev r1z1-10.1.1.1/sdd Placed 133/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 155/0 onto dev r1z1-10.1.1.1/sda Placed 155/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 155/1 onto dev r1z1-10.1.1.1/sdb Placed 155/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 155/2 onto dev r1z1-10.1.1.1/sdc Placed 155/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 188/0 onto dev r1z1-10.1.1.1/sdd Placed 188/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 188/1 onto dev r1z1-10.1.1.1/sda Placed 188/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 188/2 onto dev r1z1-10.1.1.1/sdb Placed 188/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 235/0 onto dev r1z1-10.1.1.1/sdc Placed 235/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 235/1 onto dev r1z1-10.1.1.1/sdd Placed 235/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 235/2 onto dev r1z1-10.1.1.1/sda Placed 235/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 110/0 onto dev r1z1-10.1.1.1/sdb Placed 110/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 110/1 onto dev r1z1-10.1.1.1/sdc Placed 110/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 110/2 onto dev r1z1-10.1.1.1/sdd Placed 110/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 59/0 onto dev r1z1-10.1.1.1/sda Placed 59/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 59/1 onto dev r1z1-10.1.1.1/sdb Placed 59/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 59/2 onto dev r1z1-10.1.1.1/sdc Placed 59/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 192/0 onto dev r1z1-10.1.1.1/sdd Placed 192/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 192/1 onto dev r1z1-10.1.1.1/sda Placed 192/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 192/2 onto dev r1z1-10.1.1.1/sdb Placed 192/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 12/0 onto dev r1z1-10.1.1.1/sdc Placed 12/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 12/1 onto dev r1z1-10.1.1.1/sdd Placed 12/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 12/2 onto dev r1z1-10.1.1.1/sda Placed 12/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 54/0 onto dev r1z1-10.1.1.1/sdb Placed 54/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 54/1 onto dev r1z1-10.1.1.1/sdc Placed 54/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 54/2 onto dev r1z1-10.1.1.1/sdd Placed 54/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 51/0 onto dev r1z1-10.1.1.1/sda Placed 51/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 51/1 onto dev r1z1-10.1.1.1/sdb Placed 51/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 51/2 onto dev r1z1-10.1.1.1/sdc Placed 51/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 113/0 onto dev r1z1-10.1.1.1/sdd Placed 113/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 113/1 onto dev r1z1-10.1.1.1/sda Placed 113/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 113/2 onto dev r1z1-10.1.1.1/sdb Placed 113/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 62/0 onto dev r1z1-10.1.1.1/sdc Placed 62/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 62/1 onto dev r1z1-10.1.1.1/sdd Placed 62/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 62/2 onto dev r1z1-10.1.1.1/sda Placed 62/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 79/0 onto dev r1z1-10.1.1.1/sdb Placed 79/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 79/1 onto dev r1z1-10.1.1.1/sdc Placed 79/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 79/2 onto dev r1z1-10.1.1.1/sdd Placed 79/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 237/0 onto dev r1z1-10.1.1.1/sda Placed 237/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 237/1 onto dev r1z1-10.1.1.1/sdb Placed 237/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 237/2 onto dev r1z1-10.1.1.1/sdc Placed 237/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 201/0 onto dev r1z1-10.1.1.1/sdd Placed 201/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 201/1 onto dev r1z1-10.1.1.1/sda Placed 201/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 201/2 onto dev r1z1-10.1.1.1/sdb Placed 201/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 176/0 onto dev r1z1-10.1.1.1/sdc Placed 176/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 176/1 onto dev r1z1-10.1.1.1/sdd Placed 176/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 176/2 onto dev r1z1-10.1.1.1/sda Placed 176/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 35/0 onto dev r1z1-10.1.1.1/sdb Placed 35/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 35/1 onto dev r1z1-10.1.1.1/sdc Placed 35/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 35/2 onto dev r1z1-10.1.1.1/sdd Placed 35/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 172/0 onto dev r1z1-10.1.1.1/sda Placed 172/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 172/1 onto dev r1z1-10.1.1.1/sdb Placed 172/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 172/2 onto dev r1z1-10.1.1.1/sdc Placed 172/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 165/0 onto dev r1z1-10.1.1.1/sdd Placed 165/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 165/1 onto dev r1z1-10.1.1.1/sda Placed 165/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 165/2 onto dev r1z1-10.1.1.1/sdb Placed 165/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 85/0 onto dev r1z1-10.1.1.1/sdc Placed 85/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 85/1 onto dev r1z1-10.1.1.1/sdd Placed 85/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 85/2 onto dev r1z1-10.1.1.1/sda Placed 85/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 129/0 onto dev r1z1-10.1.1.1/sdb Placed 129/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 129/1 onto dev r1z1-10.1.1.1/sdc Placed 129/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 129/2 onto dev r1z1-10.1.1.1/sdd Placed 129/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 24/0 onto dev r1z1-10.1.1.1/sda Placed 24/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 24/1 onto dev r1z1-10.1.1.1/sdb Placed 24/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 24/2 onto dev r1z1-10.1.1.1/sdc Placed 24/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 154/0 onto dev r1z1-10.1.1.1/sdd Placed 154/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 154/1 onto dev r1z1-10.1.1.1/sda Placed 154/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 154/2 onto dev r1z1-10.1.1.1/sdb Placed 154/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 161/0 onto dev r1z1-10.1.1.1/sdc Placed 161/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 161/1 onto dev r1z1-10.1.1.1/sdd Placed 161/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 161/2 onto dev r1z1-10.1.1.1/sda Placed 161/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 242/0 onto dev r1z1-10.1.1.1/sdb Placed 242/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 242/1 onto dev r1z1-10.1.1.1/sdc Placed 242/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 242/2 onto dev r1z1-10.1.1.1/sdd Placed 242/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 160/0 onto dev r1z1-10.1.1.1/sda Placed 160/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 160/1 onto dev r1z1-10.1.1.1/sdb Placed 160/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 160/2 onto dev r1z1-10.1.1.1/sdc Placed 160/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 117/0 onto dev r1z1-10.1.1.1/sdd Placed 117/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 117/1 onto dev r1z1-10.1.1.1/sda Placed 117/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 117/2 onto dev r1z1-10.1.1.1/sdb Placed 117/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 216/0 onto dev r1z1-10.1.1.1/sdc Placed 216/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 216/1 onto dev r1z1-10.1.1.1/sdd Placed 216/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 216/2 onto dev r1z1-10.1.1.1/sda Placed 216/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 96/0 onto dev r1z1-10.1.1.1/sdb Placed 96/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 96/1 onto dev r1z1-10.1.1.1/sdc Placed 96/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 96/2 onto dev r1z1-10.1.1.1/sdd Placed 96/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 202/0 onto dev r1z1-10.1.1.1/sda Placed 202/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 202/1 onto dev r1z1-10.1.1.1/sdb Placed 202/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 202/2 onto dev r1z1-10.1.1.1/sdc Placed 202/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Assigned 768 parts Assigned 768 parts DEBUG: Finished rebalance plan after 1 attempts Finished rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 16 (Last start was 190) DEBUG: Gather start is 16 (Last start was 190) Gather start is 16 (Last start was 190) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 16 (Last start was 190) DEBUG: Gather start is 16 (Last start was 190) DEBUG: Gather start is 16 (Last start was 190) Gather start is 16 (Last start was 190) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload ... ok test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose test.unit.cli.test_ringcomposer.TestCommands.test_compose ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show test.unit.cli.test_ringcomposer.TestCommands.test_show ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file ... ok test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type ... ok test.unit.common.test_linkat.TestLinkat.test_available test.unit.common.test_linkat.TestLinkat.test_available ... ok test.unit.common.test_linkat.TestLinkat.test_errno test.unit.common.test_linkat.TestLinkat.test_errno ... ok test.unit.common.test_linkat.TestLinkat.test_flags test.unit.common.test_linkat.TestLinkat.test_flags ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_success test.unit.common.test_linkat.TestLinkat.test_linkat_success ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable test.unit.common.test_linkat.TestLinkat.test_unavailable ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc ... ok test.unit.common.test_manager.TestServer.test_conf_dir test.unit.common.test_manager.TestServer.test_conf_dir ... ok test.unit.common.test_manager.TestServer.test_conf_files test.unit.common.test_manager.TestServer.test_conf_files ... ok test.unit.common.test_manager.TestServer.test_create_server test.unit.common.test_manager.TestServer.test_create_server ... ok test.unit.common.test_manager.TestServer.test_expirer_conf_files test.unit.common.test_manager.TestServer.test_expirer_conf_files ... ok test.unit.common.test_manager.TestServer.test_get_conf_file_name test.unit.common.test_manager.TestServer.test_get_conf_file_name ... ok test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_pid_file_name test.unit.common.test_manager.TestServer.test_get_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_running_pids test.unit.common.test_manager.TestServer.test_get_running_pids ... ok test.unit.common.test_manager.TestServer.test_interact test.unit.common.test_manager.TestServer.test_interact ... ok test.unit.common.test_manager.TestServer.test_iter_pid_files test.unit.common.test_manager.TestServer.test_iter_pid_files ... ok test.unit.common.test_manager.TestServer.test_kill_running_pids test.unit.common.test_manager.TestServer.test_kill_running_pids ... ok test.unit.common.test_manager.TestServer.test_launch test.unit.common.test_manager.TestServer.test_launch ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir test.unit.common.test_manager.TestServer.test_named_conf_dir ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files ... ok test.unit.common.test_manager.TestServer.test_proxy_conf_dir test.unit.common.test_manager.TestServer.test_proxy_conf_dir ... ok test.unit.common.test_manager.TestServer.test_server_equality test.unit.common.test_manager.TestServer.test_server_equality ... ok test.unit.common.test_manager.TestServer.test_server_repr test.unit.common.test_manager.TestServer.test_server_repr ... ok test.unit.common.test_manager.TestServer.test_server_to_string test.unit.common.test_manager.TestServer.test_server_to_string ... ok test.unit.common.test_manager.TestServer.test_signal_pids test.unit.common.test_manager.TestServer.test_signal_pids ... ok test.unit.common.test_manager.TestServer.test_spawn test.unit.common.test_manager.TestServer.test_spawn ... ok test.unit.common.test_manager.TestServer.test_status test.unit.common.test_manager.TestServer.test_status ... ok test.unit.common.test_manager.TestServer.test_stop test.unit.common.test_manager.TestServer.test_stop ... ok test.unit.common.test_manager.TestServer.test_wait test.unit.common.test_manager.TestServer.test_wait ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_init test.unit.common.test_memcached.TestMemcacheCommand.test_init ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_filters test.unit.common.test_utils.TestAuditLocationGenerator.test_filters ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_reserved test.unit.common.test_utils.TestFallocateReserve.test_all_reserved ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_disabled test.unit.common.test_utils.TestFallocateReserve.test_disabled ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space test.unit.common.test_utils.TestFallocateReserve.test_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve ... ok test.unit.common.test_utils.TestFileLikeIter.test_close test.unit.common.test_utils.TestFileLikeIter.test_close ... ok test.unit.common.test_utils.TestFileLikeIter.test_get_hub test.unit.common.test_utils.TestFileLikeIter.test_get_hub ... ok test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter ... ok test.unit.common.test_utils.TestFileLikeIter.test_next test.unit.common.test_utils.TestFileLikeIter.test_next ... ok test.unit.common.test_utils.TestFileLikeIter.test_read test.unit.common.test_utils.TestFileLikeIter.test_read ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size test.unit.common.test_utils.TestFileLikeIter.test_read_with_size ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline test.unit.common.test_utils.TestFileLikeIter.test_readline ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline2 test.unit.common.test_utils.TestFileLikeIter.test_readline2 ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline3 test.unit.common.test_utils.TestFileLikeIter.test_readline3 ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines test.unit.common.test_utils.TestFileLikeIter.test_readlines ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking test.unit.common.test_utils.TestPipeMutex.test_blocking ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool ... ok test.unit.common.test_utils.TestPipeMutex.test_nonblocking test.unit.common.test_utils.TestPipeMutex.test_nonblocking ... ok test.unit.common.test_utils.TestPipeMutex.test_recursive test.unit.common.test_utils.TestPipeMutex.test_recursive ... ok test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire ... ok test.unit.common.test_utils.TestPipeMutex.test_too_many_releases test.unit.common.test_utils.TestPipeMutex.test_too_many_releases ... ok test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted ... ok test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0526 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0498 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0210 "-" 3610586 - test INFO: - - - [02/Jun/2024:16:48:57 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0310 "-" 3610586 - test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'Content-Md5': '0psj556+oeNaquEdDeMxig==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd29b23e79ebea1e35aaae11d0de3318a', 'Content-Length': '226', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys ... ok proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:49:05] "GET /someurl HTTP/1.0" 200 180 0.000053 proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:49:05] "GET /someurl HTTP/1.0" 200 180 0.000048 proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True proxy INFO: (3610691) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610691) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:49:05] "GET /someurl HTTP/1.0" 200 180 0.000070 proxy INFO: wsgi exiting proxy INFO: (3610691) wsgi exited, is_accepting=True test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0141 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0160 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0021 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0020 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0089 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0226 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0073 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0198 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0263 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0156 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0115 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0085 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:05 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0135 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3610691 0 test ERROR: ERROR Account update failed with 127.0.0.1:41161/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0149 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0170 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0106 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0092 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0290 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0115 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0316 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0028 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0204 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0268 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0029 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0029 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0031 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0026 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0014 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0024 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0207 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0063 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0181 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0129 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0017 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0026 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0152 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0017 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0092 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0022 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0130 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0025 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0013 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0025 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0018 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:06 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0023 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0031 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0027 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0110 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0018 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0054 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0029 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0023 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0141 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0027 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0019 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0019 "-" 3610691 - test DEBUG: Unlinked retiring db '/tmp/tmp3sghfxod/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0202 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0023 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0160 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0071 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestContainerController.test_GET_auto_record_type test.unit.container.test_server.TestContainerController.test_GET_auto_record_type ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter test.unit.container.test_server.TestContainerController.test_GET_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim ... ok test ERROR: Failed to unlink '/tmp/tmp_qydr7mj/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp_qydr7mj/containers/part/suffix/hash/container_1717346970.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp_qydr7mj/containers/part/suffix/hash/container_1717346970.00000.db': ['/tmp/tmp_qydr7mj/containers/part/suffix/hash/container.db', '/tmp/tmp_qydr7mj/containers/part/suffix/hash/container_1717346971.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp_qydr7mj/containers/part/suffix/hash/container_1717346971.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717346971.6986582, time window end: 1717346971.6986582, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 5, sync_point2: None, total_rows: 1 test-container-sync ERROR: ERROR Syncing : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 388, in container_sync if not self.container_sync_row( ^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: TestContainerSync.test_container_report..fake_container_sync_row() missing 1 required positional argument: 'realm_key' test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717346971.70193, time window end: 1717346971.7019403, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync ERROR: ERROR Syncing {'ROWID': 1, 'name': 'o'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 534, in container_sync_row row['created_at']) ~~~^^^^^^^^^^^^^^ KeyError: 'created_at' test-container-sync INFO: Container sync report: a/c, time window start: 1717346971.702116, time window end: 1717346971.7024825, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync INFO: Container sync report: a/c, time window start: 1717346971.7026358, time window end: 1717346971.702712, puts: 0, posts: 0, deletes: 1, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms ... ok test.unit.container.test_sync.TestContainerSync.test_run_forever test.unit.container.test_sync.TestContainerSync.test_run_forever ... ok test.unit.container.test_sync.TestContainerSync.test_run_once test.unit.container.test_sync.TestContainerSync.test_run_once ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one ... ok test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init test.unit.obj.test_diskfile.TestDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_base.TestGetterSource.test_close test.unit.proxy.controllers.test_base.TestGetterSource.test_close ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_init test.unit.proxy.controllers.test_base.TestGetterSource.test_init ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_sort test.unit.proxy.controllers.test_base.TestGetterSource.test_sort ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit "once" mode test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmp1_mfv2nf/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmp1_mfv2nf/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit "once" mode completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx4541df3e545947db8609a-00665ca29b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: ValueError("upper ('a') must be greater than or equal to lower ('z')") (txn: txe01dc15002784b6b93818-00665ca29b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: KeyError('name') (txn: txe072ce82939b4b7db806d-00665ca29b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type None (txn: tx7d948685ff9a4803912df-00665ca29b) test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission ... ok swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. Reassigned 768 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (0:59:59 remaining) Reassigned 192 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (0:24:00 remaining) No partitions could be reassigned. There is no need to do so at this time Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 1000.0 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) ------------------------------------------------------------------------------- An error has occurred during ring validation. Common causes of failure are rings that are empty or do not have enough devices to accommodate the replica count. Original exception message: There are no devices in this ring, or all devices have been deleted ------------------------------------------------------------------------------- Partition power increase in progress. You need to finish the increase first before rebalancing. Reassigned 0 (0.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 20 (31.25%) partitions. Balance is now 24.48. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 24.48 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 33.33 ------------------------------------------------------------------------------- NOTE: Dispersion of 33.333333 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 16.67 ------------------------------------------------------------------------------- NOTE: Dispersion of 16.666667 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. test.unit.cli.test_ringbuilder.TestCommands.test_remove_device test.unit.cli.test_ringbuilder.TestCommands.test_remove_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Not a path query: /, skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Cached creds valid test DEBUG: Connecting with tenant: tenant_id test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Cached keystone credentials test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test ERROR: Invalid current //REALM/CLUSTER (foo) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) user quit No .MockServer object at 0x7f67a7db4ec0> running .MockServerFactory.MockServer object at 0x7f67a88d6960> (1) appears to have stopped No .MockServerFactory.MockServer object at 0x7f67a7db4fe0> running No .MockServerFactory.MockServer object at 0x7f67a7db4140> running Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f67a7db4fe0> to die; giving up Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f67a7cd0050> to die; killing Signal .MockServerFactory.MockServer object at 0x7f67a7cd0050> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f67a7db4140> to die; killing Signal .MockServerFactory.MockServer object at 0x7f67a7db4140> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7f67a7db45c0> to die; killing Signal .MockServerFactory.MockServer object at 0x7f67a7db45c0> pid: None signal: 9 test WARNING: More than one part in a single-part response? sharder-test WARNING: Reclaimable db stuck waiting for shrinking, path: a/c, db: /tmp/tmp7mhuha4q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txc0a204d10301450aa6f0e-00665ca28d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txc0a204d10301450aa6f0e-00665ca28d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc0a204d10301450aa6f0e-00665ca28d) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc0a204d10301450aa6f0e-00665ca28d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc0a204d10301450aa6f0e-00665ca28d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Setting object count to 1065 (txn: txd3698bed500e49c5ab071-00665ca297) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Setting object count to 1065 (txn: txa3323e79cbfd4bb5ad293-00665ca29a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7e87ba8edc4446c68a74e-00665ca29d) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: tx7e87ba8edc4446c68a74e-00665ca29d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: txbe58fc3d2eaa416491103-00665ca2a0) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error ... ok test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test DEBUG: Update failed for /a/c/o /tmp/tmpyb8q491u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.88216 test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.08s seconds:, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.08s, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test DEBUG: Error code 500 is returned from remote server 127.0.0.1:34269/sda1 test DEBUG: Error code 500 is returned from remote server 127.0.0.1:34269/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpyb8q491u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.88216 test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.02s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test DEBUG: Error code 404 is returned from remote server 127.0.0.1:34269/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpyb8q491u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.88216 test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.02s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test INFO: Timeout waiting on remote server 127.0.0.1:34269/sda1: 99 seconds test DEBUG: Update failed for /a/c/o /tmp/tmpyb8q491u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.88216 test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.00s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test INFO: Timeout connecting to remote server 127.0.0.1:34269/sda1: 9 seconds test DEBUG: Update failed for /a/c/o /tmp/tmpyb8q491u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.88216 test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.00s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpyb8q491u/devices/sda1 (pid: 3610687) test DEBUG: Update sent for /a/c/o /tmp/tmpyb8q491u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717346946.88216 test INFO: Object update sweep completed on /tmp/tmpyb8q491u/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test WARNING: Skipping: test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Object update sweep starting on /tmp/tmp1xqmefyd/devices/sda1 (pid: 3610687) test INFO: Object update sweep progress on /tmp/tmp1xqmefyd/devices/sda1: 16.00s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update sweep progress on /tmp/tmp1xqmefyd/devices/sda1: 32.00s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update sweep completed on /tmp/tmp1xqmefyd/devices/sda1 in 44.00s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test INFO: Object update sweep starting on /tmp/tmpfd8kef33/devices/sda1 (pid: 3610687) test INFO: Object update sweep completed on /tmp/tmpfd8kef33/devices/sda1 in 0.13s seconds:, 6 successes, 0 failures, 0 quarantines, 6 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3610687) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission ... ok test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0109 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0024 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0126 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0120 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0157 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0023 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0038 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0022 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0181 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0243 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0004 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0269 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0297 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0054 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0046 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0101 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:07 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0044 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0058 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0055 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0043 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0042 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0050 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0052 "-" 3610691 0 test DEBUG: Unlinked retiring db '/tmp/tmppq_mwxj_/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0090 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0041 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0051 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0051 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0011 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0183 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0025 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0142 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0044 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0178 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0279 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestContainerController.test_GET_invalid_accept test.unit.container.test_server.TestContainerController.test_GET_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_GET_json test.unit.container.test_server.TestContainerController.test_GET_json ... ok test.unit.container.test_server.TestContainerController.test_GET_json_last_modified test.unit.container.test_server.TestContainerController.test_GET_json_last_modified ... ok test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_limit test.unit.container.test_server.TestContainerController.test_GET_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_false test.unit.container.test_server.TestContainerController.test_GET_log_requests_false ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_true test.unit.container.test_server.TestContainerController.test_GET_log_requests_true ... ok test.unit.container.test_server.TestContainerController.test_GET_marker test.unit.container.test_server.TestContainerController.test_GET_marker ... ok test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter ... ok test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmp_44cngny/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmp_44cngny/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/c /tmp/tmp_44cngny/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test WARNING: /tmp/tmp70ngcnk5/tmp_test_container_updater/devices/sda1 is not mounted test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Failed to get container info (Lock timeout: 10 seconds: /tmp/tmpb_kvu7ii/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db); skipping. test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpcw87u9hc/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpcw87u9hc/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpcw87u9hc/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test.unit.container.test_updater.TestContainerUpdater.test_shard_container test.unit.container.test_updater.TestContainerUpdater.test_shard_container ... ok test.unit.container.test_updater.TestContainerUpdater.test_unicode test.unit.container.test_updater.TestContainerUpdater.test_unicode ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestReplicatorStats.test_add test.unit.obj.test_replicator.TestReplicatorStats.test_add ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon ... ok test.unit.obj.test_server.TestObjectController.test_DELETE test.unit.obj.test_server.TestObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET test.unit.obj.test_server.TestObjectController.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_but_expired test.unit.obj.test_server.TestObjectController.test_GET_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match test.unit.obj.test_server.TestObjectController.test_GET_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_none_match test.unit.obj.test_server.TestObjectController.test_GET_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine test.unit.obj.test_server.TestObjectController.test_GET_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD test.unit.obj.test_server.TestObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_OPTIONS test.unit.obj.test_server.TestObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_but_expired test.unit.obj.test_server.TestObjectController.test_POST_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_connection test.unit.obj.test_server.TestObjectController.test_POST_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_invalid_path test.unit.obj.test_server.TestObjectController.test_POST_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_not_exist test.unit.obj.test_server.TestObjectController.test_POST_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_system_metadata test.unit.obj.test_server.TestObjectController.test_POST_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_update_meta test.unit.obj.test_server.TestObjectController.test_POST_update_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim ... ok test.unit.obj.test_server.TestObjectController.test_PUT_commits_data test.unit.obj.test_server.TestObjectController.test_PUT_commits_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_common test.unit.obj.test_server.TestObjectController.test_PUT_common ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_connection test.unit.obj.test_server.TestObjectController.test_PUT_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_etag test.unit.obj.test_server.TestObjectController.test_PUT_no_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite test.unit.obj.test_server.TestObjectController.test_PUT_overwrite ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_slow test.unit.obj.test_server.TestObjectController.test_PUT_slow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_works test.unit.obj.test_server.TestObjectController.test_REPLICATE_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test__call__returns_500 test.unit.obj.test_server.TestObjectController.test__call__returns_500 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_allowed_headers test.unit.obj.test_server.TestObjectController.test_allowed_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_http_connect test.unit.obj.test_server.TestObjectController.test_async_update_http_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_bad_sinces test.unit.obj.test_server.TestObjectController.test_bad_sinces ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_method test.unit.obj.test_server.TestObjectController.test_call_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_request test.unit.obj.test_server.TestObjectController.test_call_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_name_collision test.unit.obj.test_server.TestObjectController.test_call_name_collision ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_not_found test.unit.obj.test_server.TestObjectController.test_call_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_put test.unit.obj.test_server.TestObjectController.test_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_async test.unit.obj.test_server.TestObjectController.test_container_update_async ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_bad_args test.unit.obj.test_server.TestObjectController.test_container_update_bad_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_success test.unit.obj.test_server.TestObjectController.test_container_update_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_content_encoding test.unit.obj.test_server.TestObjectController.test_content_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_correct_allowed_method test.unit.obj.test_server.TestObjectController.test_correct_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_cap test.unit.obj.test_server.TestObjectController.test_delete_at_cap ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_negative test.unit.obj.test_server.TestObjectController.test_delete_at_negative ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_backend_replication test.unit.obj.test_server.TestObjectController.test_delete_backend_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_device_unavailable test.unit.obj.test_server.TestObjectController.test_device_unavailable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_dynamic_datadir test.unit.obj.test_server.TestObjectController.test_dynamic_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_init test.unit.obj.test_server.TestObjectController.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_list_allowed_methods test.unit.obj.test_server.TestObjectController.test_list_allowed_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_log_line_format test.unit.obj.test_server.TestObjectController.test_log_line_format ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_object_name_length test.unit.obj.test_server.TestObjectController.test_max_object_name_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_upload_time test.unit.obj.test_server.TestObjectController.test_max_upload_time ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_allowed_method test.unit.obj.test_server.TestObjectController.test_not_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_update_with_offset test.unit.obj.test_server.TestObjectController.test_object_update_with_offset ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_serv_reserv test.unit.obj.test_server.TestObjectController.test_serv_reserv ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_short_body test.unit.obj.test_server.TestObjectController.test_short_body ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file ... ok d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" with 3.14159265359 weight got id 4 d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Partition power increase in progress. You need to finish the increase first before removing devices. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 test.test.com 6200 r.test.com 7000 sda3 3.14 2 33.67 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data No matching devices found swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info test.unit.cli.test_ringbuilder.TestCommands.test_set_info ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices ... ok swift: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c" 507 - "-" "-" "-" 0.0036 "-" 3610691 - swift: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda-null/p/a/c" 400 75 "-" "-" "-" 0.0003 "-" 3610691 - test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: ValueError('not a list') (txn: tx5df341f7e7f54cd985623-00665ca2a1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Failed to get container listing from /v1/a/c/o: 404 (txn: txb6c12a1aa37c46e2bd569-00665ca2a1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type 'object' (txn: txf70bfdb2006e417ab5b57-00665ca2a1) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpwzawg04_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpwzawg04_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpwzawg04_/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpwzawg04_/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:37 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0115 "-" 3610546 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:37 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0147 "-" 3610546 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:37 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0102 "-" 3610546 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0140 "-" 3610546 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txafc837eaf13d452c93ab9-00665ca2a2" "proxy-server 3610546" 0.0018 "-" 3610546 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:45819/sda1/1/a/c" "txafc837eaf13d452c93ab9-00665ca2a2" "container-server 3610546" 0.0036 "-" 3610546 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:59795/sdb1/1/a/c" "txafc837eaf13d452c93ab9-00665ca2a2" "container-server 3610546" 0.0014 "-" 3610546 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txafc837eaf13d452c93ab9-00665ca2a2" "proxy-server 3610546" 0.0232 "-" 3610546 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txafc837eaf13d452c93ab9-00665ca2a2" "proxy-server 3610546" 0.0493 "-" 3610546 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/38 PUT /v1/a/c HTTP/1.0 201 - - t - - - txafc837eaf13d452c93ab9-00665ca2a2 - 0.0596 - - 1717346978.002948761 1717346978.062521696 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx89acc55055bd4cde80131-00665ca2a2" "proxy-server 3610546" 0.0015 "-" 3610546 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:59795/sdb1/1/a1/c1" "tx89acc55055bd4cde80131-00665ca2a2" "container-server 3610546" 0.0016 "-" 3610546 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:45819/sda1/1/a1/c1" "tx89acc55055bd4cde80131-00665ca2a2" "container-server 3610546" 0.0014 "-" 3610546 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx89acc55055bd4cde80131-00665ca2a2" "proxy-server 3610546" 0.0264 "-" 3610546 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx89acc55055bd4cde80131-00665ca2a2" "proxy-server 3610546" 0.0560 "-" 3610546 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/38 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx89acc55055bd4cde80131-00665ca2a2 - 0.0658 - - 1717346978.064413071 1717346978.130174160 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb7e3a4d8444145089c269-00665ca2a2" "proxy-server 3610546" 0.0142 "-" 3610546 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:45819/sda1/2/a/c1" "txb7e3a4d8444145089c269-00665ca2a2" "container-server 3610546" 0.0043 "-" 3610546 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59795/sdb1/2/a/c1" "txb7e3a4d8444145089c269-00665ca2a2" "container-server 3610546" 0.0145 "-" 3610546 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb7e3a4d8444145089c269-00665ca2a2" "proxy-server 3610546" 0.0636 "-" 3610546 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb7e3a4d8444145089c269-00665ca2a2" "proxy-server 3610546" 0.0489 "-" 3610546 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/38 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txb7e3a4d8444145089c269-00665ca2a2 - 0.0914 - - 1717346978.131989956 1717346978.223423958 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx122d284daba34a3fa7ec7-00665ca2a2" "proxy-server 3610546" 0.0135 "-" 3610546 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:59795/sdb1/1/a/c2" "tx122d284daba34a3fa7ec7-00665ca2a2" "container-server 3610546" 0.0117 "-" 3610546 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:45819/sda1/1/a/c2" "tx122d284daba34a3fa7ec7-00665ca2a2" "container-server 3610546" 0.0026 "-" 3610546 2 test.unit.proxy.test_server.TestAccountController.test_DELETE test.unit.proxy.test_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_server.TestAccountController.test_GET test.unit.proxy.test_server.TestAccountController.test_GET ... ok test.unit.proxy.test_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD test.unit.proxy.test_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_OPTIONS test.unit.proxy.test_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_POST_metadata test.unit.proxy.test_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT test.unit.proxy.test_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_status test.unit.proxy.test_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_server.TestAccountController.test_connection_refused test.unit.proxy.test_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_server.TestAccountController.test_other_socket_error test.unit.proxy.test_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx122d284daba34a3fa7ec7-00665ca2a2" "proxy-server 3610546" 0.0388 "-" 3610546 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:38 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx122d284daba34a3fa7ec7-00665ca2a2" "proxy-server 3610546" 0.0682 "-" 3610546 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/38 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx122d284daba34a3fa7ec7-00665ca2a2 - 0.0967 - - 1717346978.226836681 1717346978.323529243 2 test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test NOTICE: Closing unnecessary sock for port 6006 (child pid 88) test NOTICE: Closing unnecessary sock for port 6006 (child pid 89) test NOTICE: Closing unnecessary sock for port 6006 (child pid 90) test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test NOTICE: Started child 0 (PID 94) for port 6009 test NOTICE: Started child 1 (PID 95) for port 6009 test NOTICE: Started child 2 (PID 96) for port 6009 test NOTICE: Started child 1 (PID 97) for port 6009 test NOTICE: Child 2 (PID 3610637, port 6007) exiting normally test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test ERROR: Problem finding shard upper in '/tmp/tmpxlo0045i/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpxlo0045i/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpg7wiugn0/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpg7wiugn0/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'Content-Md5': 'i4CS9WiTvEfCFGqAHP8j0A==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8b8092f56893bc47c2146a801cff23d0', 'Content-Length': '1030946'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf7e0aa3dc22240f7a3999-00665ca296) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf7e0aa3dc22240f7a3999-00665ca296) test.unit.proxy.controllers.test_base.TestFuncs.test_options test.unit.proxy.controllers.test_base.TestFuncs.test_options ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object ... ok test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpx_d7vvok/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpx_d7vvok/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpx_d7vvok/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpx_d7vvok/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0157 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0110 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0130 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0139 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx389169bcb9484cb289501-00665ca29b" "proxy-server 3610570" 0.0020 "-" 3610570 - test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE test.unit.proxy.test_mem_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you ... ok test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0158 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0176 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0165 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0041 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0042 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0043 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0289 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0164 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0183 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0017 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0149 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0328 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0044 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0144 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0056 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0204 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0048 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0187 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0047 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0043 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0148 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0168 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0044 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0039 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0115 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_non_ascii test.unit.container.test_server.TestContainerController.test_GET_non_ascii ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/Ω /tmp/tmpwx7f9h2e/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.04s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txab10889e369e410a9fcaf-00665ca2ad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"something" Device d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" is now d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"something" Device d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" is now d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"something" Device d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" is now d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"something" Device 1 already uses 127.0.0.2:6201/sda2. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" is now d4r2z3-test.test2.com:9000Rr.test2.com:9000/sda300_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.2.1:9000R127.0.2.1:9000/sda100_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.1.1:8000/sda10_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[4001:0:1234::c1c0:abcd:876]:9000R[4::10]:9000/sda300_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:8000/sda30_"other meta data" Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The minimum number of hours before a partition can be reassigned is now set to 24 test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload test.unit.cli.test_ringbuilder.TestCommands.test_set_overload ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region test.unit.cli.test_ringbuilder.TestCommands.test_set_region ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight test.unit.cli.test_ringbuilder.TestCommands.test_set_weight ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock ... ok test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: txbe58fc3d2eaa416491103-00665ca2a0) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txbe58fc3d2eaa416491103-00665ca2a0) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: txbe58fc3d2eaa416491103-00665ca2a0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx951d988c3a194a89aca84-00665ca2a0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx951d988c3a194a89aca84-00665ca2a0) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx951d988c3a194a89aca84-00665ca2a0) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: tx951d988c3a194a89aca84-00665ca2a0) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx6a5ecb1e35e7432f82dc1-00665ca2a0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6a5ecb1e35e7432f82dc1-00665ca2a0) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6a5ecb1e35e7432f82dc1-00665ca2a0) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx6a5ecb1e35e7432f82dc1-00665ca2a0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: Setting object count to 3120 (txn: txcb17a11531fd4e5eac407-00665ca2a3) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: Setting object count to 3120 (txn: txf78f82aeddde44e290782-00665ca2a3) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: Setting object count to 3120 (txn: tx7130bc5e27dc482e9c9a5-00665ca2a3) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: Setting object count to 3120 (txn: txca0e4e47fb3b4b19ba3cb-00665ca2a4) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: Setting object count to 3120 (txn: txec54a93246ff4919832c4-00665ca2a4) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx009ee32fd7f04622af802-00665ca2a4) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache ... ok test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf7e0aa3dc22240f7a3999-00665ca296) test ERROR: Container HEAD returning 503 for [] (txn: txf7e0aa3dc22240f7a3999-00665ca296) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf6af0d1c278b402b97749-00665ca296) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf6af0d1c278b402b97749-00665ca296) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf6af0d1c278b402b97749-00665ca296) test ERROR: Container HEAD returning 503 for [] (txn: txf6af0d1c278b402b97749-00665ca296) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpw_k5vvjn/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpw_k5vvjn/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpw_k5vvjn/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpw_k5vvjn/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0589 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0269 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0499 "-" 3610616 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0421 "-" 3610616 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx586defe3a0c74974a7af0-00665ca296" "proxy-server 3610616" 0.0032 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:39449/sdb1/1/a/c" "tx586defe3a0c74974a7af0-00665ca296" "container-server 3610616" 0.0059 "-" 3610616 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:44353/sda1/1/a/c" "tx586defe3a0c74974a7af0-00665ca296" "container-server 3610616" 0.0022 "-" 3610616 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx586defe3a0c74974a7af0-00665ca296" "proxy-server 3610616" 0.0680 "-" 3610616 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx586defe3a0c74974a7af0-00665ca296" "proxy-server 3610616" 0.1019 "-" 3610616 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/26 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx586defe3a0c74974a7af0-00665ca296 - 0.1431 - - 1717346966.774227381 1717346966.917299509 0 test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'Content-Md5': '/JzPJjlXCqpRBmlyhGXNDg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'fc9ccf2639570aaa510669728465cd0e', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'Content-Md5': 'PggRsy85Tu07EsbATYB/cA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '3e0811b32f394eed3b12c6c04d807f70', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'Content-Md5': 'zDBBHq17IpuHzskOm+GHEQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'cc30411ead7b229b87cec90e9be18711', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'Content-Md5': 'hTnOh9sxGnzPiScnD+t0Mg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8539ce87db311a7ccf8927270feb7432', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'Content-Md5': 'mbEtd8ufKpkpCIfMjGD7Tw==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '99b12d77cb9f2a99290887cc8c60fb4f', 'Content-Length': '90'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:54405/sdb1/1/a/c" "tx389169bcb9484cb289501-00665ca29b" "container-server 3610570" 0.0016 "-" 3610570 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:42401/sda1/1/a/c" "tx389169bcb9484cb289501-00665ca29b" "container-server 3610570" 0.0017 "-" 3610570 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx389169bcb9484cb289501-00665ca29b" "proxy-server 3610570" 0.0253 "-" 3610570 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx389169bcb9484cb289501-00665ca29b" "proxy-server 3610570" 0.0400 "-" 3610570 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/31 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx389169bcb9484cb289501-00665ca29b - 0.0515 - - 1717346971.914544344 1717346971.966085911 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:31 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx5f8b79fd4758406fa1986-00665ca29b" "proxy-server 3610570" 0.0017 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:54405/sdb1/1/a1/c1" "tx5f8b79fd4758406fa1986-00665ca29b" "container-server 3610570" 0.0031 "-" 3610570 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:42401/sda1/1/a1/c1" "tx5f8b79fd4758406fa1986-00665ca29b" "container-server 3610570" 0.0023 "-" 3610570 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx5f8b79fd4758406fa1986-00665ca29b" "proxy-server 3610570" 0.0589 "-" 3610570 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx5f8b79fd4758406fa1986-00665ca29b" "proxy-server 3610570" 0.0456 "-" 3610570 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/32 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx5f8b79fd4758406fa1986-00665ca29b - 0.0721 - - 1717346971.968192101 1717346972.040266275 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb7c83e5d5bc742e9b2aac-00665ca29c" "proxy-server 3610570" 0.0172 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:42401/sda1/2/a/c1" "txb7c83e5d5bc742e9b2aac-00665ca29c" "container-server 3610570" 0.0149 "-" 3610570 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:54405/sdb1/2/a/c1" "txb7c83e5d5bc742e9b2aac-00665ca29c" "container-server 3610570" 0.0021 "-" 3610570 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb7c83e5d5bc742e9b2aac-00665ca29c" "proxy-server 3610570" 0.0386 "-" 3610570 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txb7c83e5d5bc742e9b2aac-00665ca29c" "proxy-server 3610570" 0.0573 "-" 3610570 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/32 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txb7c83e5d5bc742e9b2aac-00665ca29c - 0.0871 - - 1717346972.043115854 1717346972.130254507 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdfd61a465bcc4b288eca3-00665ca29c" "proxy-server 3610570" 0.0116 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:54405/sdb1/1/a/c2" "txdfd61a465bcc4b288eca3-00665ca29c" "container-server 3610570" 0.0131 "-" 3610570 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:42401/sda1/1/a/c2" "txdfd61a465bcc4b288eca3-00665ca29c" "container-server 3610570" 0.0020 "-" 3610570 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txdfd61a465bcc4b288eca3-00665ca29c" "proxy-server 3610570" 0.0386 "-" 3610570 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:32 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txdfd61a465bcc4b288eca3-00665ca29c" "proxy-server 3610570" 0.0599 "-" 3610570 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/32 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txdfd61a465bcc4b288eca3-00665ca29c - 0.0834 - - 1717346972.132978678 1717346972.216424465 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx16d1e1a28088415c9414f-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx8ac58977e3bc4702a147d-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: txd83642afaf0f434f8adb0-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: txd83642afaf0f434f8adb0-00665ca29c) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txd83642afaf0f434f8adb0-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx135278d5c8364224a9afe-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx135278d5c8364224a9afe-00665ca29c) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx135278d5c8364224a9afe-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: tx39a4f312ddf344de835ae-00665ca29c) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx39a4f312ddf344de835ae-00665ca29c) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txc32c73a3770f473a88afd-00665ca29c) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txc32c73a3770f473a88afd-00665ca29c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST test.unit.proxy.test_mem_server.TestContainerController.test_POST ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. The overload factor is now 19.88% (0.198780) The change will take effect after the next rebalance. Overload must be non-negative. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. 'swedish fish' is not a valid number. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The replica count is now 3.141593. The change will take effect after the next rebalance. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. "test" is not a valid number. Replica count must be at least 1. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 200.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 200.0 d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 200.0 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 200.0 test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone test.unit.cli.test_ringbuilder.TestCommands.test_set_zone ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching ... ok test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0133 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0120 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0144 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0023 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0153 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0136 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0162 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0019 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0317 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0099 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0126 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:08 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0063 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0192 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0267 "-" 3610691 0 test DEBUG: Unlinked retiring db '/tmp/tmpfskzl0cb/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0045 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0039 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0235 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0154 "-" 3610691 0 test DEBUG: Unlinked retiring db '/tmp/tmphjzj4iwr/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0039 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0226 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0084 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0037 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0178 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0251 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0034 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0165 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0250 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0033 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0206 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0219 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0031 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0166 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:09 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0155 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:09 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0043 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestContainerController.test_GET_over_limit test.unit.container.test_server.TestContainerController.test_GET_over_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_path test.unit.container.test_server.TestContainerController.test_GET_path ... ok test.unit.container.test_server.TestContainerController.test_GET_plain test.unit.container.test_server.TestContainerController.test_GET_plain ... ok test.unit.container.test_server.TestContainerController.test_GET_prefix test.unit.container.test_server.TestContainerController.test_GET_prefix ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:26 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx2fa03dd154b14aa6ac9df-00665ca296" "proxy-server 3610616" 0.0021 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:39449/sdb1/1/a1/c1" "tx2fa03dd154b14aa6ac9df-00665ca296" "container-server 3610616" 0.0028 "-" 3610616 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:44353/sda1/1/a1/c1" "tx2fa03dd154b14aa6ac9df-00665ca296" "container-server 3610616" 0.0046 "-" 3610616 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2fa03dd154b14aa6ac9df-00665ca296" "proxy-server 3610616" 0.0396 "-" 3610616 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2fa03dd154b14aa6ac9df-00665ca296" "proxy-server 3610616" 0.0957 "-" 3610616 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx2fa03dd154b14aa6ac9df-00665ca296 - 1.2096 - - 1717346966.920421362 1717346968.130070448 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txca4e77432aed4922bfe7b-00665ca298" "proxy-server 3610616" 0.0198 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:39449/sdb1/2/a/c1" "txca4e77432aed4922bfe7b-00665ca298" "container-server 3610616" 0.0225 "-" 3610616 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:44353/sda1/2/a/c1" "txca4e77432aed4922bfe7b-00665ca298" "container-server 3610616" 0.0020 "-" 3610616 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txca4e77432aed4922bfe7b-00665ca298" "proxy-server 3610616" 0.0834 "-" 3610616 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txca4e77432aed4922bfe7b-00665ca298" "proxy-server 3610616" 0.0565 "-" 3610616 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txca4e77432aed4922bfe7b-00665ca298 - 0.1165 - - 1717346968.132794380 1717346968.249279737 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txbe9adaa44790475f80aba-00665ca298" "proxy-server 3610616" 0.0143 "-" 3610616 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:39449/sdb1/1/a/c2" "txbe9adaa44790475f80aba-00665ca298" "container-server 3610616" 0.0102 "-" 3610616 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:44353/sda1/1/a/c2" "txbe9adaa44790475f80aba-00665ca298" "container-server 3610616" 0.0018 "-" 3610616 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txbe9adaa44790475f80aba-00665ca298" "proxy-server 3610616" 0.0355 "-" 3610616 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txbe9adaa44790475f80aba-00665ca298" "proxy-server 3610616" 0.0559 "-" 3610616 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txbe9adaa44790475f80aba-00665ca298 - 0.0809 - - 1717346968.254622459 1717346968.335562229 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpa1b5rwxj/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpa1b5rwxj/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpa1b5rwxj/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpa1b5rwxj/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0188 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0117 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0131 "-" 3610616 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0126 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf260ff9286bf43abb67b8-00665ca298" "proxy-server 3610616" 0.0018 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:46213/sda1/1/a/c" "txf260ff9286bf43abb67b8-00665ca298" "container-server 3610616" 0.0027 "-" 3610616 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:38377/sdb1/1/a/c" "txf260ff9286bf43abb67b8-00665ca298" "container-server 3610616" 0.0014 "-" 3610616 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf260ff9286bf43abb67b8-00665ca298" "proxy-server 3610616" 0.0425 "-" 3610616 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf260ff9286bf43abb67b8-00665ca298" "proxy-server 3610616" 0.0308 "-" 3610616 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a/c HTTP/1.0 201 - - t - - - txf260ff9286bf43abb67b8-00665ca298 - 0.0550 - - 1717346968.494255304 1717346968.549245358 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txf55e3eeb831649f385338-00665ca298" "proxy-server 3610616" 0.0017 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:38377/sdb1/1/a1/c1" "txf55e3eeb831649f385338-00665ca298" "container-server 3610616" 0.0016 "-" 3610616 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:46213/sda1/1/a1/c1" "txf55e3eeb831649f385338-00665ca298" "container-server 3610616" 0.0016 "-" 3610616 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txf55e3eeb831649f385338-00665ca298" "proxy-server 3610616" 0.0228 "-" 3610616 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txf55e3eeb831649f385338-00665ca298" "proxy-server 3610616" 0.0404 "-" 3610616 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txf55e3eeb831649f385338-00665ca298 - 0.0516 - - 1717346968.551322937 1717346968.602970600 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdeb03922a7dc48aa8c997-00665ca298" "proxy-server 3610616" 0.0118 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:38377/sdb1/2/a/c1" "txdeb03922a7dc48aa8c997-00665ca298" "container-server 3610616" 0.0158 "-" 3610616 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:46213/sda1/2/a/c1" "txdeb03922a7dc48aa8c997-00665ca298" "container-server 3610616" 0.0014 "-" 3610616 1 test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx009ee32fd7f04622af802-00665ca2a4) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx07004e897119467695428-00665ca2a4) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx9e14e1ab8800413ea2a2c-00665ca2a4) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txc43f82371c794deeaa22b-00665ca2a4) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc43f82371c794deeaa22b-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc43f82371c794deeaa22b-00665ca2a4) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txc43f82371c794deeaa22b-00665ca2a4) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txe6cbc91a640e4622bc39f-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txe6cbc91a640e4622bc39f-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txe6cbc91a640e4622bc39f-00665ca2a4) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txe6cbc91a640e4622bc39f-00665ca2a4) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx9b5fcca06d544ff8a2e53-00665ca2a4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9b5fcca06d544ff8a2e53-00665ca2a4) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx9b5fcca06d544ff8a2e53-00665ca2a4) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx9b5fcca06d544ff8a2e53-00665ca2a4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: Setting object count to 1068 (txn: tx68db5d81475744e6b204e-00665ca2a7) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: txe19d69e522d1479e93df4-00665ca2a7) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim ... ok swift: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 507 - "-" "-" "-" 0.0004 "-" 3610691 - test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/48/47 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0016 - - 1717346927.835229874 1717346927.836847067 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/48/47 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0014 - - 1717346927.837867498 1717346927.839259624 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware test NOTICE: Child 3610507 exiting normally test NOTICE: Started child 88 from parent 3610507 test NOTICE: Started child 89 from parent 3610507 test ERROR: Removing dead child 88 from parent 3610507 test NOTICE: Started child 90 from parent 3610507 test NOTICE: Started child 88 from parent 3610507 test NOTICE: Started child 89 from parent 3610507 test DEBUG: set nice priority to 1 test DEBUG: set ionice class 1 priority 0 WARNING: Unable to modify I/O scheduling class and priority of process. Keeping unchanged! Check logs for more info. test ERROR: Unable to modify ionice priority: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/libc.py", line 326, in _ioprio_set io_class = IO_CLASS_ENUM[io_class] ~~~~~~~~~~~~~^^^^^^^^^^ KeyError: 'class_foo' test DEBUG: set ionice class 2 priority 4 test DEBUG: set nice priority to -15 test DEBUG: set ionice class 3 priority 6 test ERROR: Problem finding shard upper in '/tmp/tmpxzuujch2/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpxzuujch2/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmps362_5g_/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmps362_5g_/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors ... ok cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txdeb03922a7dc48aa8c997-00665ca298" "proxy-server 3610616" 0.0410 "-" 3610616 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txdeb03922a7dc48aa8c997-00665ca298" "proxy-server 3610616" 0.0574 "-" 3610616 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txdeb03922a7dc48aa8c997-00665ca298 - 0.0799 - - 1717346968.605016470 1717346968.684923172 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx19d8d46735ed4f719e974-00665ca298" "proxy-server 3610616" 0.0108 "-" 3610616 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:46213/sda1/1/a/c2" "tx19d8d46735ed4f719e974-00665ca298" "container-server 3610616" 0.0078 "-" 3610616 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:38377/sdb1/1/a/c2" "tx19d8d46735ed4f719e974-00665ca298" "container-server 3610616" 0.0019 "-" 3610616 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx19d8d46735ed4f719e974-00665ca298" "proxy-server 3610616" 0.0507 "-" 3610616 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:49:28 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx19d8d46735ed4f719e974-00665ca298" "proxy-server 3610616" 0.0315 "-" 3610616 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/49/28 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx19d8d46735ed4f719e974-00665ca298 - 0.0729 - - 1717346968.687075138 1717346968.759985924 2 test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'Content-Md5': '9Z+cE+YYsVYXLtR89GLUSA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'f59f9c13e618b156172ed47cf462d448', 'Content-Length': '88'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_conf test.unit.container.test_sharder.TestContainerSharderConf.test_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_default test.unit.container.test_sharder.TestContainerSharderConf.test_default ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate test.unit.container.test_sharder.TestContainerSharderConf.test_validate ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset ... ok test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txdd11df1bebf74f9aa999d-00665ca29f) test INFO: Client disconnected on read of '/a/c' (txn: txdd11df1bebf74f9aa999d-00665ca29f) test INFO: Client disconnected on read of '/a/c' (txn: tx9438fc7971e7461fa9842-00665ca29f) test INFO: Client disconnected on read of '/a/c' (txn: tx9438fc7971e7461fa9842-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx59119659e87444459b7af-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txeb8822c3b7ce429e9103e-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx3cb7acdc7fdf4786afd08-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx3cb7acdc7fdf4786afd08-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx3cb7acdc7fdf4786afd08-00665ca29f) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx3cb7acdc7fdf4786afd08-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx2188182f787449cf9d2f1-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx2188182f787449cf9d2f1-00665ca29f) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx2188182f787449cf9d2f1-00665ca29f) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx2188182f787449cf9d2f1-00665ca29f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: tx37b4064b8abd4bd5a964d-00665ca29f) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: tx37b4064b8abd4bd5a964d-00665ca29f) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: tx37b4064b8abd4bd5a964d-00665ca29f) test ERROR: Account HEAD returning 503 for [] (txn: tx37b4064b8abd4bd5a964d-00665ca29f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx2505351b85bf46f1a4c4c-00665ca29f) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx2b7d8945f24440969e8c0-00665ca29f) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx2b7d8945f24440969e8c0-00665ca29f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8e6ae671b0b4474087768-00665ca2a0) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txdbd2246d43384fe59f819-00665ca2a0) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6b323ae21798472b9d485-00665ca2a0) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e93f04488bb4916bcba8-00665ca2a0) test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT test.unit.proxy.test_mem_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test ERROR: Failed to unlink '/tmp/tmpxquo6a3f/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpxquo6a3f/containers/part/suffix/hash/container_1717346992.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpxquo6a3f/containers/part/suffix/hash/container_1717346992.00000.db': ['/tmp/tmpxquo6a3f/containers/part/suffix/hash/container.db', '/tmp/tmpxquo6a3f/containers/part/suffix/hash/container_1717346993.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpxquo6a3f/containers/part/suffix/hash/container_1717346993.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0225 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0201 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0124 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0188 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0124 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:55 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0296 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0148 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0178 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0128 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0152 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3610637 2 test ERROR: ERROR Account update failed with 127.0.0.1:58295/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0158 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0106 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0091 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0166 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0042 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0121 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0273 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:49:55 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0153 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0254 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:55 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:57 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 1.0201 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:57 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:57 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.1291 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0603 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0159 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0167 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0014 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0198 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0043 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0126 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0043 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0013 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0149 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0135 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0011 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0168 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0177 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0072 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:58 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0137 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0128 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.8048 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:49:59 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0721 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0213 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0018 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0127 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:00 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0880 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0144 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0011 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0014 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmpvhpvzd28/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0139 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0129 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 220, in PUT self.req.get_acl_response(app, 'PUT', self.container) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail ... ok test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Setting object count to 1068 (txn: txe19d69e522d1479e93df4-00665ca2a7) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: tx97bb949f25fc4af99ebde-00665ca2aa) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx57113dfbc57947f7a6415-00665ca2ad) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx57113dfbc57947f7a6415-00665ca2ad) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx57113dfbc57947f7a6415-00665ca2ad) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx57113dfbc57947f7a6415-00665ca2ad) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx57113dfbc57947f7a6415-00665ca2ad) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx57113dfbc57947f7a6415-00665ca2ad) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: tx57113dfbc57947f7a6415-00665ca2ad) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: tx57113dfbc57947f7a6415-00665ca2ad) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 3.14 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 6.28 Search value matched 0 devices. The on-disk ring builder is unchanged. d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_unknown test.unit.cli.test_ringbuilder.TestCommands.test_unknown ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0249 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0011 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0007 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0006 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0010 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0007 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0006 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0011 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0011 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0010 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0011 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0012 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0239 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0049 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0051 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:14 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0022 "-" 3610691 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0006 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0181 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0212 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0231 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0076 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0033 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0148 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0013 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:19 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0149 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:20 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0190 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0031 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'Content-Md5': 'j7DkaSPeAAbx0dPUl3OK9A==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8fb0e46923de0006f1d1d3d497738af4', 'Content-Length': '289'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended ... ok test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0156 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0018 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0124 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0056 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0190 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0010 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0216 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0002 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0003 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0165 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0286 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0040 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:01 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0079 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0063 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0088 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0063 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0061 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0073 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0072 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmpb7tvr0y_/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0108 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0063 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0056 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0048 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0194 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0018 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0374 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0261 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0018 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0014 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0364 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0047 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0185 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker ... ok test ERROR: Failed to unlink '/tmp/tmpelnr7_gl/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpelnr7_gl/containers/part/suffix/hash/container_1717347005.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpelnr7_gl/containers/part/suffix/hash/container_1717347005.00000.db': ['/tmp/tmpelnr7_gl/containers/part/suffix/hash/container.db', '/tmp/tmpelnr7_gl/containers/part/suffix/hash/container_1717347006.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpelnr7_gl/containers/part/suffix/hash/container_1717347006.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: txf05d449cfd9043f38f25b-00665ca2c2) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: txf05d449cfd9043f38f25b-00665ca2c2) test INFO: Client disconnected on read of '/a/c' (txn: txf05d449cfd9043f38f25b-00665ca2c2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx41361b41f6494393a5578-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx41361b41f6494393a5578-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx41361b41f6494393a5578-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx41361b41f6494393a5578-00665ca2c2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: txb0386db2ee4b40a68a88d-00665ca2c2) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: txfe85a1cdeb734b07aa377-00665ca2c2) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx3b8c4b8b1879466798d43-00665ca2c2) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd (txn: txbb83f9f10c3d4176b1b97-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: tx927a58f71c464095a6a70-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txc76293f97a4347fb81a78-00665ca2c2) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txb64a7991a4d54533a8a4b-00665ca2c2) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: txbb3deb12397840aba37b2-00665ca2c2) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx9bf5f50415a146d1803b5-00665ca2c2) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: txf384095235db42b38f972-00665ca2c2) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx7f5bb6c9c8694526ae128-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: tx1689a6a6e48d421c925c8-00665ca2c2) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response ... ok test DEBUG: Setting object count to 1058 (txn: tx9ba8d9656ac1434aa437b-00665ca2b0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Setting object count to 1024 (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Setting object count to 1024 (txn: txb4890795e4ca47dba16fb-00665ca2b3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Setting object count to 1072 (txn: tx9869518b4ea24c43b6f45-00665ca2b6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx4e889d5ee8564aa5bcf98-00665ca2a6) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txcd6302ae6d4548ebb77ef-00665ca2a6) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txcd6302ae6d4548ebb77ef-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2b55595d72574f3187704-00665ca2a6) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx2b55595d72574f3187704-00665ca2a6) test WARNING: Could not autocreate account '/account' (txn: tx2b55595d72574f3187704-00665ca2a6) test INFO: autocreate account '/account' (txn: txf2deb1f2e6b54217bb73a-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test ERROR: Account HEAD returning 503 for [503, 503] (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test WARNING: Could not autocreate account '/account' (txn: tx9bf00f4fb7e647789e2d9-00665ca2a6) test INFO: autocreate account '/account' (txn: tx37230db089c140a0a9101-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txe27036d62e7c48f0a0ead-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txe27036d62e7c48f0a0ead-00665ca2a6) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txe27036d62e7c48f0a0ead-00665ca2a6) test INFO: autocreate account '/account' (txn: txc4e9eb14a1a84dc790f2f-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx255b502dc8c548918374c-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx255b502dc8c548918374c-00665ca2a6) test INFO: autocreate account '/account' (txn: tx255b502dc8c548918374c-00665ca2a6) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx513ca25ac60f43e9ac14d-00665ca2a6) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx513ca25ac60f43e9ac14d-00665ca2a6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: tx58a11b1c394d45a2b5ec7-00665ca2a6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: tx894f8ef416dd4da4abf71-00665ca2a6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx927fd9584c0b435885340-00665ca2a6) test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata ... ok test INFO: - - - [02/Jun/2024:16:49:20 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0021 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:20 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0021 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0159 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0001 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:28 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0153 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0161 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0166 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0070 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0035 "-" 3610691 0 test DEBUG: Unlinked retiring db '/tmp/tmp6_8h98in/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0073 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0037 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0157 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:28 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0043 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0091 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0176 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0031 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0196 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0105 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0163 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0065 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0160 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0194 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0044 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0220 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0023 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0247 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0154 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0117 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0020 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:29 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0023 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestContainerController.test_GET_xml test.unit.container.test_server.TestContainerController.test_GET_xml ... ok test.unit.container.test_server.TestContainerController.test_HEAD test.unit.container.test_server.TestContainerController.test_HEAD ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_HEAD_not_found test.unit.container.test_server.TestContainerController.test_HEAD_not_found ... ok test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestContainerController.test_OPTIONS test.unit.container.test_server.TestContainerController.test_OPTIONS ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:48:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Ignoring 404 from handoff (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server ERROR: Object returning 503 for [] (txn: txbf7340d614f646c78e2f7-00665ca2ad) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.7:1007/sdh (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.11:1011/sdl (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.3:1003/sdd (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.9:1009/sdj (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx1a840e9e69a44585ae5ec-00665ca2ae) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 Trying to GET /a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx16a852ee4b7b4d1baac69-00665ca2ae) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb474233547ac4e5ca5db5-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Ignoring 412 from primary (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match ... ok test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0454 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0205 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0243 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0256 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0109 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0213 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0040 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0040 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0041 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0046 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0196 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0239 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0163 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0348 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0198 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0160 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0055 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0177 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0187 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0041 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0041 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0233 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0040 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0033 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0031 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Setting object count to 1072 (txn: txc8447c895cd041c28b65f-00665ca2b9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Setting object count to 1072 (txn: txef3e13c0570c45d9b8db8-00665ca2bc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx7e746eb84e774229b853a-00665ca2bf) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error ... ok test INFO: - - - [02/Jun/2024:16:49:29 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0150 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:41 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0192 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:41 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:41 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0022 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:41 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:46 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.2300 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:46 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.2646 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:46 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.3721 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:46 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.1993 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:48 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 1.1156 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:49 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 1.5015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:49 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0113 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.5986 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0024 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0230 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0051 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0050 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0056 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0362 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0010 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0008 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.1974 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0014 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0196 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0170 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0096 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0263 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0049 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0217 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0181 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0206 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0206 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0104 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestContainerController.test_POST_insufficient_space test.unit.container.test_server.TestContainerController.test_POST_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_partition test.unit.container.test_server.TestContainerController.test_POST_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT test.unit.container.test_server.TestContainerController.test_PUT ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics ... ok test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3f0539eb9d9042b9900b5-00665ca2a6) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx396da57609494389a39be-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx318fcfd1d58c4fc59f2f8-00665ca2aa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx57800592af374e53b2938-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx66d65fac8fc84a79807a7-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx958e80fa318d4634b98a2-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx958e80fa318d4634b98a2-00665ca2aa) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx958e80fa318d4634b98a2-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txb311a4acfcfa4903965b4-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txb311a4acfcfa4903965b4-00665ca2aa) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txb311a4acfcfa4903965b4-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txa003e49c99a547f7a64b1-00665ca2aa) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa003e49c99a547f7a64b1-00665ca2aa) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txa003e49c99a547f7a64b1-00665ca2aa) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txa003e49c99a547f7a64b1-00665ca2aa) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txb46780b268e34652909ec-00665ca2aa) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txb46780b268e34652909ec-00665ca2aa) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txb46780b268e34652909ec-00665ca2aa) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txb46780b268e34652909ec-00665ca2aa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'Content-Md5': 'uXN9heP0Iedb6eM97zaw2A==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9737d85e3f421e75be9e33def36b0d8', 'Content-Length': '230'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. Unknown command: unknown swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate test.unit.cli.test_ringbuilder.TestCommands.test_validate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder test.unit.cli.test_ringbuilder.TestCommands.test_write_builder ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_ring ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission ... ok test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0041 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0242 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0135 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0126 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0139 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0017 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:02 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0009 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:03 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0135 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0062 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0217 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0109 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0130 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0109 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0020 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0022 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0247 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0128 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0115 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0201 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0061 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0381 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0489 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmp8xbe6y7t/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0041 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0035 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0180 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0249 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmpd_0mam6j/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0035 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0177 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0297 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0079 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0262 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0206 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0176 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0420 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0041 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0032 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0208 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0197 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0033 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states ... ok swift: - - - [02/Jun/2024:16:50:28 +0000] "PUT /sda1/p/a/c" "-" "-" "-" 0.0056 "-" 3610691 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0116 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0241 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0229 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0187 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0156 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0018 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0163 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0088 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0243 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0200 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0316 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0117 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0071 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0298 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0245 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0122 "-" 3610691 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3610691 - test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0193 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0099 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0026 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0022 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0067 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0026 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0027 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0021 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0025 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0034 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim ... ok sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717346958.00000, (0, 0) as of 1717346958.00000, active as of 1717346958.00000> to ShardRange<'k' to 't' as of 1717346959.00000, (0, 0) as of 1717346959.00000, active as of 1717346959.00000>, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346963.00000, (0, 0) as of 1717346963.00000, shrinking as of 1717346960.00000> to ShardRange<'k' to 't' as of 1717346963.00000, (0, 0) as of 1717346963.00000, sharding as of 1717346964.00000>, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpjkw2mhh0/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346964.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/8b2/04737de3895180ccaecd99af3f2bd8b2/04737de3895180ccaecd99af3f2bd8b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346966.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/77b/df877e71ec0e41657e75b98afed3377b/df877e71ec0e41657e75b98afed3377b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346965.00000, (0, 0) as of 1717346965.00000, found as of 1717346965.00000> to ShardRange<'k' to 't' as of 1717346966.00000, (0, 0) as of 1717346966.00000, created as of 1717346966.00000>, path: .shards_a/shard_c_1717346966.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/77b/df877e71ec0e41657e75b98afed3377b/df877e71ec0e41657e75b98afed3377b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346968.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fa8/e691c20083bdee6e13f53afef54e4fa8/e691c20083bdee6e13f53afef54e4fa8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346967.00000, (0, 0) as of 1717346967.00000, found as of 1717346967.00000> to ShardRange<'k' to 't' as of 1717346968.00000, (0, 0) as of 1717346968.00000, cleaved as of 1717346968.00000>, path: .shards_a/shard_c_1717346968.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fa8/e691c20083bdee6e13f53afef54e4fa8/e691c20083bdee6e13f53afef54e4fa8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346970.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f36/c0df4e240c9c6b76d4d66123a4c4cf36/c0df4e240c9c6b76d4d66123a4c4cf36.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346969.00000, (0, 0) as of 1717346969.00000, found as of 1717346969.00000> to ShardRange<'k' to 't' as of 1717346970.00000, (0, 0) as of 1717346970.00000, active as of 1717346970.00000>, path: .shards_a/shard_c_1717346970.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f36/c0df4e240c9c6b76d4d66123a4c4cf36/c0df4e240c9c6b76d4d66123a4c4cf36.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346972.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/115/21da6e959302796e44b6fde052627115/21da6e959302796e44b6fde052627115.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346971.00000, (0, 0) as of 1717346971.00000, found as of 1717346971.00000> to ShardRange<'k' to 't' as of 1717346972.00000, (0, 0) as of 1717346972.00000, shrinking as of 1717346972.00000>, path: .shards_a/shard_c_1717346972.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/115/21da6e959302796e44b6fde052627115/21da6e959302796e44b6fde052627115.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346972.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/115/21da6e959302796e44b6fde052627115/21da6e959302796e44b6fde052627115.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346974.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d21/3ad020a5aedff47c5a3fbf947216fd21/3ad020a5aedff47c5a3fbf947216fd21.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346973.00000, (0, 0) as of 1717346973.00000, found as of 1717346973.00000> to ShardRange<'k' to 't' as of 1717346974.00000, (0, 0) as of 1717346974.00000, sharding as of 1717346974.00000>, path: .shards_a/shard_c_1717346974.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d21/3ad020a5aedff47c5a3fbf947216fd21/3ad020a5aedff47c5a3fbf947216fd21.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346974.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d21/3ad020a5aedff47c5a3fbf947216fd21/3ad020a5aedff47c5a3fbf947216fd21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346976.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/689/209298c8aca233980f23c6792f13a689/209298c8aca233980f23c6792f13a689.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346975.00000, (0, 0) as of 1717346975.00000, found as of 1717346975.00000> to ShardRange<'k' to 't' as of 1717346976.00000, (0, 0) as of 1717346976.00000, sharded as of 1717346976.00000>, path: .shards_a/shard_c_1717346976.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/689/209298c8aca233980f23c6792f13a689/209298c8aca233980f23c6792f13a689.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346976.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/689/209298c8aca233980f23c6792f13a689/209298c8aca233980f23c6792f13a689.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346978.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/156/4c0a2d2abea81320a33944a8b3e58156/4c0a2d2abea81320a33944a8b3e58156.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346977.00000, (0, 0) as of 1717346977.00000, found as of 1717346977.00000> to ShardRange<'k' to 't' as of 1717346978.00000, (0, 0) as of 1717346978.00000, shrunk as of 1717346978.00000>, path: .shards_a/shard_c_1717346978.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/156/4c0a2d2abea81320a33944a8b3e58156/4c0a2d2abea81320a33944a8b3e58156.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346978.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/156/4c0a2d2abea81320a33944a8b3e58156/4c0a2d2abea81320a33944a8b3e58156.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346980.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/246/ff3136bf5ccb94119e76512114e48246/ff3136bf5ccb94119e76512114e48246.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346979.00000, (0, 0) as of 1717346979.00000, created as of 1717346979.00000> to ShardRange<'k' to 't' as of 1717346980.00000, (0, 0) as of 1717346980.00000, found as of 1717346980.00000>, path: .shards_a/shard_c_1717346980.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/246/ff3136bf5ccb94119e76512114e48246/ff3136bf5ccb94119e76512114e48246.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346982.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/43b/825f24f02891ff2ca01514dcd948c43b/825f24f02891ff2ca01514dcd948c43b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346984.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/87f/f68e9dc2195a3802bfb0f3c02bda087f/f68e9dc2195a3802bfb0f3c02bda087f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346983.00000, (0, 0) as of 1717346983.00000, created as of 1717346983.00000> to ShardRange<'k' to 't' as of 1717346984.00000, (0, 0) as of 1717346984.00000, cleaved as of 1717346984.00000>, path: .shards_a/shard_c_1717346984.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/87f/f68e9dc2195a3802bfb0f3c02bda087f/f68e9dc2195a3802bfb0f3c02bda087f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346986.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/9ea/d46479bfbdbf2256319cd28352f129ea/d46479bfbdbf2256319cd28352f129ea.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346985.00000, (0, 0) as of 1717346985.00000, created as of 1717346985.00000> to ShardRange<'k' to 't' as of 1717346986.00000, (0, 0) as of 1717346986.00000, active as of 1717346986.00000>, path: .shards_a/shard_c_1717346986.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/9ea/d46479bfbdbf2256319cd28352f129ea/d46479bfbdbf2256319cd28352f129ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346988.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/b39/8eed7e5a266ea87485a754cff6c4db39/8eed7e5a266ea87485a754cff6c4db39.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346987.00000, (0, 0) as of 1717346987.00000, created as of 1717346987.00000> to ShardRange<'k' to 't' as of 1717346988.00000, (0, 0) as of 1717346988.00000, shrinking as of 1717346988.00000>, path: .shards_a/shard_c_1717346988.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/b39/8eed7e5a266ea87485a754cff6c4db39/8eed7e5a266ea87485a754cff6c4db39.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346988.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/b39/8eed7e5a266ea87485a754cff6c4db39/8eed7e5a266ea87485a754cff6c4db39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346990.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ffd/32cbd0081670efb6d2c2a66a07593ffd/32cbd0081670efb6d2c2a66a07593ffd.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346989.00000, (0, 0) as of 1717346989.00000, created as of 1717346989.00000> to ShardRange<'k' to 't' as of 1717346990.00000, (0, 0) as of 1717346990.00000, sharding as of 1717346990.00000>, path: .shards_a/shard_c_1717346990.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ffd/32cbd0081670efb6d2c2a66a07593ffd/32cbd0081670efb6d2c2a66a07593ffd.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346990.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ffd/32cbd0081670efb6d2c2a66a07593ffd/32cbd0081670efb6d2c2a66a07593ffd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346992.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ff5/a68280f43b59ae2d65b342c2185a2ff5/a68280f43b59ae2d65b342c2185a2ff5.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346991.00000, (0, 0) as of 1717346991.00000, created as of 1717346991.00000> to ShardRange<'k' to 't' as of 1717346992.00000, (0, 0) as of 1717346992.00000, sharded as of 1717346992.00000>, path: .shards_a/shard_c_1717346992.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ff5/a68280f43b59ae2d65b342c2185a2ff5/a68280f43b59ae2d65b342c2185a2ff5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346992.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ff5/a68280f43b59ae2d65b342c2185a2ff5/a68280f43b59ae2d65b342c2185a2ff5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346994.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d34/4679776277e17a5dc387516992695d34/4679776277e17a5dc387516992695d34.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346993.00000, (0, 0) as of 1717346993.00000, created as of 1717346993.00000> to ShardRange<'k' to 't' as of 1717346994.00000, (0, 0) as of 1717346994.00000, shrunk as of 1717346994.00000>, path: .shards_a/shard_c_1717346994.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d34/4679776277e17a5dc387516992695d34/4679776277e17a5dc387516992695d34.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717346994.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d34/4679776277e17a5dc387516992695d34/4679776277e17a5dc387516992695d34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346996.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/82c/255f26178cf8ee84a7e582502455482c/255f26178cf8ee84a7e582502455482c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346995.00000, (0, 0) as of 1717346995.00000, cleaved as of 1717346995.00000> to ShardRange<'k' to 't' as of 1717346996.00000, (0, 0) as of 1717346996.00000, found as of 1717346996.00000>, path: .shards_a/shard_c_1717346996.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/82c/255f26178cf8ee84a7e582502455482c/255f26178cf8ee84a7e582502455482c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717346998.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/97b/ed5fffae56ec5e9a061735f7bfcb697b/ed5fffae56ec5e9a061735f7bfcb697b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717346997.00000, (0, 0) as of 1717346997.00000, cleaved as of 1717346997.00000> to ShardRange<'k' to 't' as of 1717346998.00000, (0, 0) as of 1717346998.00000, created as of 1717346998.00000>, path: .shards_a/shard_c_1717346998.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/97b/ed5fffae56ec5e9a061735f7bfcb697b/ed5fffae56ec5e9a061735f7bfcb697b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347000.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ef7/4b46d4b3a909d4f9ce0a4c3c721c1ef7/4b46d4b3a909d4f9ce0a4c3c721c1ef7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347002.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f9f/ede2f0ba9746529c8d8c34220ee3cf9f/ede2f0ba9746529c8d8c34220ee3cf9f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347001.00000, (0, 0) as of 1717347001.00000, cleaved as of 1717347001.00000> to ShardRange<'k' to 't' as of 1717347002.00000, (0, 0) as of 1717347002.00000, active as of 1717347002.00000>, path: .shards_a/shard_c_1717347002.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f9f/ede2f0ba9746529c8d8c34220ee3cf9f/ede2f0ba9746529c8d8c34220ee3cf9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347004.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f9e/981b2a4351a7a7adbb8c048933958f9e/981b2a4351a7a7adbb8c048933958f9e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347003.00000, (0, 0) as of 1717347003.00000, cleaved as of 1717347003.00000> to ShardRange<'k' to 't' as of 1717347004.00000, (0, 0) as of 1717347004.00000, shrinking as of 1717347004.00000>, path: .shards_a/shard_c_1717347004.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f9e/981b2a4351a7a7adbb8c048933958f9e/981b2a4351a7a7adbb8c048933958f9e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347004.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f9e/981b2a4351a7a7adbb8c048933958f9e/981b2a4351a7a7adbb8c048933958f9e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347006.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0c6/e0c48087ccbc5177f666144cd7b5d0c6/e0c48087ccbc5177f666144cd7b5d0c6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347005.00000, (0, 0) as of 1717347005.00000, cleaved as of 1717347005.00000> to ShardRange<'k' to 't' as of 1717347006.00000, (0, 0) as of 1717347006.00000, sharding as of 1717347006.00000>, path: .shards_a/shard_c_1717347006.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0c6/e0c48087ccbc5177f666144cd7b5d0c6/e0c48087ccbc5177f666144cd7b5d0c6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347006.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0c6/e0c48087ccbc5177f666144cd7b5d0c6/e0c48087ccbc5177f666144cd7b5d0c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347008.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/285/7d0114ec2f19de0a78841a33d20fc285/7d0114ec2f19de0a78841a33d20fc285.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347007.00000, (0, 0) as of 1717347007.00000, cleaved as of 1717347007.00000> to ShardRange<'k' to 't' as of 1717347008.00000, (0, 0) as of 1717347008.00000, sharded as of 1717347008.00000>, path: .shards_a/shard_c_1717347008.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/285/7d0114ec2f19de0a78841a33d20fc285/7d0114ec2f19de0a78841a33d20fc285.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347008.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/285/7d0114ec2f19de0a78841a33d20fc285/7d0114ec2f19de0a78841a33d20fc285.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347010.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/de4/a9d88f0a3f4183a84cc2bb3cc178dde4/a9d88f0a3f4183a84cc2bb3cc178dde4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347009.00000, (0, 0) as of 1717347009.00000, cleaved as of 1717347009.00000> to ShardRange<'k' to 't' as of 1717347010.00000, (0, 0) as of 1717347010.00000, shrunk as of 1717347010.00000>, path: .shards_a/shard_c_1717347010.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/de4/a9d88f0a3f4183a84cc2bb3cc178dde4/a9d88f0a3f4183a84cc2bb3cc178dde4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347010.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/de4/a9d88f0a3f4183a84cc2bb3cc178dde4/a9d88f0a3f4183a84cc2bb3cc178dde4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347012.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f33/738733dd55247f709f6aabab343d5f33/738733dd55247f709f6aabab343d5f33.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347011.00000, (0, 0) as of 1717347011.00000, active as of 1717347011.00000> to ShardRange<'k' to 't' as of 1717347012.00000, (0, 0) as of 1717347012.00000, found as of 1717347012.00000>, path: .shards_a/shard_c_1717347012.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f33/738733dd55247f709f6aabab343d5f33/738733dd55247f709f6aabab343d5f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347014.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e29/1b19d797b74b31c4754aacbec55dbe29/1b19d797b74b31c4754aacbec55dbe29.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347013.00000, (0, 0) as of 1717347013.00000, active as of 1717347013.00000> to ShardRange<'k' to 't' as of 1717347014.00000, (0, 0) as of 1717347014.00000, created as of 1717347014.00000>, path: .shards_a/shard_c_1717347014.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e29/1b19d797b74b31c4754aacbec55dbe29/1b19d797b74b31c4754aacbec55dbe29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347016.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/497/c9897a1dc74e3d292fe5256275c17497/c9897a1dc74e3d292fe5256275c17497.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347015.00000, (0, 0) as of 1717347015.00000, active as of 1717347015.00000> to ShardRange<'k' to 't' as of 1717347016.00000, (0, 0) as of 1717347016.00000, cleaved as of 1717347016.00000>, path: .shards_a/shard_c_1717347016.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/497/c9897a1dc74e3d292fe5256275c17497/c9897a1dc74e3d292fe5256275c17497.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347018.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/264/77512d5f55b51fefd020d0feb5090264/77512d5f55b51fefd020d0feb5090264.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347020.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/c94/9e6fa30b95e9d00dba80fd7e157f2c94/9e6fa30b95e9d00dba80fd7e157f2c94.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347019.00000, (0, 0) as of 1717347019.00000, active as of 1717347019.00000> to ShardRange<'k' to 't' as of 1717347020.00000, (0, 0) as of 1717347020.00000, shrinking as of 1717347020.00000>, path: .shards_a/shard_c_1717347020.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/c94/9e6fa30b95e9d00dba80fd7e157f2c94/9e6fa30b95e9d00dba80fd7e157f2c94.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347020.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/c94/9e6fa30b95e9d00dba80fd7e157f2c94/9e6fa30b95e9d00dba80fd7e157f2c94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347022.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/5bf/21efb6f6b5c7fcaac7131472dc1b15bf/21efb6f6b5c7fcaac7131472dc1b15bf.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347021.00000, (0, 0) as of 1717347021.00000, active as of 1717347021.00000> to ShardRange<'k' to 't' as of 1717347022.00000, (0, 0) as of 1717347022.00000, sharding as of 1717347022.00000>, path: .shards_a/shard_c_1717347022.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/5bf/21efb6f6b5c7fcaac7131472dc1b15bf/21efb6f6b5c7fcaac7131472dc1b15bf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347022.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/5bf/21efb6f6b5c7fcaac7131472dc1b15bf/21efb6f6b5c7fcaac7131472dc1b15bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347024.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/b75/a9ace6b49816980d8ae92d77a5392b75/a9ace6b49816980d8ae92d77a5392b75.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347023.00000, (0, 0) as of 1717347023.00000, active as of 1717347023.00000> to ShardRange<'k' to 't' as of 1717347024.00000, (0, 0) as of 1717347024.00000, sharded as of 1717347024.00000>, path: .shards_a/shard_c_1717347024.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/b75/a9ace6b49816980d8ae92d77a5392b75/a9ace6b49816980d8ae92d77a5392b75.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347024.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/b75/a9ace6b49816980d8ae92d77a5392b75/a9ace6b49816980d8ae92d77a5392b75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347026.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/97e/2fde3a10e472b19e0a3cb20ce3a6c97e/2fde3a10e472b19e0a3cb20ce3a6c97e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347025.00000, (0, 0) as of 1717347025.00000, active as of 1717347025.00000> to ShardRange<'k' to 't' as of 1717347026.00000, (0, 0) as of 1717347026.00000, shrunk as of 1717347026.00000>, path: .shards_a/shard_c_1717347026.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/97e/2fde3a10e472b19e0a3cb20ce3a6c97e/2fde3a10e472b19e0a3cb20ce3a6c97e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347026.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/97e/2fde3a10e472b19e0a3cb20ce3a6c97e/2fde3a10e472b19e0a3cb20ce3a6c97e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347028.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/3d0/acf6911e0a6d6ce98226b382cfe3b3d0/acf6911e0a6d6ce98226b382cfe3b3d0.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347027.00000, (0, 0) as of 1717347027.00000, shrinking as of 1717347027.00000> to ShardRange<'k' to 't' as of 1717347028.00000, (0, 0) as of 1717347028.00000, found as of 1717347028.00000>, path: .shards_a/shard_c_1717347028.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/3d0/acf6911e0a6d6ce98226b382cfe3b3d0/acf6911e0a6d6ce98226b382cfe3b3d0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347030.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/bc9/8dee6cf99f0718c91fc3bc11885fdbc9/8dee6cf99f0718c91fc3bc11885fdbc9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347029.00000, (0, 0) as of 1717347029.00000, shrinking as of 1717347029.00000> to ShardRange<'k' to 't' as of 1717347030.00000, (0, 0) as of 1717347030.00000, created as of 1717347030.00000>, path: .shards_a/shard_c_1717347030.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/bc9/8dee6cf99f0718c91fc3bc11885fdbc9/8dee6cf99f0718c91fc3bc11885fdbc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347032.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/cf4/191a0b86b11dbf14318897764a6bacf4/191a0b86b11dbf14318897764a6bacf4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347031.00000, (0, 0) as of 1717347031.00000, shrinking as of 1717347031.00000> to ShardRange<'k' to 't' as of 1717347032.00000, (0, 0) as of 1717347032.00000, cleaved as of 1717347032.00000>, path: .shards_a/shard_c_1717347032.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/cf4/191a0b86b11dbf14318897764a6bacf4/191a0b86b11dbf14318897764a6bacf4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347034.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/3b1/3bf99bf1eea9e2d63954f83d7cccf3b1/3bf99bf1eea9e2d63954f83d7cccf3b1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347033.00000, (0, 0) as of 1717347033.00000, shrinking as of 1717347033.00000> to ShardRange<'k' to 't' as of 1717347034.00000, (0, 0) as of 1717347034.00000, active as of 1717347034.00000>, path: .shards_a/shard_c_1717347034.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/3b1/3bf99bf1eea9e2d63954f83d7cccf3b1/3bf99bf1eea9e2d63954f83d7cccf3b1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347036.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/53f/cf7633f2d74d78bf9a2f9a4cad6f953f/cf7633f2d74d78bf9a2f9a4cad6f953f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347036.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/53f/cf7633f2d74d78bf9a2f9a4cad6f953f/cf7633f2d74d78bf9a2f9a4cad6f953f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347038.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/27d/7312a2cc0b5df0a09d6b0c03d8d0927d/7312a2cc0b5df0a09d6b0c03d8d0927d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347037.00000, (0, 0) as of 1717347037.00000, shrinking as of 1717347037.00000> to ShardRange<'k' to 't' as of 1717347038.00000, (0, 0) as of 1717347038.00000, sharding as of 1717347038.00000>, path: .shards_a/shard_c_1717347038.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/27d/7312a2cc0b5df0a09d6b0c03d8d0927d/7312a2cc0b5df0a09d6b0c03d8d0927d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347038.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/27d/7312a2cc0b5df0a09d6b0c03d8d0927d/7312a2cc0b5df0a09d6b0c03d8d0927d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347040.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1a1/9cd3611626e53743b09202b8044c41a1/9cd3611626e53743b09202b8044c41a1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347039.00000, (0, 0) as of 1717347039.00000, shrinking as of 1717347039.00000> to ShardRange<'k' to 't' as of 1717347040.00000, (0, 0) as of 1717347040.00000, sharded as of 1717347040.00000>, path: .shards_a/shard_c_1717347040.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1a1/9cd3611626e53743b09202b8044c41a1/9cd3611626e53743b09202b8044c41a1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347040.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1a1/9cd3611626e53743b09202b8044c41a1/9cd3611626e53743b09202b8044c41a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347042.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/a75/65bb127cb5a38b1bf5c60a2851a4ca75/65bb127cb5a38b1bf5c60a2851a4ca75.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347041.00000, (0, 0) as of 1717347041.00000, shrinking as of 1717347041.00000> to ShardRange<'k' to 't' as of 1717347042.00000, (0, 0) as of 1717347042.00000, shrunk as of 1717347042.00000>, path: .shards_a/shard_c_1717347042.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/a75/65bb127cb5a38b1bf5c60a2851a4ca75/65bb127cb5a38b1bf5c60a2851a4ca75.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347042.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/a75/65bb127cb5a38b1bf5c60a2851a4ca75/65bb127cb5a38b1bf5c60a2851a4ca75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347044.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/7de/ad62a5d6d8d190a59b606758fba147de/ad62a5d6d8d190a59b606758fba147de.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347043.00000, (0, 0) as of 1717347043.00000, sharding as of 1717347043.00000> to ShardRange<'k' to 't' as of 1717347044.00000, (0, 0) as of 1717347044.00000, found as of 1717347044.00000>, path: .shards_a/shard_c_1717347044.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/7de/ad62a5d6d8d190a59b606758fba147de/ad62a5d6d8d190a59b606758fba147de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347046.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/346/ba3360760d78c1f8422e41dddc890346/ba3360760d78c1f8422e41dddc890346.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347045.00000, (0, 0) as of 1717347045.00000, sharding as of 1717347045.00000> to ShardRange<'k' to 't' as of 1717347046.00000, (0, 0) as of 1717347046.00000, created as of 1717347046.00000>, path: .shards_a/shard_c_1717347046.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/346/ba3360760d78c1f8422e41dddc890346/ba3360760d78c1f8422e41dddc890346.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347048.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/3bf/c9737776334fb42b40d7e3244f62b3bf/c9737776334fb42b40d7e3244f62b3bf.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347047.00000, (0, 0) as of 1717347047.00000, sharding as of 1717347047.00000> to ShardRange<'k' to 't' as of 1717347048.00000, (0, 0) as of 1717347048.00000, cleaved as of 1717347048.00000>, path: .shards_a/shard_c_1717347048.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/3bf/c9737776334fb42b40d7e3244f62b3bf/c9737776334fb42b40d7e3244f62b3bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347050.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/8b8/efc1da84f858252595c1b1e07e6448b8/efc1da84f858252595c1b1e07e6448b8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347049.00000, (0, 0) as of 1717347049.00000, sharding as of 1717347049.00000> to ShardRange<'k' to 't' as of 1717347050.00000, (0, 0) as of 1717347050.00000, active as of 1717347050.00000>, path: .shards_a/shard_c_1717347050.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/8b8/efc1da84f858252595c1b1e07e6448b8/efc1da84f858252595c1b1e07e6448b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347052.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/bfa/d6f2b364000511abcd31f45f08ab4bfa/d6f2b364000511abcd31f45f08ab4bfa.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347051.00000, (0, 0) as of 1717347051.00000, sharding as of 1717347051.00000> to ShardRange<'k' to 't' as of 1717347052.00000, (0, 0) as of 1717347052.00000, shrinking as of 1717347052.00000>, path: .shards_a/shard_c_1717347052.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/bfa/d6f2b364000511abcd31f45f08ab4bfa/d6f2b364000511abcd31f45f08ab4bfa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347052.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/bfa/d6f2b364000511abcd31f45f08ab4bfa/d6f2b364000511abcd31f45f08ab4bfa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347054.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/690/a62d3e7a5dca358a5e4fbfa8e1d1d690/a62d3e7a5dca358a5e4fbfa8e1d1d690.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347054.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/690/a62d3e7a5dca358a5e4fbfa8e1d1d690/a62d3e7a5dca358a5e4fbfa8e1d1d690.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347056.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/c27/6669b4dbad7e57bb5a2d40f5e98b0c27/6669b4dbad7e57bb5a2d40f5e98b0c27.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347055.00000, (0, 0) as of 1717347055.00000, sharding as of 1717347055.00000> to ShardRange<'k' to 't' as of 1717347056.00000, (0, 0) as of 1717347056.00000, sharded as of 1717347056.00000>, path: .shards_a/shard_c_1717347056.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/c27/6669b4dbad7e57bb5a2d40f5e98b0c27/6669b4dbad7e57bb5a2d40f5e98b0c27.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347056.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/c27/6669b4dbad7e57bb5a2d40f5e98b0c27/6669b4dbad7e57bb5a2d40f5e98b0c27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347058.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/369/cc52094b338256a4783bf420509b8369/cc52094b338256a4783bf420509b8369.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347057.00000, (0, 0) as of 1717347057.00000, sharding as of 1717347057.00000> to ShardRange<'k' to 't' as of 1717347058.00000, (0, 0) as of 1717347058.00000, shrunk as of 1717347058.00000>, path: .shards_a/shard_c_1717347058.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/369/cc52094b338256a4783bf420509b8369/cc52094b338256a4783bf420509b8369.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347058.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/369/cc52094b338256a4783bf420509b8369/cc52094b338256a4783bf420509b8369.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347060.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d8a/8dc27574fc211945da808401f1691d8a/8dc27574fc211945da808401f1691d8a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347059.00000, (0, 0) as of 1717347059.00000, sharded as of 1717347059.00000> to ShardRange<'k' to 't' as of 1717347060.00000, (0, 0) as of 1717347060.00000, found as of 1717347060.00000>, path: .shards_a/shard_c_1717347060.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/d8a/8dc27574fc211945da808401f1691d8a/8dc27574fc211945da808401f1691d8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347062.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fae/dcefeba524ca022b04304ad762ca2fae/dcefeba524ca022b04304ad762ca2fae.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347061.00000, (0, 0) as of 1717347061.00000, sharded as of 1717347061.00000> to ShardRange<'k' to 't' as of 1717347062.00000, (0, 0) as of 1717347062.00000, created as of 1717347062.00000>, path: .shards_a/shard_c_1717347062.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fae/dcefeba524ca022b04304ad762ca2fae/dcefeba524ca022b04304ad762ca2fae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347064.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f83/00d44deda34d77389fa078a1e2646f83/00d44deda34d77389fa078a1e2646f83.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347063.00000, (0, 0) as of 1717347063.00000, sharded as of 1717347063.00000> to ShardRange<'k' to 't' as of 1717347064.00000, (0, 0) as of 1717347064.00000, cleaved as of 1717347064.00000>, path: .shards_a/shard_c_1717347064.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f83/00d44deda34d77389fa078a1e2646f83/00d44deda34d77389fa078a1e2646f83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347066.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f87/c9773b9e95cce7abafd6d3075914ef87/c9773b9e95cce7abafd6d3075914ef87.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347065.00000, (0, 0) as of 1717347065.00000, sharded as of 1717347065.00000> to ShardRange<'k' to 't' as of 1717347066.00000, (0, 0) as of 1717347066.00000, active as of 1717347066.00000>, path: .shards_a/shard_c_1717347066.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f87/c9773b9e95cce7abafd6d3075914ef87/c9773b9e95cce7abafd6d3075914ef87.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347068.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/884/f3e30a93426fcf16dee46158c58ef884/f3e30a93426fcf16dee46158c58ef884.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347067.00000, (0, 0) as of 1717347067.00000, sharded as of 1717347067.00000> to ShardRange<'k' to 't' as of 1717347068.00000, (0, 0) as of 1717347068.00000, shrinking as of 1717347068.00000>, path: .shards_a/shard_c_1717347068.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/884/f3e30a93426fcf16dee46158c58ef884/f3e30a93426fcf16dee46158c58ef884.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347068.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/884/f3e30a93426fcf16dee46158c58ef884/f3e30a93426fcf16dee46158c58ef884.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347070.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/13b/3faed025583b1436472eef27034af13b/3faed025583b1436472eef27034af13b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347069.00000, (0, 0) as of 1717347069.00000, sharded as of 1717347069.00000> to ShardRange<'k' to 't' as of 1717347070.00000, (0, 0) as of 1717347070.00000, sharding as of 1717347070.00000>, path: .shards_a/shard_c_1717347070.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/13b/3faed025583b1436472eef27034af13b/3faed025583b1436472eef27034af13b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347070.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/13b/3faed025583b1436472eef27034af13b/3faed025583b1436472eef27034af13b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347072.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/854/40b126d0f114bbaa285735632fbd3854/40b126d0f114bbaa285735632fbd3854.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347072.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/854/40b126d0f114bbaa285735632fbd3854/40b126d0f114bbaa285735632fbd3854.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347074.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1b8/40208c20d1c03368b4f91f174f9d51b8/40208c20d1c03368b4f91f174f9d51b8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347073.00000, (0, 0) as of 1717347073.00000, sharded as of 1717347073.00000> to ShardRange<'k' to 't' as of 1717347074.00000, (0, 0) as of 1717347074.00000, shrunk as of 1717347074.00000>, path: .shards_a/shard_c_1717347074.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1b8/40208c20d1c03368b4f91f174f9d51b8/40208c20d1c03368b4f91f174f9d51b8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347074.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1b8/40208c20d1c03368b4f91f174f9d51b8/40208c20d1c03368b4f91f174f9d51b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347076.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/67e/ff427f28aeca0c5ebe102ca610a1f67e/ff427f28aeca0c5ebe102ca610a1f67e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347075.00000, (0, 0) as of 1717347075.00000, shrunk as of 1717347075.00000> to ShardRange<'k' to 't' as of 1717347076.00000, (0, 0) as of 1717347076.00000, found as of 1717347076.00000>, path: .shards_a/shard_c_1717347076.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/67e/ff427f28aeca0c5ebe102ca610a1f67e/ff427f28aeca0c5ebe102ca610a1f67e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347078.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/7a0/2e24b44784890694876558828d2db7a0/2e24b44784890694876558828d2db7a0.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347077.00000, (0, 0) as of 1717347077.00000, shrunk as of 1717347077.00000> to ShardRange<'k' to 't' as of 1717347078.00000, (0, 0) as of 1717347078.00000, created as of 1717347078.00000>, path: .shards_a/shard_c_1717347078.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/7a0/2e24b44784890694876558828d2db7a0/2e24b44784890694876558828d2db7a0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347080.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/674/8c4ff932331b17b8d6b294913d00f674/8c4ff932331b17b8d6b294913d00f674.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347079.00000, (0, 0) as of 1717347079.00000, shrunk as of 1717347079.00000> to ShardRange<'k' to 't' as of 1717347080.00000, (0, 0) as of 1717347080.00000, cleaved as of 1717347080.00000>, path: .shards_a/shard_c_1717347080.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/674/8c4ff932331b17b8d6b294913d00f674/8c4ff932331b17b8d6b294913d00f674.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347082.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/653/22dd04a079c9c73286fcd327ffd59653/22dd04a079c9c73286fcd327ffd59653.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347081.00000, (0, 0) as of 1717347081.00000, shrunk as of 1717347081.00000> to ShardRange<'k' to 't' as of 1717347082.00000, (0, 0) as of 1717347082.00000, active as of 1717347082.00000>, path: .shards_a/shard_c_1717347082.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/653/22dd04a079c9c73286fcd327ffd59653/22dd04a079c9c73286fcd327ffd59653.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347084.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/528/eb8e19f4369587c82c63766dd20bb528/eb8e19f4369587c82c63766dd20bb528.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347083.00000, (0, 0) as of 1717347083.00000, shrunk as of 1717347083.00000> to ShardRange<'k' to 't' as of 1717347084.00000, (0, 0) as of 1717347084.00000, shrinking as of 1717347084.00000>, path: .shards_a/shard_c_1717347084.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/528/eb8e19f4369587c82c63766dd20bb528/eb8e19f4369587c82c63766dd20bb528.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347084.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/528/eb8e19f4369587c82c63766dd20bb528/eb8e19f4369587c82c63766dd20bb528.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347086.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6c8/ab3e3aaa93bf298fc902a6ba9a2856c8/ab3e3aaa93bf298fc902a6ba9a2856c8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347085.00000, (0, 0) as of 1717347085.00000, shrunk as of 1717347085.00000> to ShardRange<'k' to 't' as of 1717347086.00000, (0, 0) as of 1717347086.00000, sharding as of 1717347086.00000>, path: .shards_a/shard_c_1717347086.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6c8/ab3e3aaa93bf298fc902a6ba9a2856c8/ab3e3aaa93bf298fc902a6ba9a2856c8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347086.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6c8/ab3e3aaa93bf298fc902a6ba9a2856c8/ab3e3aaa93bf298fc902a6ba9a2856c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347088.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4a2/e1f03f7ab73490ec83cf32b1b52824a2/e1f03f7ab73490ec83cf32b1b52824a2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347087.00000, (0, 0) as of 1717347087.00000, shrunk as of 1717347087.00000> to ShardRange<'k' to 't' as of 1717347088.00000, (0, 0) as of 1717347088.00000, sharded as of 1717347088.00000>, path: .shards_a/shard_c_1717347088.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4a2/e1f03f7ab73490ec83cf32b1b52824a2/e1f03f7ab73490ec83cf32b1b52824a2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347088.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4a2/e1f03f7ab73490ec83cf32b1b52824a2/e1f03f7ab73490ec83cf32b1b52824a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347090.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/441/5704168f2c5587131cac6ae15fb23441/5704168f2c5587131cac6ae15fb23441.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347090.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/441/5704168f2c5587131cac6ae15fb23441/5704168f2c5587131cac6ae15fb23441.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347091.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e79/d3163dd4e3f2f0d281c6e966af9aee79/d3163dd4e3f2f0d281c6e966af9aee79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347093.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/228/4abca9ba86e86c1e526e46772a661228/4abca9ba86e86c1e526e46772a661228.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347095.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/de0/723c57733fb4c9f0e258e134cf42ede0/723c57733fb4c9f0e258e134cf42ede0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347097.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/dcf/da82565b5a44da03ec4ba607dc73adcf/da82565b5a44da03ec4ba607dc73adcf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347099.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/8d6/68eac319ae9eacc0f3131b43f97a58d6/68eac319ae9eacc0f3131b43f97a58d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347101.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fc9/6e5d4b4dc3e11a6df919d121c8c9cfc9/6e5d4b4dc3e11a6df919d121c8c9cfc9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347103.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ba7/8853bf86190ce8e22bc7269f10106ba7/8853bf86190ce8e22bc7269f10106ba7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347105.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/053/c6bb675aa7c08f447cb97e2f0610c053/c6bb675aa7c08f447cb97e2f0610c053.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347107.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/2bf/0f30d260dfe2d19615fc6a1a458a72bf/0f30d260dfe2d19615fc6a1a458a72bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347109.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/82a/a1e2d1303f3a882419b8d51b2e9b682a/a1e2d1303f3a882419b8d51b2e9b682a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347111.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e79/e273715ebb04d1a2c0b1492a564bde79/e273715ebb04d1a2c0b1492a564bde79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347113.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1ee/2accfdaae174f5a25484799c6e09e1ee/2accfdaae174f5a25484799c6e09e1ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347115.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e59/dbd39f5efc249de57de809f64197ce59/dbd39f5efc249de57de809f64197ce59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347117.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/78f/4fe57b6dd36cbd76f0a91dde5104d78f/4fe57b6dd36cbd76f0a91dde5104d78f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347119.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/64e/64abe1adc5f768e391579d6a0caed64e/64abe1adc5f768e391579d6a0caed64e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347121.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/5b9/ed43009d669845cb95aa79e3865025b9/ed43009d669845cb95aa79e3865025b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347123.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e86/31a893677733ece76ffa980c57ff8e86/31a893677733ece76ffa980c57ff8e86.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347125.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/437/cbbea1bf6f93cdf140186e8767e02437/cbbea1bf6f93cdf140186e8767e02437.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347127.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e65/3ef77dd83df3f11580ffb16714429e65/3ef77dd83df3f11580ffb16714429e65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347129.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e68/5d9546b7afbfd08b74a0044d522b8e68/5d9546b7afbfd08b74a0044d522b8e68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347131.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/592/0b18c3bd3cc038a6b67c7dffe912a592/0b18c3bd3cc038a6b67c7dffe912a592.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347133.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6eb/6549f8c3cfa1a0ec48755020a1be56eb/6549f8c3cfa1a0ec48755020a1be56eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347135.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/55e/6f98317c4795e7eac42f665addee355e/6f98317c4795e7eac42f665addee355e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347137.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/bdc/0944df055132702a21f6dceefebe6bdc/0944df055132702a21f6dceefebe6bdc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347139.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1a6/ae7c61a974a728d17a1b7ad3114cb1a6/ae7c61a974a728d17a1b7ad3114cb1a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347141.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/be0/1f02f1890e682aff919f390ae23a6be0/1f02f1890e682aff919f390ae23a6be0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347143.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/353/7205603407c34c09cdeff60b9f00a353/7205603407c34c09cdeff60b9f00a353.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347145.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ba0/11d127c019a226761101c3f50c8d6ba0/11d127c019a226761101c3f50c8d6ba0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347147.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/85a/4541b1f3add72b05b3b2bbf0cb19885a/4541b1f3add72b05b3b2bbf0cb19885a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347149.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/26c/98676988f1c6d979667f8e6b8e90c26c/98676988f1c6d979667f8e6b8e90c26c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347151.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/218/e7532896a458a1822a625a3cb9875218/e7532896a458a1822a625a3cb9875218.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347153.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/de8/0fbf5567fd47ef9c9ebbfc35211bade8/0fbf5567fd47ef9c9ebbfc35211bade8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347155.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/354/7a0ca7773bc99b45103f6debd0c72354/7a0ca7773bc99b45103f6debd0c72354.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347155.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/354/7a0ca7773bc99b45103f6debd0c72354/7a0ca7773bc99b45103f6debd0c72354.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347157.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/13d/e20b8d80c38f2f9d6ff9fac26b27913d/e20b8d80c38f2f9d6ff9fac26b27913d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347157.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/13d/e20b8d80c38f2f9d6ff9fac26b27913d/e20b8d80c38f2f9d6ff9fac26b27913d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347159.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/918/2e19e90a50bc96146e7884137d1a4918/2e19e90a50bc96146e7884137d1a4918.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347159.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/918/2e19e90a50bc96146e7884137d1a4918/2e19e90a50bc96146e7884137d1a4918.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347161.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/cc7/ba4edd1f363582f51afa5aeebbbfecc7/ba4edd1f363582f51afa5aeebbbfecc7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347161.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/cc7/ba4edd1f363582f51afa5aeebbbfecc7/ba4edd1f363582f51afa5aeebbbfecc7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347163.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6e1/c1ea0f2476fa7569953d39474f67d6e1/c1ea0f2476fa7569953d39474f67d6e1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347163.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6e1/c1ea0f2476fa7569953d39474f67d6e1/c1ea0f2476fa7569953d39474f67d6e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347165.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6be/9b0b0a4ed60b911617351b3b867cd6be/9b0b0a4ed60b911617351b3b867cd6be.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347165.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6be/9b0b0a4ed60b911617351b3b867cd6be/9b0b0a4ed60b911617351b3b867cd6be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347167.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/08c/ee3d9778b313ee122dd6522fb83f208c/ee3d9778b313ee122dd6522fb83f208c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347167.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/08c/ee3d9778b313ee122dd6522fb83f208c/ee3d9778b313ee122dd6522fb83f208c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347169.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f96/1f30fe955c5e57a2fbdd5298f1a87f96/1f30fe955c5e57a2fbdd5298f1a87f96.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347169.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/f96/1f30fe955c5e57a2fbdd5298f1a87f96/1f30fe955c5e57a2fbdd5298f1a87f96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347171.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/8b9/969663808f61d2e58a1ac320e55488b9/969663808f61d2e58a1ac320e55488b9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347171.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/8b9/969663808f61d2e58a1ac320e55488b9/969663808f61d2e58a1ac320e55488b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347173.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0d7/182d2a0836e2412a89f1f75f610b80d7/182d2a0836e2412a89f1f75f610b80d7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347173.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0d7/182d2a0836e2412a89f1f75f610b80d7/182d2a0836e2412a89f1f75f610b80d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347175.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fc2/274317b258308e97992b9980fa27efc2/274317b258308e97992b9980fa27efc2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347175.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/fc2/274317b258308e97992b9980fa27efc2/274317b258308e97992b9980fa27efc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347177.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/50e/8463b49565a2fbbf1ffea13d82c2650e/8463b49565a2fbbf1ffea13d82c2650e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347177.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/50e/8463b49565a2fbbf1ffea13d82c2650e/8463b49565a2fbbf1ffea13d82c2650e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347179.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/261/4bcf6e1ac57d7592c73425100b614261/4bcf6e1ac57d7592c73425100b614261.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347179.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/261/4bcf6e1ac57d7592c73425100b614261/4bcf6e1ac57d7592c73425100b614261.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347181.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e4a/4ff52a4f0b8d105c4b061e6bd8f46e4a/4ff52a4f0b8d105c4b061e6bd8f46e4a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347181.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/e4a/4ff52a4f0b8d105c4b061e6bd8f46e4a/4ff52a4f0b8d105c4b061e6bd8f46e4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347183.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0c9/6765f1d56d4cc2d69b82045edc1ce0c9/6765f1d56d4cc2d69b82045edc1ce0c9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347183.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/0c9/6765f1d56d4cc2d69b82045edc1ce0c9/6765f1d56d4cc2d69b82045edc1ce0c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347185.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/61b/7422b6dece8da72045f762e62e12e61b/7422b6dece8da72045f762e62e12e61b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347185.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/61b/7422b6dece8da72045f762e62e12e61b/7422b6dece8da72045f762e62e12e61b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347187.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/632/efd2821d5d594a8bfbd14c39865f9632/efd2821d5d594a8bfbd14c39865f9632.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347187.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/632/efd2821d5d594a8bfbd14c39865f9632/efd2821d5d594a8bfbd14c39865f9632.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347189.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/14b/4769539ea5b078a53bccc806f614914b/4769539ea5b078a53bccc806f614914b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347189.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/14b/4769539ea5b078a53bccc806f614914b/4769539ea5b078a53bccc806f614914b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347191.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/77c/7a76ee61f8cd601f78f48b6bca7da77c/7a76ee61f8cd601f78f48b6bca7da77c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347191.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/77c/7a76ee61f8cd601f78f48b6bca7da77c/7a76ee61f8cd601f78f48b6bca7da77c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347193.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/079/3f6d98fe9831a9511c49d0c344f27079/3f6d98fe9831a9511c49d0c344f27079.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347193.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/079/3f6d98fe9831a9511c49d0c344f27079/3f6d98fe9831a9511c49d0c344f27079.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347195.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ada/b7a8e1317dc5fc5adf23a2b73dad2ada/b7a8e1317dc5fc5adf23a2b73dad2ada.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347195.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ada/b7a8e1317dc5fc5adf23a2b73dad2ada/b7a8e1317dc5fc5adf23a2b73dad2ada.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347197.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1bb/34b15d57ad92b60dff1c9f9daa7651bb/34b15d57ad92b60dff1c9f9daa7651bb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347197.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/1bb/34b15d57ad92b60dff1c9f9daa7651bb/34b15d57ad92b60dff1c9f9daa7651bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347199.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/65f/b216fed295bf092710bd07ed6dfc065f/b216fed295bf092710bd07ed6dfc065f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347199.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/65f/b216fed295bf092710bd07ed6dfc065f/b216fed295bf092710bd07ed6dfc065f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347201.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6da/5ddf682320c190bdb9ce673e929bb6da/5ddf682320c190bdb9ce673e929bb6da.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347201.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/6da/5ddf682320c190bdb9ce673e929bb6da/5ddf682320c190bdb9ce673e929bb6da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347203.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4aa/e9f7890b43e9758a60e0fa08dccd24aa/e9f7890b43e9758a60e0fa08dccd24aa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347203.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4aa/e9f7890b43e9758a60e0fa08dccd24aa/e9f7890b43e9758a60e0fa08dccd24aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347205.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ade/701daeeb47c391acdaa3cd54396e2ade/701daeeb47c391acdaa3cd54396e2ade.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges ... ok swift: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c" 507 - "-" "-" "-" 0.0004 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3610691 0 test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path ... ok swift: - - - [02/Jun/2024:16:50:29 +0000] "POST /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "POST /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c" 507 - "-" "-" "-" 0.0005 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0215 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:29 +0000] "UPDATE /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0010 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c" 507 - "-" "-" "-" 0.0006 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:29 +0000] "POST /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "POST /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c" 507 - "-" "-" "-" 0.0004 "-" 3610691 - test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file ... ok swift: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0431 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:29 +0000] "UPDATE /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:29 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0008 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 405 - "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete ... ok 2 RLock(s) were not greened, to fix this error make sure you run eventlet.monkey_patch() before importing any other modules. /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0164 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0011 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0147 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0041 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0385 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0009 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0013 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0006 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0005 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0009 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0018 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0005 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0014 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0005 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0006 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0005 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0013 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0010 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0005 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0005 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0005 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0008 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0006 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0006 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0019 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0013 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0305 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0034 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0032 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0018 "-" 3610637 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0043 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0214 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0015 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0013 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0150 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0206 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0012 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0011 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0174 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0095 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0042 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0036 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0035 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0011 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error Problem occurred while reading builder file: /tmp/tmp14u25_xu/tmpctxqrpiq. Generic error occurred Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sdb_"" with 1.0 weight got id 4 Device d5r1z1-127.0.0.1:6200R127.0.0.1:6200/sdc_"" with 1.0 weight got id 5 Reassigned 192 (300.00%) partitions. Balance is now 109.38. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 109.38 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 214.58. Dispersion is now 1.04 ------------------------------------------------------------------------------- NOTE: Dispersion of 1.041667 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Note: using /tmp/tmpbr4din5g/tmp80e2_wi1.builder instead of /tmp/tmpbr4din5g/tmp80e2_wi1.ring.gz as builder file Reassigned 3 (4.69%) partitions. Balance is now 57.29. Dispersion is now 0.52 ------------------------------------------------------------------------------- NOTE: Dispersion of 0.520833 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 24 (37.50%) partitions. Balance is now 1.04. Dispersion is now 0.00 Note: using /tmp/tmp4cbyo5pp/tmpoeeleh0d.builder instead of /tmp/tmp4cbyo5pp/tmpoeeleh0d.ring.gz as builder file Note: using /tmp/tmp0lczx9sb/tmp35_wouhg.builder instead of /tmp/tmp0lczx9sb/tmp35_wouhg.ring.gz as builder file Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/service.py", line 35, in GET resp = req.get_response(self.app, query={'format': 'json'}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. Unable to locate config number 5 for account-server Unable to locate config number 5 for object-server Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 1 Signal object-replicator pid: 11 signal: 15 Signal object-replicator pid: 12 signal: 15 Signal proxy-server pid: 1 signal: 1 output Removing pid file /tmp/tmpz13cgq8o/var/run/one-server.pid with wrong pid 3 output Removing pid file /tmp/tmpz13cgq8o/var/run/zero-server.pid with invalid pid output Removing pid file /tmp/tmpz13cgq8o/var/run/invalid-server.pid with invalid pid Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 3 signal: 15 test WARNING: Skipping /tmp/tmplurqxnp_/devices3/device_is_file/data because [Errno 20] Not a directory: '/tmp/tmplurqxnp_/devices3/device_is_file/data' test WARNING: Skipping /tmp/tmplurqxnp_/devices4/device/data because test WARNING: Skipping device as it is not mounted test WARNING: Skipping /tmp/tmp4pz6cmpc/asdf/data because [Errno 20] Not a directory: '/tmp/tmp4pz6cmpc/asdf/data' test WARNING: Skipping /tmp/tmp4pz6cmpc/asdf/data because [Errno 20] Not a directory: '/tmp/tmp4pz6cmpc/asdf/data' test WARNING: Skipping asdf as it is not mounted test WARNING: Skipping drive as it is not mounted test WARNING: Skipping /tmp/tmplg4kxuuf/asdf/data because [Errno 20] Not a directory: '/tmp/tmplg4kxuuf/asdf/data' test ERROR: Problem finding shard upper in '/tmp/tmpi8a8gqss/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpi8a8gqss/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmppca5vuwb/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmppca5vuwb/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging ... ok test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init test.unit.obj.test_diskfile.TestECDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx685108eca87c4dad9971c-00665ca2aa) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4396ab7096834e21bb5e1-00665ca2aa) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4396ab7096834e21bb5e1-00665ca2aa) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4396ab7096834e21bb5e1-00665ca2aa) test ERROR: Account HEAD returning 503 for [] (txn: tx4396ab7096834e21bb5e1-00665ca2aa) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txe0856a33ade8467ab2e39-00665ca2aa) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txe0856a33ade8467ab2e39-00665ca2aa) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txe0856a33ade8467ab2e39-00665ca2aa) test ERROR: Account HEAD returning 503 for [] (txn: txe0856a33ade8467ab2e39-00665ca2aa) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx0edb7b398a304331a18ff-00665ca2aa) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txf93d92bc5fd741b8b5f42-00665ca2aa) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txf93d92bc5fd741b8b5f42-00665ca2aa) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf93d92bc5fd741b8b5f42-00665ca2aa) test ERROR: Account HEAD returning 503 for [] (txn: txf93d92bc5fd741b8b5f42-00665ca2aa) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx52d82a2861764b7baf42d-00665ca2aa) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx52d82a2861764b7baf42d-00665ca2aa) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx52d82a2861764b7baf42d-00665ca2aa) test ERROR: Account HEAD returning 503 for [] (txn: tx52d82a2861764b7baf42d-00665ca2aa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txdeca9f4c5391420496441-00665ca2aa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: txdd93a61250824ff7a8045-00665ca2aa) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: txdd93a61250824ff7a8045-00665ca2aa) test INFO: Client disconnected on read of '/a/c' (txn: txdd93a61250824ff7a8045-00665ca2aa) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx1d2aaeee3c33406e89afd-00665ca2aa) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx6ee143a5487e4d64a218b-00665ca2aa) test INFO: Client disconnected on read of '/a/c' (txn: tx6ee143a5487e4d64a218b-00665ca2aa) test INFO: Client disconnected on read of '/a/c/' (txn: tx6ee143a5487e4d64a218b-00665ca2aa) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx0c6b5cea093a41b791db0-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx0c6b5cea093a41b791db0-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx0c6b5cea093a41b791db0-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx0c6b5cea093a41b791db0-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx3f751b79879c4b62a0ac7-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx3f751b79879c4b62a0ac7-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx3f751b79879c4b62a0ac7-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx3f751b79879c4b62a0ac7-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx74dd07ce2f3b4ffd9a1e5-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx74dd07ce2f3b4ffd9a1e5-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx74dd07ce2f3b4ffd9a1e5-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx74dd07ce2f3b4ffd9a1e5-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx4811ffaf4cd14ddb9e63c-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx4811ffaf4cd14ddb9e63c-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx4811ffaf4cd14ddb9e63c-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx4811ffaf4cd14ddb9e63c-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx0ab035d8b56c4cb79b748-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx0ab035d8b56c4cb79b748-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx0ab035d8b56c4cb79b748-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx0ab035d8b56c4cb79b748-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx0dca855e794b4e0ea2197-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx0dca855e794b4e0ea2197-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx0dca855e794b4e0ea2197-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx0dca855e794b4e0ea2197-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx1ed334891b60439db05fe-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx1ed334891b60439db05fe-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx1ed334891b60439db05fe-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx1ed334891b60439db05fe-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx61da21b567374ea3b354a-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx61da21b567374ea3b354a-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx61da21b567374ea3b354a-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx61da21b567374ea3b354a-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txf7f0ab241df94bc690b39-00665ca2af) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: txf7f0ab241df94bc690b39-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txf7f0ab241df94bc690b39-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf7f0ab241df94bc690b39-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf7f0ab241df94bc690b39-00665ca2af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txf7f0ab241df94bc690b39-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx822ac907e7dc416ea7403-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx822ac907e7dc416ea7403-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx822ac907e7dc416ea7403-00665ca2af) test ERROR: Container GET returning 503 for [503, 503] (txn: tx822ac907e7dc416ea7403-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test ERROR: Container GET returning 503 for [503, 503] (txn: tx3e29610b203b4eb6b6b09-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx1983668aac834dcebf134-00665ca2af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx1983668aac834dcebf134-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx1983668aac834dcebf134-00665ca2af) test ERROR: Container GET returning 503 for [] (txn: tx1983668aac834dcebf134-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx96963edaae6f4567b8991-00665ca2af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx96963edaae6f4567b8991-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx96963edaae6f4567b8991-00665ca2af) test ERROR: Container GET returning 503 for [] (txn: tx96963edaae6f4567b8991-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txb6aa9cb2de6946a8ae21a-00665ca2af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txb6aa9cb2de6946a8ae21a-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txb6aa9cb2de6946a8ae21a-00665ca2af) test ERROR: Container GET returning 503 for [] (txn: txb6aa9cb2de6946a8ae21a-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx24f94a8d7baf450ebfb03-00665ca2af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx24f94a8d7baf450ebfb03-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx24f94a8d7baf450ebfb03-00665ca2af) test ERROR: Container GET returning 503 for [] (txn: tx24f94a8d7baf450ebfb03-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx846d8f3492f04a679b375-00665ca2af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx846d8f3492f04a679b375-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx846d8f3492f04a679b375-00665ca2af) test ERROR: Container GET returning 503 for [] (txn: tx846d8f3492f04a679b375-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txf6b2ecf69567412f836c0-00665ca2af) test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test ERROR: Failed to unlink '/tmp/tmp17xu0of1/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp17xu0of1/containers/part/suffix/hash/container_1717347032.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp17xu0of1/containers/part/suffix/hash/container_1717347032.00000.db': ['/tmp/tmp17xu0of1/containers/part/suffix/hash/container.db', '/tmp/tmp17xu0of1/containers/part/suffix/hash/container_1717347033.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp17xu0of1/containers/part/suffix/hash/container_1717347033.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test INFO: - - - [02/Jun/2024:16:49:51 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0043 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0085 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0075 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0038 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0072 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0027 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0041 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0039 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0051 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0055 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0071 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0054 "-" 3610691 0 test DEBUG: Unlinked retiring db '/tmp/tmpck3yswn2/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0047 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0091 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0054 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0048 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0054 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0050 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0045 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0042 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0041 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0083 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0047 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0048 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:49:54 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0049 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0180 "-" 3610691 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3610691 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:49:58 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0326 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestContainerController.test_PUT_account_update test.unit.container.test_server.TestContainerController.test_PUT_account_update ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone ... ok test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx76e64160525f40f4ab31d-00665ca2c2) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx78c18b2f53bb472688797-00665ca2c2) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: tx4144a880ca9c4c108564d-00665ca2c2) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txe82b666e877f422f9f881-00665ca2c2) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx202bccf58f5a4172985c3-00665ca2c2) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd (txn: txcc202192ebe943a7a39e0-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: tx8f16458de5574117beaeb-00665ca2c2) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx1cc7f3bdb1364a5a8be31-00665ca2c2) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx9bcaa218f3e54f21bc02f-00665ca2c2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx799f2b3e97d641dab525d-00665ca2c2) test INFO: Client disconnected on read of '/a/c' (txn: tx7134ffadcaf3402f975ca-00665ca2c2) test INFO: Client disconnected on read of '/a/c' (txn: txdbfeb8b7ac1f4f4598906-00665ca2c2) test INFO: Client disconnected on read of '/a/c' (txn: txdbfeb8b7ac1f4f4598906-00665ca2c2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Use tempauth middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Use of the auth_admin_prefix, auth_host, auth_port, auth_protocol, identity_uri, admin_token, admin_user, admin_password, and admin_tenant_name configuration options was deprecated in the Mitaka release in favor of an auth_plugin and its related options. This class may be removed in a future release. Configuring admin URI using auth fragments was deprecated in the Kilo release, and will be removed in the Newton release, use 'identity_uri' instead. Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint test DEBUG: Calling S3Api Middleware Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_container test.unit.common.middleware.test_bulk.TestUntar.test_bad_container ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar ... ok test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required ... ok test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_call test.unit.common.middleware.test_bulk.TestUntar.test_extract_call ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works ... ok test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination ... ok test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_del test.unit.common.test_swob.TestHeaderEnvironProxy.test_del ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy ... ok test.unit.common.test_utils.TestClosingIterator.test_close test.unit.common.test_utils.TestClosingIterator.test_close ... ok test.unit.common.test_utils.TestClosingIterator.test_close_gen test.unit.common.test_utils.TestClosingIterator.test_close_gen ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_exception test.unit.common.test_utils.TestClosingIterator.test_close_on_exception ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration ... ok test.unit.common.test_utils.TestClosingIterator.test_close_others test.unit.common.test_utils.TestClosingIterator.test_close_others ... ok test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter ... ok test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable ... ok test.unit.common.test_utils.TestClosingIterator.test_nested_iters test.unit.common.test_utils.TestClosingIterator.test_nested_iters ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads ... ok test.unit.common.test_utils.TestNamespace.test_bounds_serialization test.unit.common.test_utils.TestNamespace.test_bounds_serialization ... ok test.unit.common.test_utils.TestNamespace.test_comparisons test.unit.common.test_utils.TestNamespace.test_comparisons ... ok test.unit.common.test_utils.TestNamespace.test_contains test.unit.common.test_utils.TestNamespace.test_contains ... ok test.unit.common.test_utils.TestNamespace.test_end_marker test.unit.common.test_utils.TestNamespace.test_end_marker ... ok test.unit.common.test_utils.TestNamespace.test_entire_namespace test.unit.common.test_utils.TestNamespace.test_entire_namespace ... ok test.unit.common.test_utils.TestNamespace.test_expand test.unit.common.test_utils.TestNamespace.test_expand ... ok test.unit.common.test_utils.TestNamespace.test_includes test.unit.common.test_utils.TestNamespace.test_includes ... ok test.unit.common.test_utils.TestNamespace.test_lower_setter test.unit.common.test_utils.TestNamespace.test_lower_setter ... ok test.unit.common.test_utils.TestNamespace.test_name test.unit.common.test_utils.TestNamespace.test_name ... ok test.unit.common.test_utils.TestNamespace.test_name_unexpected_format test.unit.common.test_utils.TestNamespace.test_name_unexpected_format ... ok test.unit.common.test_utils.TestNamespace.test_total_ordering test.unit.common.test_utils.TestNamespace.test_total_ordering ... ok test.unit.common.test_utils.TestNamespace.test_unicode_name test.unit.common.test_utils.TestNamespace.test_unicode_name ... ok test.unit.common.test_utils.TestNamespace.test_upper_setter test.unit.common.test_utils.TestNamespace.test_upper_setter ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes ... ok test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency test.unit.container.test_reconciler.TestReconciler.test_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency_config test.unit.container.test_reconciler.TestReconciler.test_concurrency_config ... ok test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi ... ok test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward ... ok test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse ... ok test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key ... ok test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0014 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0012 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0323 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0178 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0235 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0021 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0161 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0058 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:04 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0108 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0170 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0192 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0105 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0033 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0033 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmpv_z03x__/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0143 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0143 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0029 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0217 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0076 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0261 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0183 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0183 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0036 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0089 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0336 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0036 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0204 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:04 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0026 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission ... ok test INFO: - - - [02/Jun/2024:16:49:58 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0137 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0226 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0027 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0379 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.6384 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:07 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0237 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0011 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:07 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0195 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:07 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0057 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0194 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0204 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0188 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0213 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0164 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:07 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0104 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0139 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0254 "-" 3610691 - test ERROR: ERROR Account update failed with 127.0.0.1:57237/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0121 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0296 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0023 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:07 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:07 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0006 "-" 3610691 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0221 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0004 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0105 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0178 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0158 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0210 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:08 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0179 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0026 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0027 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0029 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0218 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found ... ok swift: - - - [02/Jun/2024:16:50:42 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0039 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0046 "-" 3610691 0 swift: - - - [02/Jun/2024:16:50:42 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0090 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:42 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0102 "-" 3610691 - swift: - - - [02/Jun/2024:16:50:42 +0000] "OPTIONS /sda1/p/a/c" 200 - "-" "-" "-" 0.0001 "-" 3610691 - test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txf6b2ecf69567412f836c0-00665ca2af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txf6b2ecf69567412f836c0-00665ca2af) test ERROR: Container GET returning 503 for [] (txn: txf6b2ecf69567412f836c0-00665ca2af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa38eaf9bff1b44b395bed-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa38eaf9bff1b44b395bed-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa38eaf9bff1b44b395bed-00665ca2b3) test ERROR: Container GET returning 503 for [] (txn: txa38eaf9bff1b44b395bed-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx91a01c228c434deeaa11f-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx91a01c228c434deeaa11f-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx91a01c228c434deeaa11f-00665ca2b3) test ERROR: Container GET returning 503 for [] (txn: tx91a01c228c434deeaa11f-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd6da5271b3b54761acf4f-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd6da5271b3b54761acf4f-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd6da5271b3b54761acf4f-00665ca2b3) test ERROR: Container GET returning 503 for [] (txn: txd6da5271b3b54761acf4f-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd7d69dc92d264b8bafbe7-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd7d69dc92d264b8bafbe7-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd7d69dc92d264b8bafbe7-00665ca2b3) test ERROR: Container GET returning 503 for [] (txn: txd7d69dc92d264b8bafbe7-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx31d36fa2ac1f49f39d617-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx31d36fa2ac1f49f39d617-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx31d36fa2ac1f49f39d617-00665ca2b3) test ERROR: Container GET returning 503 for [] (txn: tx31d36fa2ac1f49f39d617-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txc4d8c177cf5e417fa0252-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txc4d8c177cf5e417fa0252-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txc4d8c177cf5e417fa0252-00665ca2b3) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txc4d8c177cf5e417fa0252-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txffbec8144e4647f4b78b2-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txffbec8144e4647f4b78b2-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txffbec8144e4647f4b78b2-00665ca2b3) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txffbec8144e4647f4b78b2-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx5138a5448d3a46d79722c-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx5138a5448d3a46d79722c-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx5138a5448d3a46d79722c-00665ca2b3) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx5138a5448d3a46d79722c-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txfd822708d346489785e46-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txfd822708d346489785e46-00665ca2b3) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txfd822708d346489785e46-00665ca2b3) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txfd822708d346489785e46-00665ca2b3) test INFO: Client disconnected on read of '/a/c' (txn: tx1b194ff33c564d7faec24-00665ca2b3) test INFO: Client disconnected on read of '/a/c/' (txn: tx1b194ff33c564d7faec24-00665ca2b3) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txdf49cf2360fa400c86085-00665ca2b3) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txdf49cf2360fa400c86085-00665ca2b3) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txdf49cf2360fa400c86085-00665ca2b3) test ERROR: Account HEAD returning 503 for [] (txn: txdf49cf2360fa400c86085-00665ca2b3) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx815649d505a64b4ba487a-00665ca2b3) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx815649d505a64b4ba487a-00665ca2b3) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx815649d505a64b4ba487a-00665ca2b3) test ERROR: Account HEAD returning 503 for [] (txn: tx815649d505a64b4ba487a-00665ca2b3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txd344cd9e2a4e44c1a3ab9-00665ca2b8) test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults ... ok test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716681600 test DEBUG: looking for objects in 1716681600 test DEBUG: checking container 1716678000 test DEBUG: looking for objects in 1716678000 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'something', 'last_modified': '1970-01-01T00:00:00.000000', 'hash': '0.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test ERROR: Error listing containers in account .misplaced_objects (Unexpected response: 503 Service Unavailable) test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: ppi_skip: Container policy (0) in the middle of a part power increase (PPI) test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'ppi_skip': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'bogus', 'last_modified': '1970-01-01T01:00:18.841870', 'hash': '3618.841870', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': '00000bogus', 'last_modified': '1970-01-01T01:00:00.000000', 'hash': '3600.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'invalid_record': 1, 'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1717340400 test DEBUG: looking for objects in 1717340400 test DEBUG: checking container 1717336800 test DEBUG: looking for objects in 1717336800 test DEBUG: checking container 1717333200 test DEBUG: looking for objects in 1717333200 test DEBUG: checking container 1717329600 test DEBUG: looking for objects in 1717329600 test DEBUG: checking container 1717326000 test DEBUG: looking for objects in 1717326000 test DEBUG: checking container 1717322400 test DEBUG: looking for objects in 1717322400 test DEBUG: checking container 1717318800 test DEBUG: looking for objects in 1717318800 test DEBUG: checking container 1717315200 test DEBUG: looking for objects in 1717315200 test DEBUG: checking container 1717311600 test DEBUG: looking for objects in 1717311600 test DEBUG: checking container 1716739200 test DEBUG: looking for objects in 1716739200 test DEBUG: checking container 1716735600 test DEBUG: looking for objects in 1716735600 test DEBUG: checking container 1716732000 test DEBUG: looking for objects in 1716732000 test DEBUG: checking container 1716728400 test DEBUG: looking for objects in 1716728400 test DEBUG: checking container 1716724800 test DEBUG: looking for objects in 1716724800 test DEBUG: checking container 1716721200 test DEBUG: looking for objects in 1716721200 test DEBUG: checking container 1716717600 test DEBUG: looking for objects in 1716717600 test DEBUG: checking container 1716714000 test DEBUG: looking for objects in 1716714000 test DEBUG: checking container 1716710400 test DEBUG: looking for objects in 1716710400 test DEBUG: checking container 1716706800 test DEBUG: looking for objects in 1716706800 test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test.unit.container.test_reconciler.TestReconciler.test_no_concurrency test.unit.container.test_reconciler.TestReconciler.test_no_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_object_delete test.unit.container.test_reconciler.TestReconciler.test_object_delete ... ok test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move test.unit.container.test_reconciler.TestReconciler.test_object_move ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry ... ok test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Setting object count to 1072 (txn: tx7e746eb84e774229b853a-00665ca2bf) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx8a47e457c6034ca5b8084-00665ca2c1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx8a47e457c6034ca5b8084-00665ca2c1) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8a47e457c6034ca5b8084-00665ca2c1) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx8a47e457c6034ca5b8084-00665ca2c1) test ERROR: Aborting listing from shards due to bad response: [404] (txn: tx8a47e457c6034ca5b8084-00665ca2c1) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx57c746d707304bae91d48-00665ca2c4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: txfdc8ec98f5c34417b6266-00665ca2c6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test DEBUG: Setting object count to 3120 (txn: tx66eab1d455d74a79bfabd-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test DEBUG: Setting object count to 3120 (txn: tx9872de7468b04c0ab3dc1-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test DEBUG: Setting object count to 3120 (txn: tx7b4e12cfc90d4dd6801de-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test DEBUG: Setting object count to 3120 (txn: txf6abf64a648e4f8c95d9b-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test DEBUG: Setting object count to 3120 (txn: tx6adaa0623ead45a2a0f60-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx892c7d18fcab4f449ea85-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9234bee939174f399d862-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx9234bee939174f399d862-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user ... ok test INFO: - - - [02/Jun/2024:16:50:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0578 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:09 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0128 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:09 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0033 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:09 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:14 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 2.0512 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 1.4167 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.6419 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0043 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0072 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0098 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0109 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0174 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0328 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0047 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0036 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0044 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.5255 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0006 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0004 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0005 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0168 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0131 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0182 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0088 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0183 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0166 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0155 "-" 3610637 - test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '462'} test DEBUG: Grant test:full_control FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:10 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0007 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:15 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 2.0137 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:15 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.6615 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0173 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0032 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0016 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0041 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0019 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0022 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0019 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0017 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.1242 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0019 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:16 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0099 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0028 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0049 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0018 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0120 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0022 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0172 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0033 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0051 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0111 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0020 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0019 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0088 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0246 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0073 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0190 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0069 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0205 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0024 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0128 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0184 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0036 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0191 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx96f25042fb2446f7b093c-00665ca2bc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx4ed6872d8bbe48e484e48-00665ca2bc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpmwsqu3wg/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpmwsqu3wg/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpmwsqu3wg/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpmwsqu3wg/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0162 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0145 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0157 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0146 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx247c049f000042ee97597-00665ca2c0" "proxy-server 3610570" 0.0019 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:39011/sda1/1/a/c" "tx247c049f000042ee97597-00665ca2c0" "container-server 3610570" 0.0020 "-" 3610570 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:37205/sdb1/1/a/c" "tx247c049f000042ee97597-00665ca2c0" "container-server 3610570" 0.0027 "-" 3610570 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx247c049f000042ee97597-00665ca2c0" "proxy-server 3610570" 0.0456 "-" 3610570 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx247c049f000042ee97597-00665ca2c0" "proxy-server 3610570" 0.0256 "-" 3610570 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/50/08 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx247c049f000042ee97597-00665ca2c0 - 0.0589 - - 1717347008.899771929 1717347008.958673954 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:08 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txe7dc0a1231e4453792bf5-00665ca2c0" "proxy-server 3610570" 0.0019 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:39011/sda1/1/a1/c1" "txe7dc0a1231e4453792bf5-00665ca2c0" "container-server 3610570" 0.0016 "-" 3610570 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:37205/sdb1/1/a1/c1" "txe7dc0a1231e4453792bf5-00665ca2c0" "container-server 3610570" 0.0033 "-" 3610570 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe7dc0a1231e4453792bf5-00665ca2c0" "proxy-server 3610570" 0.0428 "-" 3610570 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txe7dc0a1231e4453792bf5-00665ca2c0" "proxy-server 3610570" 0.0241 "-" 3610570 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/50/09 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txe7dc0a1231e4453792bf5-00665ca2c0 - 0.0550 - - 1717347008.960794210 1717347009.015826225 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe94e7effa9fa4c8e85fc4-00665ca2c1" "proxy-server 3610570" 0.0099 "-" 3610570 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:37205/sdb1/2/a/c1" "txe94e7effa9fa4c8e85fc4-00665ca2c1" "container-server 3610570" 0.0015 "-" 3610570 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:39011/sda1/2/a/c1" "txe94e7effa9fa4c8e85fc4-00665ca2c1" "container-server 3610570" 0.0115 "-" 3610570 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe94e7effa9fa4c8e85fc4-00665ca2c1" "proxy-server 3610570" 0.0419 "-" 3610570 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe94e7effa9fa4c8e85fc4-00665ca2c1" "proxy-server 3610570" 0.0604 "-" 3610570 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/50/09 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txe94e7effa9fa4c8e85fc4-00665ca2c1 - 0.0799 - - 1717347009.017747402 1717347009.097635984 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1e4153c9dec74a03938a3-00665ca2c1" "proxy-server 3610570" 0.0167 "-" 3610570 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:39011/sda1/1/a/c2" "tx1e4153c9dec74a03938a3-00665ca2c1" "container-server 3610570" 0.0016 "-" 3610570 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:37205/sdb1/1/a/c2" "tx1e4153c9dec74a03938a3-00665ca2c1" "container-server 3610570" 0.0135 "-" 3610570 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx1e4153c9dec74a03938a3-00665ca2c1" "proxy-server 3610570" 0.0392 "-" 3610570 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:50:09 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx1e4153c9dec74a03938a3-00665ca2c1" "proxy-server 3610570" 0.0606 "-" 3610570 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/50/09 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx1e4153c9dec74a03938a3-00665ca2c1 - 0.0867 - - 1717347009.099646807 1717347009.186344147 2 test.unit.proxy.test_server.TestContainerController.test_CORS_valid test.unit.proxy.test_server.TestContainerController.test_CORS_valid ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE test.unit.proxy.test_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_GET_no_content test.unit.proxy.test_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_OPTIONS test.unit.proxy.test_server.TestContainerController.test_OPTIONS ... ok test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347024.944660) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347024.944660) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347024.944660) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347024.944660) from storage policy 1 test DEBUG: delete_attempt: '/AUTH_bob/c/o1' (1717347024.944660) in policy_index 1 will be deleted from policy_index 0 (1717347024.94466_0000000000000002) test INFO: delete_success: '/AUTH_bob/c/o1' (1717347024.944660) deleted from policy_index 0 (1717347024.94466_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (1717347024.944660) from policy_index 1 (1717347024.94466_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (1717347024.944660) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (1717347024.944660) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1717347024.944660) from the queue (1717347024.94466_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'delete_attempt': 1, 'delete_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3679.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3680.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383930) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383930) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (36123.383930) moved from policy_index 1 to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 (0000036123.38393_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (36123.383930) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 was handled successfully test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission ... ok test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission ... ok test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx29924fbfa9784210bffa4-00665ca2c9) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx29924fbfa9784210bffa4-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx737d0cfaaa804b19a314a-00665ca2c9) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx737d0cfaaa804b19a314a-00665ca2c9) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx737d0cfaaa804b19a314a-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx737d0cfaaa804b19a314a-00665ca2c9) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx737d0cfaaa804b19a314a-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txc0820b4195ac44af87b39-00665ca2c9) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txc0820b4195ac44af87b39-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc0820b4195ac44af87b39-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txc0820b4195ac44af87b39-00665ca2c9) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txc0820b4195ac44af87b39-00665ca2c9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx71247038216a4943b5180-00665ca2c9) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx71247038216a4943b5180-00665ca2c9) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx71247038216a4943b5180-00665ca2c9) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx71247038216a4943b5180-00665ca2c9) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx71247038216a4943b5180-00665ca2c9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1e202c267f1f400b91e08-00665ca2cc) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx1e202c267f1f400b91e08-00665ca2cc) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: tx1e202c267f1f400b91e08-00665ca2cc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txbab90b9da38746aeb47d4-00665ca2cf) test DEBUG: Setting object count to 1065 (txn: txbab90b9da38746aeb47d4-00665ca2cf) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test INFO: - - 02/Jun/2024/16/49/58 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717346998.430616617 1717346998.431086540 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0016 - - 1717346998.430492163 1717346998.432106495 0 test INFO: - - 02/Jun/2024/16/49/58 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717346998.434866667 1717346998.435325384 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0016 - - 1717346998.434704781 1717346998.436284304 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717346998.439150333 1717346998.439538956 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0014 - - 1717346998.439058065 1717346998.440507650 0 test INFO: - - 02/Jun/2024/16/49/58 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0006 FA - 1717346998.444289446 1717346998.444886684 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0030 - - 1717346998.443007946 1717346998.446031332 0 test INFO: - - 02/Jun/2024/16/49/58 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717346998.448763847 1717346998.449110031 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0015 - - 1717346998.448664427 1717346998.450209856 0 test INFO: - - 02/Jun/2024/16/49/58 HEAD /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717346998.453815460 1717346998.454211473 0 test INFO: - - 02/Jun/2024/16/49/58 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0003 SSC - 1717346998.454836130 1717346998.455139399 0 test INFO: - - 02/Jun/2024/16/49/58 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717346998.455911636 1717346998.456171274 0 test INFO: - - 02/Jun/2024/16/49/58 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0035 - - 1717346998.453536987 1717346998.457008839 - test INFO: - - 02/Jun/2024/16/49/58 GET /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717346998.459510088 1717346998.459922314 0 test INFO: - - 02/Jun/2024/16/49/58 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717346998.460570335 1717346998.461038589 0 test INFO: - - 02/Jun/2024/16/49/58 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717346998.461946726 1717346998.462415934 0 test INFO: - - 02/Jun/2024/16/49/58 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0042 - - 1717346998.459234715 1717346998.463387251 - test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0008 FA - 1717346998.465855122 1717346998.466666460 0 test INFO: - - 02/Jun/2024/16/49/58 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717346998.467406034 1717346998.467920542 0 test INFO: - - 02/Jun/2024/16/49/58 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717346998.468788624 1717346998.469160557 0 test INFO: - - 02/Jun/2024/16/49/58 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0047 - - 1717346998.465617895 1717346998.470322371 - test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values ... ok test.unit.common.test_db.TestDictFactory.test_normal_case test.unit.common.test_db.TestDictFactory.test_normal_case ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_gen_headers test.unit.common.test_direct_client.TestDirectClient.test_gen_headers ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry test.unit.common.test_direct_client.TestDirectClient.test_retry ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception ... ok test.unit.common.test_swob.TestAccept.test_accept_invalid test.unit.common.test_swob.TestAccept.test_accept_invalid ... ok test.unit.common.test_swob.TestAccept.test_accept_json test.unit.common.test_swob.TestAccept.test_accept_json ... ok test.unit.common.test_swob.TestAccept.test_accept_plain test.unit.common.test_swob.TestAccept.test_accept_plain ... ok test.unit.common.test_swob.TestAccept.test_accept_xml test.unit.common.test_swob.TestAccept.test_accept_xml ... ok test.unit.common.test_swob.TestAccept.test_repr test.unit.common.test_swob.TestAccept.test_repr ... ok test.unit.common.test_swob.TestRange.test_range test.unit.common.test_swob.TestRange.test_range ... ok test.unit.common.test_swob.TestRange.test_range_invalid_syntax test.unit.common.test_swob.TestRange.test_range_invalid_syntax ... ok test.unit.common.test_swob.TestRange.test_ranges_for_large_length test.unit.common.test_swob.TestRange.test_ranges_for_large_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length test.unit.common.test_swob.TestRange.test_ranges_for_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_edges test.unit.common.test_swob.TestRange.test_ranges_for_length_edges ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_multi test.unit.common.test_swob.TestRange.test_ranges_for_length_multi ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many ... ok test.unit.common.test_swob.TestRange.test_str test.unit.common.test_swob.TestRange.test_str ... ok test.unit.common.test_swob.TestRange.test_upsidedown_range test.unit.common.test_swob.TestRange.test_upsidedown_range ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_percent test.unit.common.test_utils.TestFsHasFreeSpace.test_percent ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf8b13fc785cc43ab89bd1-00665ca2ae) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0401ca4e875945af90c16-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx995f3d3e3d904bf49c1b0-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cd25bef89094e6e993b5-00665ca2ae) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission ... ok test INFO: - - 02/Jun/2024/16/49/58 PUT /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717346998.473488569 1717346998.473809958 0 test INFO: - - 02/Jun/2024/16/50/06 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0005 SSC - 1717347006.782051563 1717347006.782520294 0 test INFO: - - 02/Jun/2024/16/50/06 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347006.783398390 1717347006.783737659 0 test INFO: - - 02/Jun/2024/16/50/06 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 8.3113 - - 1717346998.473205328 1717347006.784549236 - test INFO: - - 02/Jun/2024/16/50/06 DELETE /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347006.787151575 1717347006.787413597 0 test INFO: - - 02/Jun/2024/16/50/06 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0004 SSC - 1717347006.787909031 1717347006.788278341 0 test INFO: - - 02/Jun/2024/16/50/06 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347006.788946390 1717347006.789197683 0 test INFO: - - 02/Jun/2024/16/50/06 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0031 - - 1717347006.786934853 1717347006.790002584 - direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! test1 STDOUT: test2 STDERR: test4 test5 STDERR: test6 test8 STDOUT: a#012b#012c STDOUT: d test.unit.common.test_utils.TestUtils.test_LoggerFileObject test.unit.common.test_utils.TestUtils.test_LoggerFileObject ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '446'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '450'} test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: txda1741984426427680fca-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txd7d0e268939845c986c01-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: tx13669a759f3f44df9ae28-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx13669a759f3f44df9ae28-00665ca2c3) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx13669a759f3f44df9ae28-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx35feddf8ae2e42cdabfa2-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx35feddf8ae2e42cdabfa2-00665ca2c3) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx35feddf8ae2e42cdabfa2-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx41fbf6189f594bd5b935c-00665ca2c3) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx41fbf6189f594bd5b935c-00665ca2c3) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx51074568c8934277aba6d-00665ca2c3) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx51074568c8934277aba6d-00665ca2c3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx94e54ef74b684cfe9248f-00665ca2c6) test INFO: Client disconnected on read of '/a/c' (txn: tx94e54ef74b684cfe9248f-00665ca2c6) test INFO: Client disconnected on read of '/a/c' (txn: tx8b664ccd25b14e7d9c9b3-00665ca2c6) test INFO: Client disconnected on read of '/a/c' (txn: tx8b664ccd25b14e7d9c9b3-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx8cc0e8425f404f8da9ac4-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx46110f50329d4be3b8f28-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx7ddf8598a2b244c7a25b5-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx7ddf8598a2b244c7a25b5-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx7ddf8598a2b244c7a25b5-00665ca2c6) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx7ddf8598a2b244c7a25b5-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx20df575796e4433f94385-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx20df575796e4433f94385-00665ca2c6) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx20df575796e4433f94385-00665ca2c6) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx20df575796e4433f94385-00665ca2c6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: txc1018389b73646fabec78-00665ca2c6) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: txc1018389b73646fabec78-00665ca2c6) test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.proxy.test_server.TestContainerController.test_POST test.unit.proxy.test_server.TestContainerController.test_POST ... ok test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_server.TestContainerController.test_POST_metadata test.unit.proxy.test_server.TestContainerController.test_POST_metadata ... ok I like ham STDERR: --- Logging error --- STDERR: Traceback (most recent call last): STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 845, in emit raise socket.error(errno.ENOTCONN, "This is an ex-syslog") STDERR: OSError: [Errno 107] This is an ex-syslog STDERR: Call stack: STDERR: File "", line 198, in _run_module_as_main STDERR: File "", line 88, in _run_code STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 88, in main() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 82, in main program.TestProgram( STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 197, in __init__ self.runTests() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 247, in runTests self.result = testRunner.run(self.test) STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 51, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.12/unittest/case.py", line 690, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.12/unittest/case.py", line 634, in run self._callTestMethod(testMethod) STDERR: File "/usr/lib/python3.12/unittest/case.py", line 589, in _callTestMethod if method() is not None: STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 877, in test_LoggerFileObject_recursion logger.info("I like ham") STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1539, in info self._log(INFO, msg, args, **kwargs) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1684, in _log self.handle(record) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1700, in handle self.callHandlers(record) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1762, in callHandlers hdlr.handle(record) STDERR: File "/usr/lib/python3.12/logging/__init__.py", line 1028, in handle self.emit(record) STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 847, in emit self.handleError(record) STDERR: Message: 'I like ham' Arguments: () test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion ... ok test.unit.common.test_utils.TestUtils.test_NullLogger test.unit.common.test_utils.TestUtils.test_NullLogger ... ok test.unit.common.test_utils.TestUtils.test_TRUE_VALUES test.unit.common.test_utils.TestUtils.test_TRUE_VALUES ... ok test.unit.common.test_utils.TestUtils.test_backwards test.unit.common.test_utils.TestUtils.test_backwards ... ok test.unit.common.test_utils.TestUtils.test_cache_from_env test.unit.common.test_utils.TestUtils.test_cache_from_env ... ok test.unit.common.test_utils.TestUtils.test_cap_length test.unit.common.test_utils.TestUtils.test_cap_length ... ok test.unit.common.test_utils.TestUtils.test_capture_stdio test.unit.common.test_utils.TestUtils.test_capture_stdio ... ok test.unit.common.test_utils.TestUtils.test_clean_content_type test.unit.common.test_utils.TestUtils.test_clean_content_type ... ok test.unit.common.test_utils.TestUtils.test_clean_logger_exception test.unit.common.test_utils.TestUtils.test_clean_logger_exception ... ok test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene ... ok test.unit.common.test_utils.TestUtils.test_config_auto_int_value test.unit.common.test_utils.TestUtils.test_config_auto_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_fallocate_value test.unit.common.test_utils.TestUtils.test_config_fallocate_value ... ok test.unit.common.test_utils.TestUtils.test_config_float_value test.unit.common.test_utils.TestUtils.test_config_float_value ... ok test.unit.common.test_utils.TestUtils.test_config_percent_value test.unit.common.test_utils.TestUtils.test_config_percent_value ... ok test.unit.common.test_utils.TestUtils.test_config_positive_int_value test.unit.common.test_utils.TestUtils.test_config_positive_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_request_node_count_value test.unit.common.test_utils.TestUtils.test_config_request_node_count_value ... ok test.unit.common.test_utils.TestUtils.test_config_true_value test.unit.common.test_utils.TestUtils.test_config_true_value ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close test.unit.common.test_utils.TestUtils.test_drain_and_close ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges test.unit.common.test_utils.TestUtils.test_drop_privileges ... ok test DEBUG: pop_queue: remove '/.misplaced_objects/36000/1:/AUTH_bob/c/o1' (36123.383930) from the queue (0000036123.38393_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.123460) in policy_index 1 will be moved to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.123460) moved from policy_index 1 to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_failed: '/AUTH_bob/c/o1' (3600.123456) was not cleaned up in storage_policy 1 (Unexpected response: 503 Service Unavailable) test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: unavailable_destination: '/AUTH_bob/c/o1' (3600.123456) unable to determine the destination timestamp, if any test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_destination': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716681600 test DEBUG: looking for objects in 1716681600 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1716681750.630418) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1716681750.630418) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1716681750.630418) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1716681750.630418) from storage policy 1 test CRITICAL: lost_source: '/AUTH_jeb/c/o1' (1716681750.63042) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_jeb/c/o1' (1716681750.630418) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/1716681600/1:/AUTH_jeb/c/o1' (1716681750.630418) from the queue (1716681750.63042_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1717347030.641935) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1717347030.641935) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1717347030.641935) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1717347030.641935) from storage policy 1 test WARNING: unavailable_source: '/AUTH_jeb/c/o1' (1717347030.64194) in policy_index 1 responded 404 (0000000000.00000) test INFO: retry: '/AUTH_jeb/c/o1' (1717347030.641935) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test ERROR: unhandled_error: unable to upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 635, in ensure_object_in_right_location self.swift.upload_object( File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 844, in upload_object self.handle_request('PUT', path, headers, acceptable_statuses, fobj, File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 264, in handle_request resp = self.make_request(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'unhandled_error': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache test.unit.common.test_utils.TestUtils.test_dump_recon_cache ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner ... ok test.unit.common.test_utils.TestUtils.test_extract_swift_bytes test.unit.common.test_utils.TestUtils.test_extract_swift_bytes ... ok test.unit.common.test_utils.TestUtils.test_find_namespace test.unit.common.test_utils.TestUtils.test_find_namespace ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_large_body test.unit.common.test_utils.TestUtils.test_friendly_close_large_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_small_body test.unit.common.test_utils.TestUtils.test_friendly_close_small_body ... ok test.unit.common.test_utils.TestUtils.test_fsync_dir test.unit.common.test_utils.TestUtils.test_fsync_dir ... ok test.unit.container.test_reconciler.TestReconciler.test_processes_config test.unit.container.test_reconciler.TestReconciler.test_processes_config ... ok test.unit.common.test_utils.TestUtils.test_generate_trans_id test.unit.common.test_utils.TestUtils.test_generate_trans_id ... ok test.unit.common.test_utils.TestUtils.test_get_db_files test.unit.common.test_utils.TestUtils.test_get_db_files ... ok test.unit.common.test_utils.TestUtils.test_get_log_line test.unit.common.test_utils.TestUtils.test_get_log_line ... ok test.unit.common.test_utils.TestUtils.test_get_logger test.unit.common.test_utils.TestUtils.test_get_logger ... ok test.unit.common.test_utils.TestUtils.test_get_logger_console test.unit.common.test_utils.TestUtils.test_get_logger_console ... ok test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route ... ok test.unit.common.test_utils.TestUtils.test_get_partition_for_hash test.unit.common.test_utils.TestUtils.test_get_partition_for_hash ... ok test.unit.common.test_utils.TestUtils.test_get_partition_from_path test.unit.common.test_utils.TestUtils.test_get_partition_from_path ... ok test.unit.common.test_utils.TestUtils.test_get_policy_index test.unit.common.test_utils.TestUtils.test_get_policy_index ... ok test.unit.common.test_utils.TestUtils.test_get_redirect_data test.unit.common.test_utils.TestUtils.test_get_redirect_data ... ok test.unit.common.test_utils.TestUtils.test_get_trans_id_time test.unit.common.test_utils.TestUtils.test_get_trans_id_time ... ok test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str ... ok test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string ... ok test.unit.common.test_utils.TestUtils.test_hash_path test.unit.common.test_utils.TestUtils.test_hash_path ... ok test.unit.common.test_utils.TestUtils.test_human_readable test.unit.common.test_utils.TestUtils.test_human_readable ... ok test.unit.common.test_utils.TestUtils.test_is_file_older test.unit.common.test_utils.TestUtils.test_is_file_older ... ok test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_error test.unit.common.test_utils.TestUtils.test_ismount_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_root test.unit.common.test_utils.TestUtils.test_ismount_path_is_root ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_dev test.unit.common.test_utils.TestUtils.test_ismount_successes_dev ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_ino test.unit.common.test_utils.TestUtils.test_ismount_successes_ino ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists ... ok test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists ... ok test ERROR: ERROR: swift.cache could not be found in env! test ERROR: ERROR: swift.cache could not be found in env! test WARNING: Unable to perform fsync() on directory /tmp/tmpusbwkqt8: Bad file descriptor Unable to locate some_not_real_function in libc. Leaving as a no-op. test.unit.common.test_utils.TestUtils.test_load_libc_function test.unit.common.test_utils.TestUtils.test_load_libc_function ... ok test.unit.common.test_utils.TestUtils.test_load_pkg_resource test.unit.common.test_utils.TestUtils.test_load_pkg_resource ... skipped 'pkg_resources loading is only available on python 3.7 and earlier' test.unit.common.test_utils.TestUtils.test_load_recon_cache test.unit.common.test_utils.TestUtils.test_load_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_lock_file test.unit.common.test_utils.TestUtils.test_lock_file ... ok test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink ... ok test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail ... ok test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open ... ok test.unit.common.test_utils.TestUtils.test_lock_path test.unit.common.test_utils.TestUtils.test_lock_path ... ok test.unit.common.test_utils.TestUtils.test_lock_path_class test.unit.common.test_utils.TestUtils.test_lock_path_class ... ok test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit ... ok test.unit.common.test_utils.TestUtils.test_lock_path_name test.unit.common.test_utils.TestUtils.test_lock_path_name ... ok test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps ... ok test.unit.common.test_utils.TestUtils.test_log_string_formatter test.unit.common.test_utils.TestUtils.test_log_string_formatter ... ok test.unit.common.test_utils.TestUtils.test_majority_size test.unit.common.test_utils.TestUtils.test_majority_size ... ok test.unit.common.test_utils.TestUtils.test_make_db_file_path test.unit.common.test_utils.TestUtils.test_make_db_file_path ... ok test.unit.common.test_utils.TestUtils.test_makedirs_count test.unit.common.test_utils.TestUtils.test_makedirs_count ... ok test.unit.common.test_utils.TestUtils.test_md5_with_data test.unit.common.test_utils.TestUtils.test_md5_with_data ... ok test.unit.common.test_utils.TestUtils.test_md5_without_data test.unit.common.test_utils.TestUtils.test_md5_without_data ... ok test.unit.common.test_utils.TestUtils.test_mkdirs test.unit.common.test_utils.TestUtils.test_mkdirs ... ok test.unit.common.test_utils.TestUtils.test_monkey_patch test.unit.common.test_utils.TestUtils.test_monkey_patch ... ok test.unit.common.test_utils.TestUtils.test_nested_prefixlogger test.unit.common.test_utils.TestUtils.test_nested_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_node_to_string test.unit.common.test_utils.TestUtils.test_node_to_string ... ok test.unit.common.test_utils.TestUtils.test_non_negative_float test.unit.common.test_utils.TestUtils.test_non_negative_float ... ok test.unit.common.test_utils.TestUtils.test_non_negative_int test.unit.common.test_utils.TestUtils.test_non_negative_int ... ok test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision ... ok test.unit.common.test_utils.TestUtils.test_normalize_timestamp test.unit.common.test_utils.TestUtils.test_normalize_timestamp ... ok test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_content_type test.unit.common.test_utils.TestUtils.test_parse_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_db_filename test.unit.common.test_utils.TestUtils.test_parse_db_filename ... ok test.unit.common.test_utils.TestUtils.test_parse_options test.unit.common.test_utils.TestUtils.test_parse_options ... ok test.unit.common.test_utils.TestUtils.test_parse_options_errors test.unit.common.test_utils.TestUtils.test_parse_options_errors ... ok test.unit.common.test_utils.TestUtils.test_parse_override_options test.unit.common.test_utils.TestUtils.test_parse_override_options ... ok test.unit.common.test_utils.TestUtils.test_prefixlogger test.unit.common.test_utils.TestUtils.test_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_quorum_size test.unit.common.test_utils.TestUtils.test_quorum_size ... ok test.unit.common.test_utils.TestUtils.test_quote_bytes test.unit.common.test_utils.TestUtils.test_quote_bytes ... ok test.unit.common.test_utils.TestUtils.test_quote_unicode test.unit.common.test_utils.TestUtils.test_quote_unicode ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep ... ok test.unit.common.test_utils.TestUtils.test_readconf test.unit.common.test_utils.TestUtils.test_readconf ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir test.unit.common.test_utils.TestUtils.test_readconf_dir ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files ... ok test.unit.common.test_utils.TestUtils.test_readconf_raw test.unit.common.test_utils.TestUtils.test_readconf_raw ... ok test.unit.common.test_utils.TestUtils.test_remove_directory test.unit.common.test_utils.TestUtils.test_remove_directory ... ok test.unit.common.test_utils.TestUtils.test_remove_file test.unit.common.test_utils.TestUtils.test_remove_file ... ok test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false ... ok test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_replace_partition_in_path test.unit.common.test_utils.TestUtils.test_replace_partition_in_path ... ok test.unit.common.test_utils.TestUtils.test_round_robin_iter test.unit.common.test_utils.TestUtils.test_round_robin_iter ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip ... ok test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation ... ok test.unit.common.test_utils.TestUtils.test_safe_json_loads test.unit.common.test_utils.TestUtils.test_safe_json_loads ... ok test.unit.common.test_utils.TestUtils.test_search_tree test.unit.common.test_utils.TestUtils.test_search_tree ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match ... ok test.unit.common.test_utils.TestUtils.test_select_node_ip test.unit.common.test_utils.TestUtils.test_select_node_ip ... ok test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0033 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0011 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0303 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0060 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0011 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0231 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0056 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0023 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0066 "-" 3610691 - test DEBUG: Unlinked retiring db '/tmp/tmpz60ng9zx/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0250 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0280 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0022 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0051 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0321 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0028 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0011 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0321 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0025 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0322 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0138 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0359 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0023 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0290 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0033 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0038 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0294 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0065 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:19 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0310 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.4640 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.4550 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0133 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.6872 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 1.8931 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.1937 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0127 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0070 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0297 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0527 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0202 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0685 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0200 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0387 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0291 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0322 "-" 3610691 0 test DEBUG: Unlinked retiring db '/tmp/tmpyddvecia/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0177 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0164 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT_utf8 test.unit.container.test_server.TestContainerController.test_PUT_utf8 ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works ... ok test.unit.common.test_utils.TestUtils.test_split_path test.unit.common.test_utils.TestUtils.test_split_path ... ok test.unit.common.test_utils.TestUtils.test_storage_directory test.unit.common.test_utils.TestUtils.test_storage_directory ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer test.unit.common.test_utils.TestUtils.test_str_anonymizer ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness ... ok test.unit.common.test_utils.TestUtils.test_str_format_time test.unit.common.test_utils.TestUtils.test_str_format_time ... ok test.unit.common.test_utils.TestUtils.test_streq_const_time test.unit.common.test_utils.TestUtils.test_streq_const_time ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestContainerController.test_UPDATE test.unit.container.test_server.TestContainerController.test_UPDATE ... ok test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestContainerController.test__call__raise_timeout test.unit.container.test_server.TestContainerController.test__call__raise_timeout ... ok test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device ... ok test.unit.container.test_server.TestContainerController.test_acl_container test.unit.container.test_server.TestContainerController.test_acl_container ... ok test.unit.common.test_utils.TestUtils.test_strict_b64decode test.unit.common.test_utils.TestUtils.test_strict_b64decode ... ok test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter test.unit.common.test_utils.TestUtils.test_swift_log_formatter ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length ... ok test.unit.common.test_utils.TestUtils.test_systemd_notify test.unit.common.test_utils.TestUtils.test_systemd_notify ... ok test.unit.common.test_utils.TestUtils.test_validate_device_partition test.unit.common.test_utils.TestUtils.test_validate_device_partition ... ok test.unit.common.test_utils.TestUtils.test_validate_hash_conf test.unit.common.test_utils.TestUtils.test_validate_hash_conf ... ok test.unit.common.test_utils.TestUtils.test_validate_sync_to test.unit.common.test_utils.TestUtils.test_validate_sync_to ... ok test.unit.common.test_utils.TestUtils.test_write_file test.unit.common.test_utils.TestUtils.test_write_file ... ok test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:49:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.11:1011/sdl, error: ERROR Insufficient Storage (txn: tx16c1264bdc1f4526ac4f0-00665ca2cf) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sdr (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sdt (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sdx (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sdq (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sdw (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sdz (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sdb (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sdv (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sdu (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sds (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sdy (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sdp (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sdo (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission ... ok test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0181 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0083 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0083 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0148 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0146 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0065 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0100 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0018 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0217 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0089 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0062 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0101 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0035 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0195 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0039 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0079 "-" 3610637 4 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0078 "-" 3610637 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0242 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0095 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0117 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0154 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.1556 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0344 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0777 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0059 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0199 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0375 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0059 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0036 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0528 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0516 "-" 3610637 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3610637 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0285 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0558 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0268 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0272 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0352 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0388 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0078 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:18 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0040 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.1602 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0041 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0113 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0036 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0066 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0044 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0070 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0043 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0040 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0079 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0033 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0074 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0037 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0033 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0032 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0057 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0028 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0050 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:19 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0048 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0036 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0043 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmpt86ox_g8/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0038 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0061 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0041 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.2595 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0050 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:23 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0051 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0050 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0053 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '456'} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:10 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '448'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range ... ok test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: txc1018389b73646fabec78-00665ca2c6) test ERROR: Account HEAD returning 503 for [] (txn: txc1018389b73646fabec78-00665ca2c6) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx84689c10f57d4c4290169-00665ca2cb) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx83124aaf06d14eb885bc8-00665ca2cb) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx83124aaf06d14eb885bc8-00665ca2cb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc173fc0ff9c94316abedf-00665ca2cb) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc49f970e29bd43c9b6f84-00665ca2cb) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbd9bcc06629d401095208-00665ca2cb) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txe335d8db0ca245f6ae39f-00665ca2cb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txf147714881484cf0a1e91-00665ca2cb) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txe7bf13f262b54e60ad73b-00665ca2cb) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txe7bf13f262b54e60ad73b-00665ca2cb) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test WARNING: Could not autocreate account '/account' (txn: tx5ac8382c1c19474c8d5a5-00665ca2cb) test INFO: autocreate account '/account' (txn: tx4b6edc0e3cec4b77aec7d-00665ca2cb) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test ERROR: Account HEAD returning 503 for [503, 503] (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test WARNING: Could not autocreate account '/account' (txn: tx03e2afdc262f41f1845c9-00665ca2cb) test INFO: autocreate account '/account' (txn: txe1263bbf9ab74edcb3885-00665ca2cb) test.unit.proxy.test_server.TestContainerController.test_PUT test.unit.proxy.test_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl ... ok test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test WARNING: copy_failed: upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 returned Unexpected response: 503 Service Unavailable test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1716739199.000000) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1716739199.000000) from storage policy 1 test CRITICAL: lost_source: '/AUTH_bob/c/o1' (1716739199.00000) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1716739199.000000) from the queue (1716739199.00000_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.234570) in policy_index 1 will be moved to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.234570) moved from policy_index 1 to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3600.123456) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3600.123456) from the queue (0000003600.12346_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347037.852508) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347037.852508) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347037.852508) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347037.852508) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717347037.85251) in policy_index 1 responded 200 (1717347036.85251) test INFO: retry: '/AUTH_bob/c/o1' (1717347037.852508) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 1 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 should be in policy_index 1 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 0 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 will be moved to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 0 to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 0 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 0 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/0:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c ت/o1 ت' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c ت/o1 ت' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c ت/o1 ت' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 should be in policy_index 0 test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone ... ok test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy ... ok test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile ... ok test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error ... ok test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0128 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0020 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0249 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0138 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:25 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:25 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:25 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0024 "-" 3610691 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:50:25 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0099 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:25 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0180 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0034 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0005 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0237 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0032 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0214 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0209 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0209 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0227 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0041 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0207 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0119 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0199 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0820 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0776 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0724 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0282 "-" 3610691 0 test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestContainerController.test_correct_allowed_method test.unit.container.test_server.TestContainerController.test_correct_allowed_method ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestContainerController.test_creation test.unit.container.test_server.TestContainerController.test_creation ... ok test.unit.container.test_server.TestContainerController.test_delete_auto_create test.unit.container.test_server.TestContainerController.test_delete_auto_create ... ok test.unit.container.test_server.TestContainerController.test_deleted_headers test.unit.container.test_server.TestContainerController.test_deleted_headers ... ok test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestContainerController.test_list_allowed_methods test.unit.container.test_server.TestContainerController.test_list_allowed_methods ... ok test.unit.container.test_server.TestContainerController.test_listing_with_reserved test.unit.container.test_server.TestContainerController.test_listing_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_log_line_format test.unit.container.test_server.TestContainerController.test_log_line_format ... ok test.unit.container.test_server.TestContainerController.test_markers_with_reserved test.unit.container.test_server.TestContainerController.test_markers_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_not_allowed_method test.unit.container.test_server.TestContainerController.test_not_allowed_method ... ok swift: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request#012 conn = http_connect(#012 ^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect#012 i, status = next(conn_id_and_code_iter)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^#012StopIteration (txn: txf5877c5a12064c36977e1-00665ca2f8) swift: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: #012Traceback (most recent call last):#012 File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request#012 conn = http_connect(#012 ^^^^^^^^^^^^^#012 File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect#012 i, status = next(conn_id_and_code_iter)#012 ^^^^^^^^^^^^^^^^^^^^^^^^^^^#012StopIteration (txn: txf5877c5a12064c36977e1-00665ca2f8) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server ERROR: Object returning 503 for [] (txn: tx231303ea455a40ff8ecb6-00665ca2d0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server ERROR: Object returning 503 for [] (txn: tx97ba7c0bb54342fa95421-00665ca2d0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test ERROR: Invalid swift_bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4161, in override_bytes_from_content_type listing_dict['bytes'] = int(swift_bytes) ^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: 'hey' test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test WARNING: Handoff requested (9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txda1130916bca45d4ac318-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx663ebf2479624e2394822-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx64ffc916fbc7498cb2c4a-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txc6fda7b9ff70432ebd3ab-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx4570d5dfcf974fa49c4a4-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txe6977266bda64359a6f56-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx61a1dbb494b64a00a9af2-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx278e318439284e32a3fe1-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx0f171ac837994b66b8a78-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd72828e8960b4901952ed-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx7cb212157b7240ecb2654-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx3165c9a1adef45a49e3d1-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx6b212ab63be04c0aa67a2-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx235d2d3259f84b71b1a00-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1613a21b17ba4684b882b-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbb4b2eda12c74ad89d9f8-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txa4d426438f9e48aab555a-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx9b3da148f90143f18be4a-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7715db61ca904e389b9ec-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx9078fef3ab0947fd9c2fe-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx0801bea6fa3443b3be1f1-00665ca2e1) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe09be12dd68f441a9c9ac-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txc33e2c71b00541e6bc5d4-00665ca2e1) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txd120284394194308b7588-00665ca2e1) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txfdedfa04402c47899742a-00665ca2e1) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: tx5552d237d83c49eda99e2-00665ca2e1) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txf66608f7bfbf4c2a8b0cf-00665ca2e1) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission ... ok test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txc58f6245da09400b93a70-00665ca2cb) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txc58f6245da09400b93a70-00665ca2cb) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txc58f6245da09400b93a70-00665ca2cb) test INFO: autocreate account '/account' (txn: tx885be9cee92a4c75a7a83-00665ca2cf) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx899a5db7450d402dbfe99-00665ca2cf) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx899a5db7450d402dbfe99-00665ca2cf) test INFO: autocreate account '/account' (txn: tx899a5db7450d402dbfe99-00665ca2cf) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx6eafaba51924484aa5f22-00665ca2cf) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx6eafaba51924484aa5f22-00665ca2cf) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: tx7870d50614924c4ebd87b-00665ca2d2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: txd07b4a7baf954a439f4a4-00665ca2d2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb4abf8d5523b4d83b0f0d-00665ca2d2) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx95ee3ea74c78402eafbc3-00665ca2d2) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx451258ed60834806b823e-00665ca2d3) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8eb874f45913497680694-00665ca2d3) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txcfdfb707283543f6ac6ee-00665ca2d3) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx8debed04045148a49bc74-00665ca2d3) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx335dd3e8caf74a239c0f4-00665ca2d3) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx335dd3e8caf74a239c0f4-00665ca2d3) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx335dd3e8caf74a239c0f4-00665ca2d3) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txe56afe4c457d441e9f32f-00665ca2d3) test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0059 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0130 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0127 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0068 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0082 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0327 "-" 3610637 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3610637 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0372 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0126 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0217 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0094 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0238 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0080 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0043 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0065 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0404 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0063 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0280 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0429 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0168 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0072 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0227 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0178 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0200 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0219 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0203 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0010 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0121 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0141 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0263 "-" 3610637 - test ERROR: ERROR Account update failed with 127.0.0.1:52201/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0133 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0401 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0044 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0007 "-" 3610637 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0185 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3610637 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0002 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0070 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0482 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0391 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0224 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0615 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0076 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:25 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0187 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:50:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value ... ok proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a0d5536b35d466dbbdc2-00665ca2b3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 3, Lengths: [458, 458, 458, 458, 458, 458, 458, 458, 458, 490]: Invalid fragment payload in ECPyECLibDriver.decode (txn: tx7f0a6f66e3134deeb67fd-00665ca2b7) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9cba3a8ff5484928951b0-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9cba3a8ff5484928951b0-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9cba3a8ff5484928951b0-00665ca2bb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 412 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from primary (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx37c4015a04a94b89a0996-00665ca2bb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (201, 404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (201, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT ... ok test DEBUG: fetching '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347038.012641) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347038.012641) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347038.012641) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347038.012641) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717347038.01264) in policy_index 1 responded 503 (0000000000.00000) test INFO: retry: '/AUTH_bob/c/o1' (1717347038.012641) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347038.021427) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347038.021427) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347038.021427) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347038.021427) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717347038.02143) in policy_index 1 responded 404 (1717347038.02143_0000000000000002) test INFO: retry: '/AUTH_bob/c/o1' (1717347038.021427) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: unavailable_container: '/AUTH_bob/c/o1' (3618.841870) unable to determine the destination policy_index test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_container': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test ERROR: Unhandled Exception trying to reconcile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 852, in run_once self.reconcile() File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 831, in reconcile for container in self._iter_containers(): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 748, in _iter_containers one_page = list(itertools.islice( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 331, in _iter_items resp = self.make_request( ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: Reconciler Stats: {} test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00053 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpv21c9p99/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00984 seconds (101.65935/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test DEBUG: Successfully deleted db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test DEBUG: Not deleting db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (2/3 success) test DEBUG: Not deleting db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (requires sharding, state unsharded) test DEBUG: Not deleting db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347038.68264.db (requires sharding, state sharding) test DEBUG: Not deleting db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347038.68264.db (2/3 success) test DEBUG: Successfully deleted db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347038.68264.db test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT ... ok test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txe56afe4c457d441e9f32f-00665ca2d3) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txe56afe4c457d441e9f32f-00665ca2d3) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx4c3dcac462864bffa5486-00665ca2d8) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx4c3dcac462864bffa5486-00665ca2d8) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx4c3dcac462864bffa5486-00665ca2d8) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx4c3dcac462864bffa5486-00665ca2d8) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx7c110886bb2845f38cb00-00665ca2d8) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx7c110886bb2845f38cb00-00665ca2d8) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx7c110886bb2845f38cb00-00665ca2d8) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx7c110886bb2845f38cb00-00665ca2d8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txfcec3ef02a854d16b462a-00665ca2db) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txabbccafc99b340849ab3b-00665ca2db) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txabbccafc99b340849ab3b-00665ca2db) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txabbccafc99b340849ab3b-00665ca2db) test ERROR: Account HEAD returning 503 for [] (txn: txabbccafc99b340849ab3b-00665ca2db) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txa4ffb4f356f2492690c08-00665ca2db) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txa4ffb4f356f2492690c08-00665ca2db) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txa4ffb4f356f2492690c08-00665ca2db) test ERROR: Account HEAD returning 503 for [] (txn: txa4ffb4f356f2492690c08-00665ca2db) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txd8fff96c90cd4c7a90658-00665ca2db) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx2f422a31054444a3a454a-00665ca2db) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx2f422a31054444a3a454a-00665ca2db) test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name ... ok test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:26 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:26 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0024 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:26 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0028 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:26 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0028 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:28 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0263 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.1829 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0919 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0172 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0002 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0295 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0364 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0462 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0484 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0033 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0076 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0317 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0012 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0008 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0282 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0065 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0144 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0639 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0004 "-" 3610691 nada test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0004 "-" 3610691 999 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0311 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c_nulo" 204 - "-" "-" "-" 0.0035 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0311 "-" 3610691 1 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "HEAD /sda1/p/a/c_unu" 204 - "-" "-" "-" 0.0053 "-" 3610691 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0013 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0005 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0211 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0009 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0307 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:29 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0054 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:29 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0492 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0031 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0021 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:30 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0355 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0121 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0130 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0143 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps ... ok test.unit.container.test_server.TestContainerController.test_object_update_with_offset test.unit.container.test_server.TestContainerController.test_object_update_with_offset ... ok test.unit.container.test_server.TestContainerController.test_params_format test.unit.container.test_server.TestContainerController.test_params_format ... ok test.unit.container.test_server.TestContainerController.test_params_utf8 test.unit.container.test_server.TestContainerController.test_params_utf8 ... ok test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_prefix_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_with_reserved ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server ERROR: Object returning 503 for [] (txn: tx33f13349548a4194beff1-00665ca2d0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server ERROR: Object returning 503 for [] (txn: tx0ee8abca3c294024aafd7-00665ca2d0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1dcd786461f84102bf49c-00665ca2eb) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx2faafd549ade4522828bf-00665ca2eb) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: txb11f1c709fc744a8b287e-00665ca2eb) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx08455685ef9f4c858c6df-00665ca2eb) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx1cb82225a94c4e3990fad-00665ca2eb) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: tx62baf30005a74594931eb-00665ca2eb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3c986eb076994d6ca637d-00665ca2eb) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx5b28b13cd5d347f3bd12d-00665ca2eb) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txc5eeb9da1b2c423ea51ac-00665ca2eb) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx1c6fc09200da4c39b720a-00665ca2eb) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7f03698fb1424a5f8a21b-00665ca2eb) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx2568ac45eed0402c920d0-00665ca2eb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: tx65b51e7644f04f4694889-00665ca2eb) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx65b51e7644f04f4694889-00665ca2eb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/ascii/ascii From Object Server 10.0.0.2:1002/sdc (txn: tx521f7a739c7f4a8b83e1b-00665ca2ec) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/툎/鐉 From Object Server 10.0.0.0:1000/sda (txn: tx712d6707979a4777bf0d3-00665ca2ec) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx5f116fa3991d4b12a444c-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txe414f6958a2a41abb3b92-00665ca2ec) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txe414f6958a2a41abb3b92-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx50688efbe7d04935a8d79-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx50688efbe7d04935a8d79-00665ca2ec) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private ... ok test INFO: - - - [02/Jun/2024:16:50:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0043 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0172 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0206 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0225 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0021 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0024 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0023 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0223 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0278 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0185 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0011 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0111 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0015 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0013 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0023 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0115 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0051 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0063 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0153 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0015 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0012 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0026 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0117 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0018 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container ... ok proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server ERROR: Object returning 503 for [] (txn: tx32a2fa9fa8a14326ad31e-00665ca2bb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sda (txn: txc735763c34d949e288520-00665ca2c9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx422b43599b8e428aa43e9-00665ca2cb) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive ... ok test DEBUG: Successfully deleted db /tmp/tmp1t3sg4id/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '4a4ed62542c5d643dbfaef4aa61ed0b0', '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda', '1717347038.78494', '1717347038.00000', '0', '', '1717347038.00000', 51, 0) test DEBUG: /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o0', 'created_at': '1717347039.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o1', 'created_at': '1717347040.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o2', 'created_at': '1717347041.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o3', 'created_at': '1717347042.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o4', 'created_at': '1717347043.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o5', 'created_at': '1717347044.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o6', 'created_at': '1717347045.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o7', 'created_at': '1717347046.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o8', 'created_at': '1717347047.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o9', 'created_at': '1717347048.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 11, 'name': 'o10', 'created_at': '1717347049.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 12, 'name': 'o11', 'created_at': '1717347050.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 13, 'name': 'o12', 'created_at': '1717347051.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 14, 'name': 'o13', 'created_at': '1717347052.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 15, 'name': 'o14', 'created_at': '1717347053.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 16, 'name': 'o15', 'created_at': '1717347054.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 17, 'name': 'o16', 'created_at': '1717347055.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 18, 'name': 'o17', 'created_at': '1717347056.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 19, 'name': 'o18', 'created_at': '1717347057.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 20, 'name': 'o19', 'created_at': '1717347058.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': 'o20', 'created_at': '1717347059.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 22, 'name': 'o21', 'created_at': '1717347060.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 23, 'name': 'o22', 'created_at': '1717347061.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 24, 'name': 'o23', 'created_at': '1717347062.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 25, 'name': 'o24', 'created_at': '1717347063.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 26, 'name': 'o25', 'created_at': '1717347064.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 27, 'name': 'o26', 'created_at': '1717347065.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 28, 'name': 'o27', 'created_at': '1717347066.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 29, 'name': 'o28', 'created_at': '1717347067.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 30, 'name': 'o29', 'created_at': '1717347068.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda') test DEBUG: /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 30 (0.0370011s) test DEBUG: Synchronization for /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db has fallen more than 30 rows behind; moving on and will try again next pass. REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '4a4ed62542c5d643dbfaef4aa61ed0b0', '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda', '1717347038.78494', '1717347038.00000', '0', '', '1717347038.00000', 51, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12548 seconds (7.96944/s) test INFO: Removed 0 dbs test INFO: 1 successes, 1 failures test INFO: diff:1 diff_capped:1 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '4a4ed62542c5d643dbfaef4aa61ed0b0', '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda', '1717347038.78494', '1717347038.00000', '0', '', '1717347038.00000', 51, 0) test DEBUG: /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row 30 test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part ... ok test.unit.container.test_replicator.TestReplicatorSync.test_local_ids test.unit.container.test_replicator.TestReplicatorSync.test_local_ids ... ok test INFO: - - - [02/Jun/2024:16:50:30 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0523 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0041 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0013 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0531 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0106 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0431 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0014 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0478 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0015 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0263 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0063 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0390 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0063 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0298 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0087 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0248 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:32 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0016 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:32 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0296 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0152 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0111 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0186 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0127 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0324 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0108 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0187 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0041 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0035 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0002 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0002 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0002 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0067 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0033 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0127 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0115 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0043 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0133 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0058 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0146 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:36 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0040 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0130 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0003 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0121 "-" 3610691 - test.unit.container.test_server.TestContainerController.test_put_auto_create test.unit.container.test_server.TestContainerController.test_put_auto_create ... ok test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestContainerController.test_serv_reserv test.unit.container.test_server.TestContainerController.test_serv_reserv ... ok test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestContainerController.test_through_call test.unit.container.test_server.TestContainerController.test_through_call ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path test.unit.container.test_server.TestContainerController.test_through_call_invalid_path ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestContainerController.test_weird_content_types test.unit.container.test_server.TestContainerController.test_weird_content_types ... ok test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_logging test.unit.obj.test_auditor.TestAuditor.test_object_run_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_sleeper test.unit.obj.test_auditor.TestAuditor.test_sleeper ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347029.25773', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347029.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347029.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347029.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347029.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestConfigOptionHandling.test_default test.unit.obj.test_server.TestConfigOptionHandling.test_default ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put test.unit.obj.test_server.TestObjectServer.test_expect_on_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_not_found test.unit.obj.test_server.TestObjectServer.test_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error ... ok sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347205.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/ade/701daeeb47c391acdaa3cd54396e2ade/701daeeb47c391acdaa3cd54396e2ade.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347207.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4f4/b40800d4b71fe79c0b9804f93af704f4/b40800d4b71fe79c0b9804f93af704f4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347207.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/4f4/b40800d4b71fe79c0b9804f93af704f4/b40800d4b71fe79c0b9804f93af704f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347209.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/efa/7dedbff713e66c465cda56a91c80defa/7dedbff713e66c465cda56a91c80defa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347209.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/efa/7dedbff713e66c465cda56a91c80defa/7dedbff713e66c465cda56a91c80defa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347211.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/cb5/bae881495856038a1730493c12ee9cb5/bae881495856038a1730493c12ee9cb5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347211.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/cb5/bae881495856038a1730493c12ee9cb5/bae881495856038a1730493c12ee9cb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347213.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/19f/a96e69dca35960f91df4068e3cac319f/a96e69dca35960f91df4068e3cac319f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347213.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/19f/a96e69dca35960f91df4068e3cac319f/a96e69dca35960f91df4068e3cac319f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347215.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/7ac/266fe037f36680583245fa60e936f7ac/266fe037f36680583245fa60e936f7ac.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347215.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/7ac/266fe037f36680583245fa60e936f7ac/266fe037f36680583245fa60e936f7ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347217.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/691/40e627a43d33aa5ebcaba87a71e11691/40e627a43d33aa5ebcaba87a71e11691.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347217.00000, db: /tmp/tmp_3ao9bcd/sda/containers/0/691/40e627a43d33aa5ebcaba87a71e11691/40e627a43d33aa5ebcaba87a71e11691.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717346996.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/6be/571fb525e01acd3acbb761325d02a6be/571fb525e01acd3acbb761325d02a6be.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347001.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/da1/7fc90f796bf6a4553969fe9a09d5ada1/7fc90f796bf6a4553969fe9a09d5ada1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347006.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/764/a3d1dba7114b0f9787f79e9f6b131764/a3d1dba7114b0f9787f79e9f6b131764.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347011.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/948/6ffda282d80e827daac1a49082a02948/6ffda282d80e827daac1a49082a02948.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347016.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/829/8d009861bef8f0680a41edad551a3829/8d009861bef8f0680a41edad551a3829.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347021.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/dc5/2fbfb95bd8fa2dcaada37604f4709dc5/2fbfb95bd8fa2dcaada37604f4709dc5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347026.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ac5/4891d07c9f3eec3af30e6d26e4044ac5/4891d07c9f3eec3af30e6d26e4044ac5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347031.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/27f/6197b9e055c824935434daa8faf3727f/6197b9e055c824935434daa8faf3727f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347036.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/770/4663a994f4035c1454a03d3622197770/4663a994f4035c1454a03d3622197770.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347041.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e85/e3d410e8f6de4c016b9be3c2e13dae85/e3d410e8f6de4c016b9be3c2e13dae85.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347046.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/28e/633d545aaef021ba39e2265ef082a28e/633d545aaef021ba39e2265ef082a28e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347051.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/101/488289e4d5ae4e748c1e33e5e1c9c101/488289e4d5ae4e748c1e33e5e1c9c101.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347056.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/fef/047c2a5e447dc6a877c826e9c4bb7fef/047c2a5e447dc6a877c826e9c4bb7fef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347061.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/be1/51a0acc1965c9c3eb824d3113c745be1/51a0acc1965c9c3eb824d3113c745be1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347066.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/36c/9ad36cc4b07fed77abdaebb7ddb5636c/9ad36cc4b07fed77abdaebb7ddb5636c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347071.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/2dc/fa108a436c2cafe7b53eac6a38b812dc/fa108a436c2cafe7b53eac6a38b812dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347076.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/56c/cb23c769282ef181a1a99567082b856c/cb23c769282ef181a1a99567082b856c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347081.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a6e/3b8bf9d3c4f4aeb87c4148d91419aa6e/3b8bf9d3c4f4aeb87c4148d91419aa6e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347086.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/2f3/bb7a5ed641f545f451637c81c68d22f3/bb7a5ed641f545f451637c81c68d22f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347091.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e83/978bb8437b1dff378a17797313ef7e83/978bb8437b1dff378a17797313ef7e83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347096.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/306/4a1815d3b2d2bd7254b62300d9f74306/4a1815d3b2d2bd7254b62300d9f74306.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347101.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b57/44044402a4a9f9b79f89f3bb3a9f5b57/44044402a4a9f9b79f89f3bb3a9f5b57.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347106.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4e9/fae6c6257108f3dc87c192e2792394e9/fae6c6257108f3dc87c192e2792394e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347111.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/cbf/49d47d49f31561d154217ecebe777cbf/49d47d49f31561d154217ecebe777cbf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347116.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ba0/2a1f84ef79830f46517498ce51263ba0/2a1f84ef79830f46517498ce51263ba0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347121.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/fe6/e487b014f5ac96e42769378fd5752fe6/e487b014f5ac96e42769378fd5752fe6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347126.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/0c6/97a01073d4abb2b96ae47f7d8babb0c6/97a01073d4abb2b96ae47f7d8babb0c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347131.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/abf/1ab8f534bfc71acaa7b7dba63c15cabf/1ab8f534bfc71acaa7b7dba63c15cabf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347136.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/274/b65ff33cb20798b522e03f6dd7268274/b65ff33cb20798b522e03f6dd7268274.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347141.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/bcf/6e621ee4c21a1808af8634a9d70a0bcf/6e621ee4c21a1808af8634a9d70a0bcf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347146.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/eb6/cf136393d10386a0a341fba0d77e5eb6/cf136393d10386a0a341fba0d77e5eb6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347151.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/6f6/57ff8893014b0adfac3add59431296f6/57ff8893014b0adfac3add59431296f6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347156.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/71f/d52a6a6ad8082a975da4513a0c3bc71f/d52a6a6ad8082a975da4513a0c3bc71f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347161.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/057/ea7530ea8eca50ea5d21dea4cd412057/ea7530ea8eca50ea5d21dea4cd412057.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347166.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/89b/25a5febe83c1248af555fc682ec0089b/25a5febe83c1248af555fc682ec0089b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347171.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e58/64187d1dbb62210e8df0a37eed086e58/64187d1dbb62210e8df0a37eed086e58.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347176.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/343/5e2d0efa8baa83468002dedd31db9343/5e2d0efa8baa83468002dedd31db9343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347181.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/71e/f3f9546e6bd95c47724d04d854a8071e/f3f9546e6bd95c47724d04d854a8071e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347186.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b68/d1cf8a60f7bf9b0415f9d61e896b3b68/d1cf8a60f7bf9b0415f9d61e896b3b68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347191.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a7f/77f119b07a582438555b16744acf8a7f/77f119b07a582438555b16744acf8a7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347196.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/af5/2722d14adf6006e454f1bd1f56ae9af5/2722d14adf6006e454f1bd1f56ae9af5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347201.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a73/ba77b742b64680e0c5d093ec0937fa73/ba77b742b64680e0c5d093ec0937fa73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347221.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/fe7/2565fb6b452c3357d772b13165edbfe7/2565fb6b452c3357d772b13165edbfe7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347226.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3b9/dfe54dbba6ff5f1625c32d18336dc3b9/dfe54dbba6ff5f1625c32d18336dc3b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347236.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a8f/ebfb13dc304bbd03a49c50d07f4e8a8f/ebfb13dc304bbd03a49c50d07f4e8a8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347241.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/da5/468ece8cf53547f1b8a568b80793eda5/468ece8cf53547f1b8a568b80793eda5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347251.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3fd/7751ae5ee3588174f058cdf892da93fd/7751ae5ee3588174f058cdf892da93fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347256.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8a1/99e66f00d4f64cf385e7c517ec9a28a1/99e66f00d4f64cf385e7c517ec9a28a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347261.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8ac/81e1169bc4e5fb82f1f14eb5ad7e78ac/81e1169bc4e5fb82f1f14eb5ad7e78ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347266.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/41e/92f67ef3444669c841ee16f4c1f1741e/92f67ef3444669c841ee16f4c1f1741e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347271.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d48/ca9cc2335300ff7749b4d62fa2484d48/ca9cc2335300ff7749b4d62fa2484d48.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347281.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/9e7/394ac10a5273e9fd36787515f1e1c9e7/394ac10a5273e9fd36787515f1e1c9e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347286.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/93f/63029f619cf6f041b5be15301935993f/63029f619cf6f041b5be15301935993f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347296.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ede/fb883c1b75b874242cbae9762c9daede/fb883c1b75b874242cbae9762c9daede.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347301.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/9b3/0ecce7ff9e1f31833bc6e5df5c9539b3/0ecce7ff9e1f31833bc6e5df5c9539b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347311.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a21/b95f4c519c69274b073697f12bc79a21/b95f4c519c69274b073697f12bc79a21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347316.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d98/ab6feeba2011355fccdf84c60aa17d98/ab6feeba2011355fccdf84c60aa17d98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347321.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/287/8358b8edaaf2a01d62c53bf67645e287/8358b8edaaf2a01d62c53bf67645e287.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347326.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8c5/7a643b855bccc4cf3c4eaeda83fcb8c5/7a643b855bccc4cf3c4eaeda83fcb8c5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347331.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/015/d3562d2e4c11fb2711fd44745c331015/d3562d2e4c11fb2711fd44745c331015.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347336.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/6a9/cb56574485967632113a25de82a576a9/cb56574485967632113a25de82a576a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347341.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e8f/524994fab6a7b9f6c50b1aadccf3fe8f/524994fab6a7b9f6c50b1aadccf3fe8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347346.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/042/b36421fbfb3a8e851264ca2526d55042/b36421fbfb3a8e851264ca2526d55042.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347351.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/07c/98151d85b578b8be96f86b27962e607c/98151d85b578b8be96f86b27962e607c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347356.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/fd9/5e75a8aabafea6a2b2a2950786459fd9/5e75a8aabafea6a2b2a2950786459fd9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347361.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f67/cead5fd9d4fc3e92b86935946cfd6f67/cead5fd9d4fc3e92b86935946cfd6f67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347366.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e53/561c335c709174eba02b2bdfe989fe53/561c335c709174eba02b2bdfe989fe53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347371.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/01c/042976de285ca25e79377d6018f6a01c/042976de285ca25e79377d6018f6a01c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347376.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/11a/1fd1ed0006e3b44abafa1ab44bc7011a/1fd1ed0006e3b44abafa1ab44bc7011a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347381.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/505/7efe0fac6ce1dead199810f434c1e505/7efe0fac6ce1dead199810f434c1e505.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347386.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5ca/e95de7b9d6e28529830faa640d79a5ca/e95de7b9d6e28529830faa640d79a5ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347391.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4f5/776a39fcf13ae439e800e99a41f964f5/776a39fcf13ae439e800e99a41f964f5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347396.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/57c/65b8a04cfc9b201d45fbeba64040e57c/65b8a04cfc9b201d45fbeba64040e57c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347401.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/001/9bf4dabc3693acf7cf251d73854c6001/9bf4dabc3693acf7cf251d73854c6001.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347406.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ed4/41f4ba63647d5a496efd0396c7710ed4/41f4ba63647d5a496efd0396c7710ed4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347411.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f46/9ebb1d40fb8411967f98e5d9e0b91f46/9ebb1d40fb8411967f98e5d9e0b91f46.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347416.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5c8/49cdec80e148f9846e74797f86acf5c8/49cdec80e148f9846e74797f86acf5c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347421.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/7db/5520c78713d238fbff5e39a4cc57c7db/5520c78713d238fbff5e39a4cc57c7db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347426.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/635/f73e832299517c227d02e17562ced635/f73e832299517c227d02e17562ced635.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347431.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c36/b4f2063855b74c713b6b6f78accabc36/b4f2063855b74c713b6b6f78accabc36.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347436.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/508/6f7bc76dab03d7cfd8769c0a0a9cc508/6f7bc76dab03d7cfd8769c0a0a9cc508.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347441.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c7a/8f309f953aa2c3faad69cbadb5b4ac7a/8f309f953aa2c3faad69cbadb5b4ac7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347446.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/39c/6b840ade6c08b0f5d0fe2c152c2de39c/6b840ade6c08b0f5d0fe2c152c2de39c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347451.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f3a/ff242e03b700be3fc02edd358a06bf3a/ff242e03b700be3fc02edd358a06bf3a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347456.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/df2/93c5c28d0484465f57318ba1e6c2adf2/93c5c28d0484465f57318ba1e6c2adf2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347461.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b0b/323b8e2f1be3a0220329bca463ffcb0b/323b8e2f1be3a0220329bca463ffcb0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347466.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/afb/a423957a5707e51dcefac3410b2fbafb/a423957a5707e51dcefac3410b2fbafb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347471.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/bdf/be189244488ca5068644e982de7aebdf/be189244488ca5068644e982de7aebdf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347476.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/96b/02634527f89f260bcea4612470ee396b/02634527f89f260bcea4612470ee396b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347481.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/7ae/d2067d78e367ddb1dbe21d8196bc37ae/d2067d78e367ddb1dbe21d8196bc37ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347486.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/697/5863b44d485bd2c1571c757cefdea697/5863b44d485bd2c1571c757cefdea697.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347491.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a73/74e53c5ecd37dd833fd159ed2cfb6a73/74e53c5ecd37dd833fd159ed2cfb6a73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347496.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b29/9f5294e180318fe2c2801f2efcec7b29/9f5294e180318fe2c2801f2efcec7b29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347501.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3e8/3bdd79d191994b4971ce3c9d17d873e8/3bdd79d191994b4971ce3c9d17d873e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347506.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8fc/631c26eae7b81c47dea411a0b50b38fc/631c26eae7b81c47dea411a0b50b38fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347511.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/09a/fbbd3e4526333b1a9174e1bd1419d09a/fbbd3e4526333b1a9174e1bd1419d09a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347516.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c83/40771d6f040a89cd670ab1e7cd402c83/40771d6f040a89cd670ab1e7cd402c83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347521.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/675/519b729bbb981564ecd13efad3f26675/519b729bbb981564ecd13efad3f26675.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347526.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a79/2f4439ce67d7a4305b21be2bf7743a79/2f4439ce67d7a4305b21be2bf7743a79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347531.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ebe/3685f67842e65c80bbf136b17d56cebe/3685f67842e65c80bbf136b17d56cebe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347536.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b2d/a9d22f3355b7608297c18e95f068eb2d/a9d22f3355b7608297c18e95f068eb2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347541.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/385/fb57c201897216b0e1c6b3c853126385/fb57c201897216b0e1c6b3c853126385.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347546.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/83d/0c2e50e8c39873593ae91e4d1c96083d/0c2e50e8c39873593ae91e4d1c96083d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347551.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ae5/32301a34a576c1c5bb1be6efe9a16ae5/32301a34a576c1c5bb1be6efe9a16ae5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347556.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/bd4/6131607b2464f36af991062975070bd4/6131607b2464f36af991062975070bd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347561.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/1c9/323bad91ef1dfccbfceda6d1c45b31c9/323bad91ef1dfccbfceda6d1c45b31c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347566.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/56d/3f38089a621431766aad7d1d4da1156d/3f38089a621431766aad7d1d4da1156d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347571.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347576.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4b8/2480e27a7bd428748e78db4a5f9694b8/2480e27a7bd428748e78db4a5f9694b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347581.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/87c/095c2a458ebb8e52c9cd8b9fbbdbb87c/095c2a458ebb8e52c9cd8b9fbbdbb87c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347586.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ab0/5c4022110f825609da641f495808dab0/5c4022110f825609da641f495808dab0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347591.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/9dc/c4990f853103b5378ab8adce6c5799dc/c4990f853103b5378ab8adce6c5799dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347596.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/952/0d04f62a905f37d3008784e8df3f8952/0d04f62a905f37d3008784e8df3f8952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347601.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/603/8c1d0fb6cd18823fe540266f482db603/8c1d0fb6cd18823fe540266f482db603.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347606.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d62/0bc6cfc11cda2d7d9ee4de6a7e320d62/0bc6cfc11cda2d7d9ee4de6a7e320d62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347611.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c15/7a1b1890283c6d39e2b2a8372f554c15/7a1b1890283c6d39e2b2a8372f554c15.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347621.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ec5/d1e5ed1cb9b3ae4c20faa5558fbefec5/d1e5ed1cb9b3ae4c20faa5558fbefec5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347626.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a98/1e3cba8648d01a23ad6749a55042aa98/1e3cba8648d01a23ad6749a55042aa98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347631.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/823/272092cf742dfbeef04f71672e31d823/272092cf742dfbeef04f71672e31d823.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347656.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b0e/0d131d949dc99f78dddc400126474b0e/0d131d949dc99f78dddc400126474b0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347661.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/fa9/2ee0011b7836ef50a6548f0f5439cfa9/2ee0011b7836ef50a6548f0f5439cfa9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347666.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b01/ded215a90f8af0bd03a459a3039a7b01/ded215a90f8af0bd03a459a3039a7b01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347671.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/474/5dd5386cac23f769594971f1284ae474/5dd5386cac23f769594971f1284ae474.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347686.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/933/5e8d9b5582f0bad722c8505f57ade933/5e8d9b5582f0bad722c8505f57ade933.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347691.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/223/9562307612d5aeab22c4e29141429223/9562307612d5aeab22c4e29141429223.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347701.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/340/338f10fab22b80566a33c54f7707b340/338f10fab22b80566a33c54f7707b340.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347706.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/250/390b9a2e91110c85bc4826e989ad2250/390b9a2e91110c85bc4826e989ad2250.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347711.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/6df/57dd647dd6d1de8ae7123608ff9736df/57dd647dd6d1de8ae7123608ff9736df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347716.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ef8/2bd31ca5e5fb2da1a764945d5aaeeef8/2bd31ca5e5fb2da1a764945d5aaeeef8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347721.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/30b/a840879dd01db1abe26a5be62c2b730b/a840879dd01db1abe26a5be62c2b730b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347726.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/919/2d3fccec5449ced10bab5e792b82d919/2d3fccec5449ced10bab5e792b82d919.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347731.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/aae/1e162350d0a369474f8a486f29a8aaae/1e162350d0a369474f8a486f29a8aaae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347746.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/294/0955b73a8c32ee5a94b397491515a294/0955b73a8c32ee5a94b397491515a294.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347751.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b8b/ef2502503030e69f7858112638ac2b8b/ef2502503030e69f7858112638ac2b8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347761.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/0c3/4b2285ee40dca96fe129bd70f95380c3/4b2285ee40dca96fe129bd70f95380c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347766.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c45/e6438e2ceb444065cbbbb631c9be9c45/e6438e2ceb444065cbbbb631c9be9c45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347771.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/41c/318dbae52b2facfd3d7308f395f2b41c/318dbae52b2facfd3d7308f395f2b41c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347776.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a49/76c0b29cc74d17206aace3a14e682a49/76c0b29cc74d17206aace3a14e682a49.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347781.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ca0/6759e6311b4637ebee8520056518fca0/6759e6311b4637ebee8520056518fca0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347786.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/48e/3cd1b208bf2bcff2ebe5a4ef91ae948e/3cd1b208bf2bcff2ebe5a4ef91ae948e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347791.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ed8/9151146d914dc706c02b972cace62ed8/9151146d914dc706c02b972cace62ed8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347796.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/2e9/b24ef43711af2c751d4669c3d515d2e9/b24ef43711af2c751d4669c3d515d2e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347801.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f97/83e0c884ad3cccd2eeb6c7628ad72f97/83e0c884ad3cccd2eeb6c7628ad72f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347806.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/51f/e0fd15fc9906f2f18fa47a8866bf551f/e0fd15fc9906f2f18fa47a8866bf551f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347811.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/733/6c0df9a203dc1579ae135234cc9fb733/6c0df9a203dc1579ae135234cc9fb733.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347816.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/48c/d67b765547bbc5013b2795518c7a348c/d67b765547bbc5013b2795518c7a348c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347821.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5e1/5c5cd5d35895fceb0a365e99a32285e1/5c5cd5d35895fceb0a365e99a32285e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347826.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/873/6416c224725f217ec6279be6fb802873/6416c224725f217ec6279be6fb802873.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347831.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/6d2/40a071409db54313470d032a2c5df6d2/40a071409db54313470d032a2c5df6d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347836.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e5c/44d385caacc6153ade9eca4f23cfde5c/44d385caacc6153ade9eca4f23cfde5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347841.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5c3/3b58f7e1fdada2f37f5bc1d682bc15c3/3b58f7e1fdada2f37f5bc1d682bc15c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347846.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/20e/c83e295a4ab6eaf41c64e3ecbffa820e/c83e295a4ab6eaf41c64e3ecbffa820e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347851.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d68/9c12eb783405094d82dc1dd8cd573d68/9c12eb783405094d82dc1dd8cd573d68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347856.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/285/d3a10181415135d1939d0a3d5bb4d285/d3a10181415135d1939d0a3d5bb4d285.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347861.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/64b/9fb673b94d6407e7932278e8e1acc64b/9fb673b94d6407e7932278e8e1acc64b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347866.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3d3/db17459df0047fc27fc4c5ef9bcad3d3/db17459df0047fc27fc4c5ef9bcad3d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347871.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8a6/0ea6faa97cde278e2326ffc6f2cce8a6/0ea6faa97cde278e2326ffc6f2cce8a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347876.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/565/b413d5e29b4d50e7764db0333e0a8565/b413d5e29b4d50e7764db0333e0a8565.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347881.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/61b/2f3aec0313542588c84e9f2dfc09e61b/2f3aec0313542588c84e9f2dfc09e61b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347886.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d2d/dde980c4b7d06283764610a1f0cf1d2d/dde980c4b7d06283764610a1f0cf1d2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347891.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/54b/a0e45c14ee21a432cb1966b74a9bb54b/a0e45c14ee21a432cb1966b74a9bb54b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347896.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/05b/e2ed524e8060da23f95e47135292005b/e2ed524e8060da23f95e47135292005b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347901.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/131/dcf7cfdeb95325c6bd5dba5eaeb06131/dcf7cfdeb95325c6bd5dba5eaeb06131.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347906.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ddc/3d623cd442e116c6aeeae75cdcc33ddc/3d623cd442e116c6aeeae75cdcc33ddc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347911.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ca8/97896419c08cadbf2fe0e41511ddaca8/97896419c08cadbf2fe0e41511ddaca8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347916.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c09/dd1ed34f0917ea13f0b85442b07ecc09/dd1ed34f0917ea13f0b85442b07ecc09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347921.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/adc/8ab0e947c4b4e8ad7b9cd32da9874adc/8ab0e947c4b4e8ad7b9cd32da9874adc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347926.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/eb8/5bf53b617ac36e7eb9df6cc941b2feb8/5bf53b617ac36e7eb9df6cc941b2feb8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347931.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/44b/69a40c5ec70ba711f39ec83a4ae7744b/69a40c5ec70ba711f39ec83a4ae7744b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347936.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/aef/350415aca3e9717e132d2b4b9c284aef/350415aca3e9717e132d2b4b9c284aef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347941.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4fb/edac15b85b7c7a507807b52cf0f844fb/edac15b85b7c7a507807b52cf0f844fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347946.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b65/c7ca2235947816d806284af631033b65/c7ca2235947816d806284af631033b65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347951.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3ca/99c2aa3fc06d541787e89474575c13ca/99c2aa3fc06d541787e89474575c13ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347956.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d2d/88c1ca3ab9b1764849c170469e2ced2d/88c1ca3ab9b1764849c170469e2ced2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347961.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/267/d6a5f2131e549f18bb5b242406490267/d6a5f2131e549f18bb5b242406490267.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347966.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/2f0/43d4820b18b0205bc3c7d2267d24a2f0/43d4820b18b0205bc3c7d2267d24a2f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347971.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ad1/4181ecfdeb223788d788ce21fd0b8ad1/4181ecfdeb223788d788ce21fd0b8ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347976.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/00f/3d2d8bbcb194eab9832eef73529dc00f/3d2d8bbcb194eab9832eef73529dc00f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347981.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/81b/4e945cb38c3ebbc1ee6fd98ed6b6281b/4e945cb38c3ebbc1ee6fd98ed6b6281b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347986.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/259/f62b66ff0f9e671fb67b8464e2bba259/f62b66ff0f9e671fb67b8464e2bba259.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347991.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4f0/118ded90726d86e6183375ad86a944f0/118ded90726d86e6183375ad86a944f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347996.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/6f8/a8a95b8853f53efd045879a9fe8456f8/a8a95b8853f53efd045879a9fe8456f8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348001.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f33/5116e0ccec346a3dc6d0aaddd04b6f33/5116e0ccec346a3dc6d0aaddd04b6f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348006.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/1d1/afa7fda231999cb100a9413335d7c1d1/afa7fda231999cb100a9413335d7c1d1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348011.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a98/b9ebf82c77391efee54587f0a8acca98/b9ebf82c77391efee54587f0a8acca98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348016.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/00b/d9ee39111d673d7cae8a0355141e000b/d9ee39111d673d7cae8a0355141e000b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348021.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5b9/dfa7c8da860bfa45974ae43fd7c8d5b9/dfa7c8da860bfa45974ae43fd7c8d5b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348026.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/79f/b0997c389df87f7ec4d9a2cc17ba779f/b0997c389df87f7ec4d9a2cc17ba779f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348031.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/7cf/8495cd601b2ed80346ed9079d9e0b7cf/8495cd601b2ed80346ed9079d9e0b7cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348036.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/1a5/514f3ee9b3f8b8fcd3adea99f02401a5/514f3ee9b3f8b8fcd3adea99f02401a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348041.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/add/ad8b005ff9a10fe82fc94bf386225add/ad8b005ff9a10fe82fc94bf386225add.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348046.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/468/7fb6074278f19cfd272e07d35dc02468/7fb6074278f19cfd272e07d35dc02468.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348051.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/604/24e10ccff6365c95971094f516db8604/24e10ccff6365c95971094f516db8604.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348056.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/9fa/cc4bcab6963551ceb295ca97841339fa/cc4bcab6963551ceb295ca97841339fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348061.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/089/4eb071be17b3731b7bcd20315fb19089/4eb071be17b3731b7bcd20315fb19089.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348066.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e5f/6207977c881c1e6f97460f0c40cf3e5f/6207977c881c1e6f97460f0c40cf3e5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348071.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/086/8018471d397b572673d41cc59e417086/8018471d397b572673d41cc59e417086.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348076.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/aa3/f42c5e768772f5ca5a9592890f391aa3/f42c5e768772f5ca5a9592890f391aa3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348081.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8cc/f0afae1cd204d19f6f07e2161f9d48cc/f0afae1cd204d19f6f07e2161f9d48cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348086.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ff3/5431f2d3ae24818660f7be43f62bdff3/5431f2d3ae24818660f7be43f62bdff3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348091.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/7e2/a059694237d153f74cc7b4091e53c7e2/a059694237d153f74cc7b4091e53c7e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348096.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f67/cc3de10bf4129ecbc2470be5ac438f67/cc3de10bf4129ecbc2470be5ac438f67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348101.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/8e1/6b64550eb0dbf684df85638591c288e1/6b64550eb0dbf684df85638591c288e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348106.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/031/c9badc067b71177cc4368aaaa8c3d031/c9badc067b71177cc4368aaaa8c3d031.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348111.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/040/bfa3568d3582f0eff86841ed6f6a6040/bfa3568d3582f0eff86841ed6f6a6040.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348116.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/c2e/8aaf648894753f3b784680267123fc2e/8aaf648894753f3b784680267123fc2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348121.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/b0c/86ad1ef9e40d63cc885a26c15d284b0c/86ad1ef9e40d63cc885a26c15d284b0c.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347031.42145', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy ... ok proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/ascii/ascii: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: tx50688efbe7d04935a8d79-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txe93233957a0048cbbfd4b-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txe93233957a0048cbbfd4b-00665ca2ec) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/툎/鐉: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: txe93233957a0048cbbfd4b-00665ca2ec) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx372d0412b5d4423aa440d-00665ca2ec) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx8174a0b257f74608a1559-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx8174a0b257f74608a1559-00665ca2ec) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx8174a0b257f74608a1559-00665ca2ec) proxy-server ERROR: Object GET returning 503 for [] (txn: tx8174a0b257f74608a1559-00665ca2ec) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from primary (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx78e179b867b14ad2bb132-00665ca2d0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from primary (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server ERROR: Object returning 503 for [] (txn: txf0add5949d8d4cab9d3dc-00665ca2d9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx87b31f38862d43458dfa1-00665ca2d9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 31, 'name': 'o30', 'created_at': '1717347069.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 32, 'name': 'o31', 'created_at': '1717347070.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 33, 'name': 'o32', 'created_at': '1717347071.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 34, 'name': 'o33', 'created_at': '1717347072.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 35, 'name': 'o34', 'created_at': '1717347073.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 36, 'name': 'o35', 'created_at': '1717347074.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 37, 'name': 'o36', 'created_at': '1717347075.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 38, 'name': 'o37', 'created_at': '1717347076.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 39, 'name': 'o38', 'created_at': '1717347077.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 40, 'name': 'o39', 'created_at': '1717347078.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 41, 'name': 'o40', 'created_at': '1717347079.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 42, 'name': 'o41', 'created_at': '1717347080.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 43, 'name': 'o42', 'created_at': '1717347081.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 44, 'name': 'o43', 'created_at': '1717347082.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 45, 'name': 'o44', 'created_at': '1717347083.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 46, 'name': 'o45', 'created_at': '1717347084.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 47, 'name': 'o46', 'created_at': '1717347085.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 48, 'name': 'o47', 'created_at': '1717347086.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 49, 'name': 'o48', 'created_at': '1717347087.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 50, 'name': 'o49', 'created_at': '1717347088.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 51, 'name': 'o101', 'created_at': '1717347189.00000', 'size': 0, 'content_type': 'content-type-new', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda') test DEBUG: /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 51 (0.0439312s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, '4a4ed62542c5d643dbfaef4aa61ed0b0', '2286e3ad-b62f-4bd0-a37c-0047486f6e53-sda', '1717347038.78494', '1717347038.00000', '0', '', '1717347038.00000', 51, 0) test DEBUG: /tmp/tmp_hcnsi98/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06726 seconds (14.86733/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:0 ts_repl:0 test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdb test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00045 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00058 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00035 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6c0r1eky/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '73f5f78f-672a-4af4-ae0d-fa641841d0d1-sda', '1717347041.38738', '1717344000.00000', '0', '', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmp6c0r1eky/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '73f5f78f-672a-4af4-ae0d-fa641841d0d1-sda') test DEBUG: /tmp/tmp6c0r1eky/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.00937128s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '73f5f78f-672a-4af4-ae0d-fa641841d0d1-sda', '1717347041.38738', '1717344000.00000', '0', '', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('73f5f78f-672a-4af4-ae0d-fa641841d0d1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmp6c0r1eky/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09156 seconds (10.92140/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write ... ok test INFO: - - - [02/Jun/2024:16:50:31 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0024 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0011 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0107 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0058 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0017 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0018 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmp3uvrksh3/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0129 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0019 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0018 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0010 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0115 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0025 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0099 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0009 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:34 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.3715 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0140 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0250 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0030 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0106 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0006 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0004 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0342 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0124 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0088 "-" 3610637 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0130 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0208 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0192 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0099 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device ... ok test INFO: - - - [02/Jun/2024:16:50:36 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:39 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0006 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:39 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.3694 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0138 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0004 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:42 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0119 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0002 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0266 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:42 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0217 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0138 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:42 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0180 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:42 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0175 "-" 3610691 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0264 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0194 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0161 "-" 3610691 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0127 "-" 3610691 0 test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:42 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3610691 - test INFO: - - - [02/Jun/2024:16:50:42 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0128 "-" 3610691 0 proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx191564ffa62743998131b-00665ca2fb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx6161075bab09479da3aa1-00665ca2fb) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestController.test_account_info_200 test.unit.proxy.test_mem_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_404 test.unit.proxy.test_mem_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_container_count test.unit.proxy.test_mem_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_account test.unit.proxy.test_mem_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_200 test.unit.proxy.test_mem_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_404 test.unit.proxy.test_mem_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_make_requests test.unit.proxy.test_mem_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_transfer_headers test.unit.proxy.test_mem_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type ... ok test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2f422a31054444a3a454a-00665ca2db) test ERROR: Account HEAD returning 503 for [] (txn: tx2f422a31054444a3a454a-00665ca2db) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txac36218c66ba47f48e2cd-00665ca2df) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txac36218c66ba47f48e2cd-00665ca2df) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txac36218c66ba47f48e2cd-00665ca2df) test ERROR: Account HEAD returning 503 for [] (txn: txac36218c66ba47f48e2cd-00665ca2df) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx87b4b00a000641bebc5ca-00665ca2e2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: tx73c47f4c9980451f84dcc-00665ca2e2) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: tx73c47f4c9980451f84dcc-00665ca2e2) test INFO: Client disconnected on read of '/a/c' (txn: tx73c47f4c9980451f84dcc-00665ca2e2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txfd20c62539104341b81dc-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx07e9d954a03e49b0b6e2b-00665ca2e2) test INFO: Client disconnected on read of '/a/c' (txn: tx07e9d954a03e49b0b6e2b-00665ca2e2) test INFO: Client disconnected on read of '/a/c/' (txn: tx07e9d954a03e49b0b6e2b-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx78cfccea447a4bc280c57-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx78cfccea447a4bc280c57-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx78cfccea447a4bc280c57-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx78cfccea447a4bc280c57-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx382b4fd2d3d54b5185e57-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx382b4fd2d3d54b5185e57-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx382b4fd2d3d54b5185e57-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx382b4fd2d3d54b5185e57-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx4b6d905262674994b3c25-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx4b6d905262674994b3c25-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx4b6d905262674994b3c25-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx4b6d905262674994b3c25-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx3f035c214620494581c0c-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx3f035c214620494581c0c-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx3f035c214620494581c0c-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx3f035c214620494581c0c-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx9cf739b27ed24c7c9e5d6-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx9cf739b27ed24c7c9e5d6-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx9cf739b27ed24c7c9e5d6-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx9cf739b27ed24c7c9e5d6-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txb88b6cfec63443cfb13ce-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txb88b6cfec63443cfb13ce-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txb88b6cfec63443cfb13ce-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txb88b6cfec63443cfb13ce-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txc2e76d04539743c087040-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txc2e76d04539743c087040-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txc2e76d04539743c087040-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txc2e76d04539743c087040-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx9e4a627e1a32471e83c5c-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx9e4a627e1a32471e83c5c-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx9e4a627e1a32471e83c5c-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx9e4a627e1a32471e83c5c-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txf53c597c13454a64a6464-00665ca2e2) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: txf53c597c13454a64a6464-00665ca2e2) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txf53c597c13454a64a6464-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf53c597c13454a64a6464-00665ca2e2) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf53c597c13454a64a6464-00665ca2e2) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txf53c597c13454a64a6464-00665ca2e2) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7a0a864f38d64913b0b42-00665ca2e6) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx7a0a864f38d64913b0b42-00665ca2e6) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx7a0a864f38d64913b0b42-00665ca2e6) test ERROR: Container GET returning 503 for [503, 503] (txn: tx7a0a864f38d64913b0b42-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd4c92747772d4d1c96859-00665ca2e6) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txd4c92747772d4d1c96859-00665ca2e6) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: txd4c92747772d4d1c96859-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd4c92747772d4d1c96859-00665ca2e6) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txd4c92747772d4d1c96859-00665ca2e6) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: txd4c92747772d4d1c96859-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd4c92747772d4d1c96859-00665ca2e6) test ERROR: Container GET returning 503 for [503, 503] (txn: txd4c92747772d4d1c96859-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx3aefad7702a24d82a86f8-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx3aefad7702a24d82a86f8-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx3aefad7702a24d82a86f8-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: tx3aefad7702a24d82a86f8-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txeb5f3a08180c4800a0e71-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txeb5f3a08180c4800a0e71-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txeb5f3a08180c4800a0e71-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txeb5f3a08180c4800a0e71-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx14441fcf1b6540f894de3-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx14441fcf1b6540f894de3-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx14441fcf1b6540f894de3-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: tx14441fcf1b6540f894de3-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txbd06371eea6b47c58000a-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txbd06371eea6b47c58000a-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txbd06371eea6b47c58000a-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txbd06371eea6b47c58000a-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txcbb91980a39e459091b17-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txcbb91980a39e459091b17-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txcbb91980a39e459091b17-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txcbb91980a39e459091b17-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txf8fd7a68589e4925ba495-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txf8fd7a68589e4925ba495-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txf8fd7a68589e4925ba495-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txf8fd7a68589e4925ba495-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txb17cfd37c7fb4f9cb265a-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txb17cfd37c7fb4f9cb265a-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txb17cfd37c7fb4f9cb265a-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txb17cfd37c7fb4f9cb265a-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx9ed1e82741a8419e92be0-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx9ed1e82741a8419e92be0-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx9ed1e82741a8419e92be0-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: tx9ed1e82741a8419e92be0-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txc542f6a8dc45442cbceaa-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txc542f6a8dc45442cbceaa-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txc542f6a8dc45442cbceaa-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txc542f6a8dc45442cbceaa-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa2ae9c35f1754091a2c6b-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa2ae9c35f1754091a2c6b-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa2ae9c35f1754091a2c6b-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txa2ae9c35f1754091a2c6b-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa9d520011b114357ba08f-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa9d520011b114357ba08f-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa9d520011b114357ba08f-00665ca2e6) test ERROR: Container GET returning 503 for [] (txn: txa9d520011b114357ba08f-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx38abd5464f324feb95e1e-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx38abd5464f324feb95e1e-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx38abd5464f324feb95e1e-00665ca2e6) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx38abd5464f324feb95e1e-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx34d8dec7f7064ebc8e462-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx34d8dec7f7064ebc8e462-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx34d8dec7f7064ebc8e462-00665ca2e6) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx34d8dec7f7064ebc8e462-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd753c26768c54a8786f24-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd753c26768c54a8786f24-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd753c26768c54a8786f24-00665ca2e6) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txd753c26768c54a8786f24-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa94b6f1d26774cdfa7e12-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa94b6f1d26774cdfa7e12-00665ca2e6) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa94b6f1d26774cdfa7e12-00665ca2e6) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txa94b6f1d26774cdfa7e12-00665ca2e6) test INFO: Client disconnected on read of '/a/c' (txn: txd4049d94883f4bcb86bfa-00665ca2e6) test INFO: Client disconnected on read of '/a/c/' (txn: txd4049d94883f4bcb86bfa-00665ca2e6) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4c3f2c139dbd4097b0c4b-00665ca2e6) test.unit.proxy.test_server.TestContainerController.test_error_limiting test.unit.proxy.test_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_transfer_headers test.unit.proxy.test_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test.unit.proxy.test_server.TestController.test_account_info_200 test.unit.proxy.test_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_404 test.unit.proxy.test_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_container_count test.unit.proxy.test_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_account test.unit.proxy.test_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_cache test.unit.proxy.test_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_200 test.unit.proxy.test_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_404 test.unit.proxy.test_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_invalid_account test.unit.proxy.test_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_no_cache test.unit.proxy.test_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_make_requests test.unit.proxy.test_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_transfer_headers test.unit.proxy.test_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348126.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/0a9/d1bd1cd25da042d18c1f1d84bc26b0a9/d1bd1cd25da042d18c1f1d84bc26b0a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348131.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/cdf/fe41fea5ffd650822d06e33a08916cdf/fe41fea5ffd650822d06e33a08916cdf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348136.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/f37/1b74102165f18aa0af90a4651688cf37/1b74102165f18aa0af90a4651688cf37.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348141.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/742/f456edcfb31fc44e9dd88ea6debff742/f456edcfb31fc44e9dd88ea6debff742.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348146.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4f1/645dcf5cdb9f24d7996ec28c13f334f1/645dcf5cdb9f24d7996ec28c13f334f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348151.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3a9/3f236f53b35ff91cbb7cc873030733a9/3f236f53b35ff91cbb7cc873030733a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348156.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/ccb/ba204a9106dec0c0e64d559cbddacccb/ba204a9106dec0c0e64d559cbddacccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348161.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/a01/d6e4713cee68cbbe26f209a599a97a01/d6e4713cee68cbbe26f209a599a97a01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348166.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/96c/9f1785ac6efaa24c572255c68955d96c/9f1785ac6efaa24c572255c68955d96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348171.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3ae/9c2d94c74d2e4e34a5f91c3f8ae253ae/9c2d94c74d2e4e34a5f91c3f8ae253ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348176.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/dce/da100bcec9c20bf1503cebc26a83bdce/da100bcec9c20bf1503cebc26a83bdce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348181.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/2fe/ea31ae915600f3cc8606ea05fcf4e2fe/ea31ae915600f3cc8606ea05fcf4e2fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348186.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/04f/4e736ed44421d4282d52ba261f96604f/4e736ed44421d4282d52ba261f96604f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348191.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/bed/5d8a45319f33a197c55b6019b5d44bed/5d8a45319f33a197c55b6019b5d44bed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348196.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/4f3/6f87571588752a39d28272e61ada34f3/6f87571588752a39d28272e61ada34f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348201.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/969/73cbf72857c4b0290fe47053b6dc8969/73cbf72857c4b0290fe47053b6dc8969.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348206.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/36d/651707b5dae18a941b43f0f61660a36d/651707b5dae18a941b43f0f61660a36d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348211.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e1a/1a647ac6f90f287730d70a464dd4ce1a/1a647ac6f90f287730d70a464dd4ce1a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348216.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/30e/e124a1ce5dea3c143f62aa560e54a30e/e124a1ce5dea3c143f62aa560e54a30e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348221.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/55d/a993e64b92658631515bc6141d80e55d/a993e64b92658631515bc6141d80e55d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348226.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/411/bf173784507133f6d5260e5a47e0e411/bf173784507133f6d5260e5a47e0e411.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348231.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/03c/c96a19e00f4fa75ead9f6dde82b3803c/c96a19e00f4fa75ead9f6dde82b3803c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348236.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/3ff/34045f77562af3d050d88046786ce3ff/34045f77562af3d050d88046786ce3ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348241.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d4d/82d04553ab33d48344d434747c45ad4d/82d04553ab33d48344d434747c45ad4d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348246.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/5e0/746bccc44d82a9a993848daa65c9a5e0/746bccc44d82a9a993848daa65c9a5e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348251.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/e87/fd503f23dd3b3b91263139c325ad5e87/fd503f23dd3b3b91263139c325ad5e87.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348256.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/9c8/3e429b05e0a104e971d064968bb569c8/3e429b05e0a104e971d064968bb569c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348261.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/d00/3e73565531f2383a60d820da70a7dd00/3e73565531f2383a60d820da70a7dd00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348266.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/1af/4b8aa28063785dbc1f6d480cf8cd91af/4b8aa28063785dbc1f6d480cf8cd91af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348271.00000-0, db: /tmp/tmp862pfffs/sda/containers/0/813/eb79afbf1dce6055e92ded3029ac1813/eb79afbf1dce6055e92ded3029ac1813.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'found': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'created': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_audit_root_container test.unit.container.test_sharder.TestSharder.test_audit_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container test.unit.container.test_sharder.TestSharder.test_audit_shard_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347038.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347038.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347038.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347038.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: my-txn-id) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txa791ef3cea8c40fd938a5-00665ca2ee) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txa791ef3cea8c40fd938a5-00665ca2ee) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.0:1000/sda (txn: txa791ef3cea8c40fd938a5-00665ca2ee) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txa791ef3cea8c40fd938a5-00665ca2ee) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx3897010f174648fb80728-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txb4a58dff7cbd413191e0a-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txb4a58dff7cbd413191e0a-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: txb4a58dff7cbd413191e0a-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txc9bea7bb331941ed8c3ba-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txc9bea7bb331941ed8c3ba-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe73210b6e86d4970bda19-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txaff914f88c4a485ba8f13-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: txaff914f88c4a485ba8f13-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx06cc21c6b2284dea8fc7a-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: tx06cc21c6b2284dea8fc7a-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx679bf0e78d27497884b72-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx679bf0e78d27497884b72-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx7606933ed4a547dbb5e9c-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx7606933ed4a547dbb5e9c-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: tx7606933ed4a547dbb5e9c-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx4fe6bc5109104cc8bfe2e-00665ca2f0) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode ... ok cont1 DEBUG: Could not load '/tmp/tmphawzdkwz/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmphawzdkwz/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmphawzdkwz/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmphawzdkwz/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:10 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0122 "-" 3610691 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:10 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0148 "-" 3610691 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:10 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0105 "-" 3610691 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:10 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0116 "-" 3610691 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:10 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa33d6a1cad0549e5ad853-00665ca2fe" "proxy-server 3610691" 0.0020 "-" 3610691 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:11 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:60079/sda1/1/a/c" "txa33d6a1cad0549e5ad853-00665ca2fe" "container-server 3610691" 0.0019 "-" 3610691 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:11 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:48149/sdb1/1/a/c" "txa33d6a1cad0549e5ad853-00665ca2fe" "container-server 3610691" 0.0016 "-" 3610691 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:11 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa33d6a1cad0549e5ad853-00665ca2fe" "proxy-server 3610691" 0.4633 "-" 3610691 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:11 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa33d6a1cad0549e5ad853-00665ca2fe" "proxy-server 3610691" 0.4794 "-" 3610691 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/11 PUT /v1/a/c HTTP/1.0 201 - - t - - - txa33d6a1cad0549e5ad853-00665ca2fe - 0.4910 - - 1717347070.591294289 1717347071.082336664 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:11 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txafd19fcf44fb4db498fba-00665ca2ff" "proxy-server 3610691" 0.0020 "-" 3610691 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:12 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:48149/sdb1/1/a1/c1" "txafd19fcf44fb4db498fba-00665ca2ff" "container-server 3610691" 0.0016 "-" 3610691 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:12 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:60079/sda1/1/a1/c1" "txafd19fcf44fb4db498fba-00665ca2ff" "container-server 3610691" 0.0032 "-" 3610691 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:12 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txafd19fcf44fb4db498fba-00665ca2ff" "proxy-server 3610691" 0.4165 "-" 3610691 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:12 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txafd19fcf44fb4db498fba-00665ca2ff" "proxy-server 3610691" 1.1338 "-" 3610691 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/12 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txafd19fcf44fb4db498fba-00665ca2ff - 1.1448 - - 1717347071.084552050 1717347072.229397774 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:12 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx86b2682ecbd341e1bdaa9-00665ca300" "proxy-server 3610691" 0.2698 "-" 3610691 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:13 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:60079/sda1/2/a/c1" "tx86b2682ecbd341e1bdaa9-00665ca300" "container-server 3610691" 0.0173 "-" 3610691 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:14 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:48149/sdb1/2/a/c1" "tx86b2682ecbd341e1bdaa9-00665ca300" "container-server 3610691" 0.0616 "-" 3610691 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:14 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx86b2682ecbd341e1bdaa9-00665ca300" "proxy-server 3610691" 1.0868 "-" 3610691 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:14 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx86b2682ecbd341e1bdaa9-00665ca300" "proxy-server 3610691" 1.5363 "-" 3610691 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/14 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx86b2682ecbd341e1bdaa9-00665ca300 - 1.8166 - - 1717347072.231336594 1717347074.047983408 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:14 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx7c4e1cd3b5b944af94f61-00665ca302" "proxy-server 3610691" 0.4199 "-" 3610691 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:15 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:48149/sdb1/1/a/c2" "tx7c4e1cd3b5b944af94f61-00665ca302" "container-server 3610691" 0.0020 "-" 3610691 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:15 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:60079/sda1/1/a/c2" "tx7c4e1cd3b5b944af94f61-00665ca302" "container-server 3610691" 0.0157 "-" 3610691 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:15 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx7c4e1cd3b5b944af94f61-00665ca302" "proxy-server 3610691" 0.5034 "-" 3610691 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:15 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx7c4e1cd3b5b944af94f61-00665ca302" "proxy-server 3610691" 0.9009 "-" 3610691 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/15 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx7c4e1cd3b5b944af94f61-00665ca302 - 1.3317 - - 1717347074.050529242 1717347075.382202625 2 test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4c3f2c139dbd4097b0c4b-00665ca2e6) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx4c3f2c139dbd4097b0c4b-00665ca2e6) test ERROR: Account HEAD returning 503 for [] (txn: tx4c3f2c139dbd4097b0c4b-00665ca2e6) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx80f6133d250b4d5588f19-00665ca2e9) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx80f6133d250b4d5588f19-00665ca2e9) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx80f6133d250b4d5588f19-00665ca2e9) test ERROR: Account HEAD returning 503 for [] (txn: tx80f6133d250b4d5588f19-00665ca2e9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx175194a7759a47f4bed13-00665ca2ec) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx2e72902e102e4f74817f9-00665ca2ed) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx7d48f420efd94119b9ca3-00665ca2ed) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant AllUsers WRITE permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type ... ok test INFO: - - - [02/Jun/2024:16:50:38 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0005 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0003 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0004 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:45 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.3086 "-" 3610637 2 test DEBUG: Unlinked retiring db '/tmp/tmpwhazhgov/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:50:48 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.5669 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:48 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:49 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 1.5015 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:49 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.1062 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:49 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0386 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:49 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:49 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0236 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:49 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0028 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0249 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0235 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:50 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0006 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:50 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:50 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0006 "-" 3610637 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:50:50 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0094 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:50:50 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0007 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0185 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0005 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0296 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0048 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0008 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.4039 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0046 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0200 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0234 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0226 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347043.63985', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347043.66246', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:40 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part ... ok sharder-test WARNING: Audit failed for root: overlapping ranges in state 'sharding': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpw97jnk8q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: own_shard_range reset to None should be 1717347038.00000, path: a/c, db: /tmp/tmpolelycpd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347038.00000.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': a-c a-f, a-f c-f, path: a/c, db: /tmp/tmpjf7wqj1p/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347039.00000-0, db: /tmp/tmppflq5c18/sda/containers/0/254/a5b7062b6aefa600679554ecde19b254/a5b7062b6aefa600679554ecde19b254.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347039.00000-0, db: /tmp/tmppflq5c18/sda/containers/0/254/a5b7062b6aefa600679554ecde19b254/a5b7062b6aefa600679554ecde19b254.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347042.00000-0, db: /tmp/tmppflq5c18/sda/containers/0/efe/7e3f3a14e65800ce89d33b68bdf6aefe/7e3f3a14e65800ce89d33b68bdf6aefe.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347042.00000-0, db: /tmp/tmppflq5c18/sda/containers/0/efe/7e3f3a14e65800ce89d33b68bdf6aefe/7e3f3a14e65800ce89d33b68bdf6aefe.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717347039.00000, (0, 0) as of 1717347039.00000, active as of 1717347039.00000> to ShardRange<'k' to 't' as of 1717347040.00000, (0, 0) as of 1717347040.00000, active as of 1717347040.00000>, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347044.00000, (0, 0) as of 1717347044.00000, shrinking as of 1717347041.00000> to ShardRange<'k' to 't' as of 1717347044.00000, (0, 0) as of 1717347044.00000, sharding as of 1717347045.00000>, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpp_smjfdf/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-304ed676a46bb5bfa920bf9a81b9572f-1717347043.00000-2, db: /tmp/tmppwa8z30x/sda/containers/0/11f/d3001c3ee0a620b1c4f8f367ffc5211f/d3001c3ee0a620b1c4f8f367ffc5211f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347040.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/605/47d127bb3fcceb3040e374e3e5f04605/47d127bb3fcceb3040e374e3e5f04605.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347040.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/605/47d127bb3fcceb3040e374e3e5f04605/47d127bb3fcceb3040e374e3e5f04605.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347049.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/f02/2f1b7eb21de7831425ff0f499e9cff02/2f1b7eb21de7831425ff0f499e9cff02.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347049.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/f02/2f1b7eb21de7831425ff0f499e9cff02/2f1b7eb21de7831425ff0f499e9cff02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347058.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/6c8/0eccbe8663701bdb7934bb03d60d46c8/0eccbe8663701bdb7934bb03d60d46c8.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347058.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/6c8/0eccbe8663701bdb7934bb03d60d46c8/0eccbe8663701bdb7934bb03d60d46c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347067.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/6d8/a6e1739d4b9fecec488cdf105f0f06d8/a6e1739d4b9fecec488cdf105f0f06d8.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347067.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/6d8/a6e1739d4b9fecec488cdf105f0f06d8/a6e1739d4b9fecec488cdf105f0f06d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347076.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/2c5/e140a7f2b59cd5060927ec25c44742c5/e140a7f2b59cd5060927ec25c44742c5.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347042.76008', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx4fe6bc5109104cc8bfe2e-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx2b12a41f2a4948e4bb688-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txace9e73533ba4c5097012-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txace9e73533ba4c5097012-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx096f78e67cc04dd9afb5e-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx096f78e67cc04dd9afb5e-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: tx096f78e67cc04dd9afb5e-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx8f0842ffd2824ba2bd433-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx8f0842ffd2824ba2bd433-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx8f0842ffd2824ba2bd433-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: tx8f0842ffd2824ba2bd433-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx9dc80656ccf14153ad055-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx9dc80656ccf14153ad055-00665ca2f0) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started ... ok test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: Container GET returning 503 for [] (txn: tx7be1cec3a1cc4a879e783-00665ca2c8) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: tx7e55e631508b4f96bb16e-00665ca2e3) test INFO: Client disconnected on read of '/a/c' (txn: txf1de2e619f77428f83e77-00665ca2e3) test INFO: Client disconnected on read of '/a/c' (txn: tx9fe95e8f872c4c3d80a0c-00665ca2e3) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9e2b83bd7cd408892dd2-00665ca2d9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest ... ok test DEBUG: Replicating db /tmp/tmp6c0r1eky/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'c149299b-70e2-4887-abfa-7cf26b483cd5-sda', '1717347041.49542', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('c149299b-70e2-4887-abfa-7cf26b483cd5-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'c149299b-70e2-4887-abfa-7cf26b483cd5-sda', '1717347041.49542', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('c149299b-70e2-4887-abfa-7cf26b483cd5-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxiv_r6jk/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'e81dd24f-5fd4-40ce-8fe5-e042e848dcba-sda', '1717347047.08910', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717347047.70712"]}', '1717344000.00000', 0, 1) test DEBUG: /tmp/tmpxiv_r6jk/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'e81dd24f-5fd4-40ce-8fe5-e042e848dcba-sda') test DEBUG: /tmp/tmpxiv_r6jk/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0161841s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'e81dd24f-5fd4-40ce-8fe5-e042e848dcba-sda', '1717347047.08910', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717347047.70712"]}', '1717344000.00000', 0, 1) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('e81dd24f-5fd4-40ce-8fe5-e042e848dcba-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpxiv_r6jk/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.38140 seconds (2.62189/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpxiv_r6jk/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '37ec186d-fbf9-4266-9268-d47563dc7b73-sda', '1717347050.11726', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('37ec186d-fbf9-4266-9268-d47563dc7b73-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', '37ec186d-fbf9-4266-9268-d47563dc7b73-sda', '1717347050.11726', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('37ec186d-fbf9-4266-9268-d47563dc7b73-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpt5fq5wqs/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'ed5fa1a4-3998-429b-a73e-b48bd62abe7e-sda', '1717347050.62251', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717347050.63114"]}', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmpt5fq5wqs/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'ed5fa1a4-3998-429b-a73e-b48bd62abe7e-sda') test DEBUG: /tmp/tmpt5fq5wqs/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.021106s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'ed5fa1a4-3998-429b-a73e-b48bd62abe7e-sda', '1717347050.62251', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717347050.63114"]}', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('ed5fa1a4-3998-429b-a73e-b48bd62abe7e-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpt5fq5wqs/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.16042 seconds (6.23369/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpt5fq5wqs/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', '0f25f3b8-52ca-4bb5-9e57-48fc830acd5a-sda', '1717347050.82469', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0f25f3b8-52ca-4bb5-9e57-48fc830acd5a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', '0f25f3b8-52ca-4bb5-9e57-48fc830acd5a-sda', '1717347050.82469', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0f25f3b8-52ca-4bb5-9e57-48fc830acd5a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpw_922c33/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '025304528eff65b5bc77d59900223ae4', '2485a675-3934-485f-a4ed-d360b0102bb6-sda', '1717347050.93946', '1717347050.00000', '0', '', '1717347050.00000', 1, 0) test DEBUG: /tmp/tmpw_922c33/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-nulo', 'created_at': '1717347052.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o-unu', 'created_at': '1717347053.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '2485a675-3934-485f-a4ed-d360b0102bb6-sda') test DEBUG: /tmp/tmpw_922c33/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0133536s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '025304528eff65b5bc77d59900223ae4', '2485a675-3934-485f-a4ed-d360b0102bb6-sda', '1717347050.93946', '1717347050.00000', '0', '', '1717347050.00000', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2485a675-3934-485f-a4ed-d360b0102bb6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpw_922c33/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14153 seconds (7.06542/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpw_922c33/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'f4f374db1f67260a3f0d702c652ef741', '7272f769-936f-4b75-a385-11578a12de91-sda', '1717347051.10517', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('7272f769-936f-4b75-a385-11578a12de91-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'f4f374db1f67260a3f0d702c652ef741', '7272f769-936f-4b75-a385-11578a12de91-sda', '1717347051.10517', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('7272f769-936f-4b75-a385-11578a12de91-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpw_922c33/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '917aa61a209546589fb6a1b608c461eb', 'fe046c1b-5a66-4ccd-92cb-f357dee5297c-sdb', '1717347050.93946', '1717347051.00000', '0', '', '1717347051.01631', 1, 0) test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row -1 REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-nulo', 'created_at': '1717347054.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o-unu', 'created_at': '1717347055.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'fe046c1b-5a66-4ccd-92cb-f357dee5297c-sdb') test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.0126965s) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': '2485a675-3934-485f-a4ed-d360b0102bb6-sda', 'sync_point': 2}],) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '917aa61a209546589fb6a1b608c461eb', 'fe046c1b-5a66-4ccd-92cb-f357dee5297c-sdb', '1717347050.93946', '1717347051.00000', '0', '', '1717347051.01631', 1, 0) test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, starting at row -1 REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-nulo', 'created_at': '1717347054.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o-unu', 'created_at': '1717347055.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'fe046c1b-5a66-4ccd-92cb-f357dee5297c-sdb') test DEBUG: Replicating db /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'f4f374db1f67260a3f0d702c652ef741', '500a81b9-76fb-420b-aa66-465cf395f0bc-', '1717347051.10517', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.2:1002/sdc, nothing to do REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'f4f374db1f67260a3f0d702c652ef741', '500a81b9-76fb-420b-aa66-465cf395f0bc-', '1717347051.10517', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.0:1000/sda, nothing to do test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0359182s) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': '2485a675-3934-485f-a4ed-d360b0102bb6-sda', 'sync_point': 2}],) test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.11667 seconds (17.14279/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:2 diff_capped:0 empty:0 hashmatch:0 no_change:2 remote_merge:0 rsync:0 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '72b5cbf5784f0aa30fb6353f4fad0e63', '500a81b9-76fb-420b-aa66-465cf395f0bc-', '1717347051.10517', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row 1 REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '1:/a/c/o-unu', 'created_at': '1717347055.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717347055.00000', 'deleted': 0, 'storage_policy_index': 0}], '500a81b9-76fb-420b-aa66-465cf395f0bc-') test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.0121458s) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': '7272f769-936f-4b75-a385-11578a12de91-sda', 'sync_point': 1}],) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '72b5cbf5784f0aa30fb6353f4fad0e63', '500a81b9-76fb-420b-aa66-465cf395f0bc-', '1717347051.10517', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, starting at row 1 REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '1:/a/c/o-unu', 'created_at': '1717347055.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717347055.00000', 'deleted': 0, 'storage_policy_index': 0}], '500a81b9-76fb-420b-aa66-465cf395f0bc-') test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize ... ok test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347050.23265', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self ... ok sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347076.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/2c5/e140a7f2b59cd5060927ec25c44742c5/e140a7f2b59cd5060927ec25c44742c5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347085.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/36f/b2cb8b75ba860f82b92ee540ea3b936f/b2cb8b75ba860f82b92ee540ea3b936f.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347085.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/36f/b2cb8b75ba860f82b92ee540ea3b936f/b2cb8b75ba860f82b92ee540ea3b936f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347094.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/bff/362a6031ae502708efb7361b11985bff/362a6031ae502708efb7361b11985bff.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347094.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/bff/362a6031ae502708efb7361b11985bff/362a6031ae502708efb7361b11985bff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347103.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/e8c/355c69648e3845527a498807316fbe8c/355c69648e3845527a498807316fbe8c.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347103.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/e8c/355c69648e3845527a498807316fbe8c/355c69648e3845527a498807316fbe8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347112.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/ff5/2a10ad94ca12b36cb0b9b12975514ff5/2a10ad94ca12b36cb0b9b12975514ff5.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347112.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/ff5/2a10ad94ca12b36cb0b9b12975514ff5/2a10ad94ca12b36cb0b9b12975514ff5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347112.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/ff5/2a10ad94ca12b36cb0b9b12975514ff5/2a10ad94ca12b36cb0b9b12975514ff5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347121.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/178/8f440d43f7d2128f420b320619ef7178/8f440d43f7d2128f420b320619ef7178.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347121.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/178/8f440d43f7d2128f420b320619ef7178/8f440d43f7d2128f420b320619ef7178.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347121.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/178/8f440d43f7d2128f420b320619ef7178/8f440d43f7d2128f420b320619ef7178.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347130.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/a07/53b2e91e870f54d29ed16769b9900a07/53b2e91e870f54d29ed16769b9900a07.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347130.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/a07/53b2e91e870f54d29ed16769b9900a07/53b2e91e870f54d29ed16769b9900a07.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347130.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/a07/53b2e91e870f54d29ed16769b9900a07/53b2e91e870f54d29ed16769b9900a07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347139.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/31e/4cecba67152b3844af978138096e131e/4cecba67152b3844af978138096e131e.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347139.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/31e/4cecba67152b3844af978138096e131e/4cecba67152b3844af978138096e131e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347139.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/31e/4cecba67152b3844af978138096e131e/4cecba67152b3844af978138096e131e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347148.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/5ce/22b9a96cd6c8dd116976801e5ddbd5ce/22b9a96cd6c8dd116976801e5ddbd5ce.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347148.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/5ce/22b9a96cd6c8dd116976801e5ddbd5ce/22b9a96cd6c8dd116976801e5ddbd5ce.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347148.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/5ce/22b9a96cd6c8dd116976801e5ddbd5ce/22b9a96cd6c8dd116976801e5ddbd5ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347157.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/95d/f566d1743c1521339be2aebe55b0f95d/f566d1743c1521339be2aebe55b0f95d.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347157.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/95d/f566d1743c1521339be2aebe55b0f95d/f566d1743c1521339be2aebe55b0f95d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347157.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/95d/f566d1743c1521339be2aebe55b0f95d/f566d1743c1521339be2aebe55b0f95d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347166.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/deb/4d7b2e3f4ec39531f57af67f232f2deb/4d7b2e3f4ec39531f57af67f232f2deb.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347166.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/deb/4d7b2e3f4ec39531f57af67f232f2deb/4d7b2e3f4ec39531f57af67f232f2deb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347166.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/deb/4d7b2e3f4ec39531f57af67f232f2deb/4d7b2e3f4ec39531f57af67f232f2deb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347175.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/3c1/efa747926e39ed5fd0059683786013c1/efa747926e39ed5fd0059683786013c1.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347175.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/3c1/efa747926e39ed5fd0059683786013c1/efa747926e39ed5fd0059683786013c1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347175.00000-2, db: /tmp/tmpzq8qso1d/sda/containers/0/3c1/efa747926e39ed5fd0059683786013c1/efa747926e39ed5fd0059683786013c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded ... ok test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0236 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0033 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0189 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0162 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0136 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0187 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0131 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0151 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0167 "-" 3610637 4 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0154 "-" 3610637 4 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0184 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:50 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0187 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:50 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0021 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0163 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0387 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0191 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0038 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0172 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0201 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0187 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0112 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:50:51 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0021 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0158 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0177 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0035 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0038 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0160 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0020 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0002 "-" 3610637 nada test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0002 "-" 3610637 999 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_legacy" 201 - "-" "-" "-" 0.0280 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "HEAD /sda1/p/a/c_legacy" 204 - "-" "-" "-" 0.0020 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0146 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "HEAD /sda1/p/a/c_one" 204 - "-" "-" "-" 0.0020 "-" 3610637 1 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0153 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "HEAD /sda1/p/a/c_two" 204 - "-" "-" "-" 0.0018 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0142 "-" 3610637 3 test INFO: - - - [02/Jun/2024:16:50:51 +0000] "HEAD /sda1/p/a/c_three" 204 - "-" "-" "-" 0.0019 "-" 3610637 3 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission ... ok test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx44755c1580034bfa86bfa-00665ca2e3) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx44755c1580034bfa86bfa-00665ca2e3) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx44755c1580034bfa86bfa-00665ca2e3) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx44755c1580034bfa86bfa-00665ca2e3) test ERROR: Container GET returning 503 for [] (txn: tx44755c1580034bfa86bfa-00665ca2e3) test INFO: Client disconnected on read of '/a/c' (txn: tx44755c1580034bfa86bfa-00665ca2e3) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: Container GET returning 503 for [] (txn: txd9baaa9ecc3941b59f16f-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: Container GET returning 503 for [] (txn: txba910aebf89346d79b6a5-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx968039d1d1ae4b46b50ba-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx968039d1d1ae4b46b50ba-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx968039d1d1ae4b46b50ba-00665ca2ea) test ERROR: Container GET returning 503 for [] (txn: tx968039d1d1ae4b46b50ba-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6b7559c713d94fe98ef85-00665ca2ea) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx6b7559c713d94fe98ef85-00665ca2ea) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx210fd4b3f7934b09991f0-00665ca2ea) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx210fd4b3f7934b09991f0-00665ca2ea) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx210fd4b3f7934b09991f0-00665ca2ea) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx210fd4b3f7934b09991f0-00665ca2ea) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txfb07aa73b1ee4f39b36b5-00665ca2ea) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx7bb0398a26e24a55a9651-00665ca2ea) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx7bb0398a26e24a55a9651-00665ca2ea) test INFO: Client disconnected on read of '/a/c' (txn: tx7bb0398a26e24a55a9651-00665ca2ea) test ERROR: Container GET returning 503 for [] (txn: txf5b11426801245cbba9af-00665ca2ea) test INFO: Client disconnected on read of '/a/c' (txn: txf5b11426801245cbba9af-00665ca2ea) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx1dc8ae895c4e43be90be8-00665ca2ee) test INFO: Client disconnected on read of '/a/c' (txn: tx1dc8ae895c4e43be90be8-00665ca2ee) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347053.44238', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy ... ok proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx9dc80656ccf14153ad055-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: tx9dc80656ccf14153ad055-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe97d183613664e4d9e818-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe97d183613664e4d9e818-00665ca2f0) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txe97d183613664e4d9e818-00665ca2f0) proxy-server ERROR: Object GET returning 503 for [] (txn: txe97d183613664e4d9e818-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx0d48f024de40484b8f085-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: tx1f266edac4264703acabb-00665ca2f0) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: tx1f266edac4264703acabb-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txfab2d4e979404c429229d-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx20b0316d9759422c83130-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txae0183e7ec51422fa6614-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx0fbc01da2411405295c4b-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txd847e2f8cc114b5da786d-00665ca2f0) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: txd847e2f8cc114b5da786d-00665ca2f0) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txd847e2f8cc114b5da786d-00665ca2f0) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txd847e2f8cc114b5da786d-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.0:1000/sda (txn: tx91a4dffa5adc49d6a5b58-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txaef1a1b583c84499b63a7-00665ca2f0) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: tx53664c2dc01643578f02b-00665ca2f1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing ... ok test DEBUG: /tmp/tmpw_922c33/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0172005s) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': '7272f769-936f-4b75-a385-11578a12de91-sda', 'sync_point': 1}],) test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '9a55ad14e3eca93b24a56567ff9fbcf9', 'c1eabc64-571c-45ce-812d-0be97280b44e-', '1717347057.73384', '1717347057.73019', '0', '', '1717347057.73019', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c1eabc64-571c-45ce-812d-0be97280b44e-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.74212.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '9a55ad14e3eca93b24a56567ff9fbcf9', 'c1eabc64-571c-45ce-812d-0be97280b44e-', '1717347057.73384', '1717347057.73019', '0', '', '1717347057.73019', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347057.74196', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347057.74196', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347057.74196', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347057.74235', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347057.74235', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347057.74212', 'epoch': '1717347057.74212', 'reported': 0, 'tombstones': -1}], 'c1eabc64-571c-45ce-812d-0be97280b44e-') test DEBUG: /tmp/tmpck6yh8m2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.74212.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpck6yh8m2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.74212.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'a8f4368f3f2fae8808112dba5fd8acfb', 'bc2973ae-a482-4619-a376-5d0150ba3e71-', '1717347057.85016', '1717347057.84579', '0', '', '1717347057.84579', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347057.92527', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347057.92527', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347057.92527', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347057.93260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347057.93260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347057.85850', 'epoch': '1717347057.85850', 'reported': 0, 'tombstones': -1}], 'bc2973ae-a482-4619-a376-5d0150ba3e71-') test DEBUG: /tmp/tmp6j75nqiu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.85850.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp6j75nqiu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.85850.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'a8f4368f3f2fae8808112dba5fd8acfb', 'bc2973ae-a482-4619-a376-5d0150ba3e71-', '1717347057.85016', '1717347057.91262', '0', '', '1717347057.84579', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347057.92527', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347057.92527', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347057.92527', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347057.93260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347057.93260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347057.85850', 'epoch': '1717347057.85850', 'reported': 0, 'tombstones': -1}], 'bc2973ae-a482-4619-a376-5d0150ba3e71-') test DEBUG: /tmp/tmp6j75nqiu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.85850.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp6j75nqiu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347057.85850.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '7002d5aff577e719cdb4c1d5b3ce33b2', '05e340ff-9dfe-4a96-97f0-3f0a027adbeb-', '1717347058.06964', '1717347058.06509', '0', '', '1717347058.06509', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347058.14416', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347058.14416', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347058.14416', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347058.15363', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347058.15363', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347058.08027', 'epoch': '1717347058.08027', 'reported': 0, 'tombstones': -1}], '05e340ff-9dfe-4a96-97f0-3f0a027adbeb-') test DEBUG: /tmp/tmpcwkp5ymi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.08027.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpcwkp5ymi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.08027.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '7002d5aff577e719cdb4c1d5b3ce33b2', '05e340ff-9dfe-4a96-97f0-3f0a027adbeb-', '1717347058.06964', '1717347058.13151', '0', '', '1717347058.06509', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347058.14416', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347058.14416', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347058.14416', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347058.15363', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347058.15363', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347058.08027', 'epoch': '1717347058.08027', 'reported': 0, 'tombstones': -1}], '05e340ff-9dfe-4a96-97f0-3f0a027adbeb-') test DEBUG: /tmp/tmpcwkp5ymi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.08027.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpcwkp5ymi/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.08027.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'e00d6a06839f07fbb9bfcbabd4c061fe', '651777dd-d21e-4bf6-972a-21c0cde27bee-', '1717347058.28831', '1717347058.28352', '0', '', '1717347058.28352', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 501 Not Implemented test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd68feb5ff38e40f88f348-00665ca2ee) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: txd68feb5ff38e40f88f348-00665ca2ee) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: tx9ecbf2c5f5d1498f982d0-00665ca2f7) test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:50 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347050.25549', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace ... ok proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7b9a23b93ebc46c59ffdf-00665ca2e4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing ... ok test INFO: - - - [02/Jun/2024:16:50:51 +0000] "PUT /sda1/p/a/c_four" 201 - "-" "-" "-" 0.0146 "-" 3610637 4 test INFO: - - - [02/Jun/2024:16:50:56 +0000] "HEAD /sda1/p/a/c_four" 204 - "-" "-" "-" 0.0112 "-" 3610637 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:59 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:59 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0002 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:59 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.3021 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0102 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0030 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:50:59 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0119 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0111 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0042 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0043 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:50:59 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0046 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0110 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0104 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0306 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0050 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0113 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0089 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0082 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0087 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0118 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0156 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0123 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0149 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0098 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0155 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0031 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0003 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0160 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3610637 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types ... ok test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible ... ok test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_options_info test.unit.proxy.controllers.test_info.TestInfoController.test_options_info ... ok proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.0:1000/sda (txn: tx6993155d7fe0455fa0bf5-00665ca2f1) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: tx6993155d7fe0455fa0bf5-00665ca2f1) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: tx6993155d7fe0455fa0bf5-00665ca2f1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx060f89c780e84c14b7dff-00665ca2f9) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx060f89c780e84c14b7dff-00665ca2f9) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txcf786a8229c44e61822d4-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx714488dee8c341d181652-00665ca2f9) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx714488dee8c341d181652-00665ca2f9) proxy-server ERROR: Object HEAD returning 503 for [] (txn: tx714488dee8c341d181652-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx95e2af021bc34283891dc-00665ca2f9) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx95e2af021bc34283891dc-00665ca2f9) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx95e2af021bc34283891dc-00665ca2f9) proxy-server ERROR: Object HEAD returning 503 for [507] (txn: tx95e2af021bc34283891dc-00665ca2f9) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: txb277bd87b3604a628fb6d-00665ca2f9) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txb277bd87b3604a628fb6d-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: txb277bd87b3604a628fb6d-00665ca2f9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbc049b37174245ce97997-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbc049b37174245ce97997-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txbc049b37174245ce97997-00665ca2f9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx91cf511a88ce4be4b890f-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx91cf511a88ce4be4b890f-00665ca2f9) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx91cf511a88ce4be4b890f-00665ca2f9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347053.45863', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:50:57 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347057.03000', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347058.36027', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347058.36027', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347058.36027', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347058.36967', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347058.36967', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347058.29585', 'epoch': '1717347058.29585', 'reported': 0, 'tombstones': -1}], '651777dd-d21e-4bf6-972a-21c0cde27bee-') test DEBUG: /tmp/tmpmtt193i6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.29585.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpmtt193i6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.29585.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'e00d6a06839f07fbb9bfcbabd4c061fe', '651777dd-d21e-4bf6-972a-21c0cde27bee-', '1717347058.28831', '1717347058.34879', '0', '', '1717347058.28352', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347058.36027', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347058.36027', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347058.36027', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347058.36967', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347058.36967', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347058.29585', 'epoch': '1717347058.29585', 'reported': 0, 'tombstones': -1}], '651777dd-d21e-4bf6-972a-21c0cde27bee-') test DEBUG: /tmp/tmpmtt193i6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.29585.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpmtt193i6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347058.29585.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'eaa909d3d9615740e255b6b91c546b06', '58f9154a-450b-4e09-84fa-b21c675df7c3-', '1717347062.94223', '1717347062.93855', '0', '', '1717347062.93855', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347063.02204', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.02204', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347063.02204', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347063.03139', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.03139', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347062.95448', 'epoch': '1717347062.95448', 'reported': 0, 'tombstones': -1}], '58f9154a-450b-4e09-84fa-b21c675df7c3-') test DEBUG: /tmp/tmpwq2kdtyf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347062.95448.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpwq2kdtyf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347062.95448.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, 'eaa909d3d9615740e255b6b91c546b06', '58f9154a-450b-4e09-84fa-b21c675df7c3-', '1717347062.94223', '1717347063.01115', '0', '', '1717347062.93855', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347063.02204', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.02204', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347063.02204', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347063.03139', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.03139', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347062.95448', 'epoch': '1717347062.95448', 'reported': 0, 'tombstones': -1}], '58f9154a-450b-4e09-84fa-b21c675df7c3-') test DEBUG: /tmp/tmpwq2kdtyf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347062.95448.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpwq2kdtyf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347062.95448.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'dfd2fa5cc5ed46a7b1902798c5ba8ddd', '95c6492a-70b7-4b64-8d12-499189bc378e-', '1717347063.14885', '1717347063.14518', '0', '', '1717347063.14518', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347063.15730', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347063.15730', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347063.15730', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347063.15761', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.15761', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347063.15742', 'epoch': '1717347063.15742', 'reported': 0, 'tombstones': -1}], '95c6492a-70b7-4b64-8d12-499189bc378e-') test DEBUG: /tmp/tmpdlv1x1aa/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.15742.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpdlv1x1aa/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.15742.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'dfd2fa5cc5ed46a7b1902798c5ba8ddd', '95c6492a-70b7-4b64-8d12-499189bc378e-', '1717347063.14885', '1717347063.21208', '0', '', '1717347063.14518', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347063.15730', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347063.15730', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347063.15730', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347063.15761', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.15761', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347063.15742', 'epoch': '1717347063.15742', 'reported': 0, 'tombstones': -1}], '95c6492a-70b7-4b64-8d12-499189bc378e-') test DEBUG: /tmp/tmpdlv1x1aa/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.15742.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpdlv1x1aa/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.15742.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:50:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347063.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347063.24119', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission ... ok test INFO: - - - [02/Jun/2024:16:51:00 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:03 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0144 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0129 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0044 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0120 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0120 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0033 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0133 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0002 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0108 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0006 "-" 3610637 0 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0121 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0006 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0005 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0122 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3610637 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0002 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0200 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:06 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0261 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0160 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:07 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0145 "-" 3610637 - test INFO: - - - [02/Jun/2024:16:51:07 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0107 "-" 3610637 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0219 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0140 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:51:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0187 "-" 3610637 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:51:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0144 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:07 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:07 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0004 "-" 3610637 2 test INFO: - - - [02/Jun/2024:16:51:07 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0105 "-" 3610637 2 sharder-test DEBUG: Unlinked retiring db '/tmp/tmp3uq21evw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf95630ad1b0042cead2f6-00665ca2fc) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf95630ad1b0042cead2f6-00665ca2fc) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txf95630ad1b0042cead2f6-00665ca2fc) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: txa000b615dd3442a4a42aa-00665ca2ff) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Expect: 100-continue on /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: Connection refused: Please insert ten dollars (txn: txa47e8b7708254516bff70-00665ca2ff) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.0:1000/sda (txn: tx8c129d80539941f2ada6a-00665ca2ff) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.1:1001/sdb (txn: tx3a44f19eeb894ddf9d571-00665ca2ff) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.2:1002/sdc (txn: tx72cdce0f0ab74b1b9502c-00665ca2ff) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Expect: 100-continue on /a/c/o.jpg: Timeout (Nones) (txn: tx5d98ab6aa55943d7977ed-00665ca2ff) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx36e2acd50e14499481436-00665ca2d1) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx3da33dab05bb4cce89da7-00665ca2d1) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txe34aa273c0dc40659fd74-00665ca2d1) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx8961f0ebfac74d668af4c-00665ca2d1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'dfd2fa5cc5ed46a7b1902798c5ba8ddd', '5071a63f-cff4-498c-8d68-724f1e931353-', '1717347063.29658', '1717347063.29229', '0', '', '1717347063.29229', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347063.89314', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347063.89314', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347063.89314', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347063.89346', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.89346', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347063.89327', 'epoch': '1717347063.89327', 'reported': 0, 'tombstones': -1}], '5071a63f-cff4-498c-8d68-724f1e931353-') test DEBUG: /tmp/tmpgti2b90b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.89327.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpgti2b90b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.89327.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'dfd2fa5cc5ed46a7b1902798c5ba8ddd', '5071a63f-cff4-498c-8d68-724f1e931353-', '1717347063.29658', '1717347063.95071', '0', '', '1717347063.29229', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347063.89314', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347063.89314', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347063.89314', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347063.89346', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347063.89346', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347063.89327', 'epoch': '1717347063.89327', 'reported': 0, 'tombstones': -1}], '5071a63f-cff4-498c-8d68-724f1e931353-') test DEBUG: /tmp/tmpgti2b90b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.89327.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpgti2b90b/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347063.89327.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '92fd7ce0f3a3a92b45bc975ca6d09821', 'bd1cf1b4-ccea-42a1-a12f-a835ce51d934-', '1717347071.99527', '1717347071.98577', '0', '', '1717347071.98577', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('bd1cf1b4-ccea-42a1-a12f-a835ce51d934-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717347073.09506.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '92fd7ce0f3a3a92b45bc975ca6d09821', 'bd1cf1b4-ccea-42a1-a12f-a835ce51d934-', '1717347071.99527', '1717347071.98577', '0', '', '1717347071.98577', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347072.22402', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347072.22402', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347072.22402', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347073.09546', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347073.09546', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347073.09506', 'epoch': '1717347073.09506', 'reported': 0, 'tombstones': -1}], 'bd1cf1b4-ccea-42a1-a12f-a835ce51d934-') test DEBUG: /tmp/tmpvu5oh_gt/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347073.09506.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpvu5oh_gt/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347073.09506.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '4e6732bb5bc507258e111f0272476480', '498b61f4-004d-4518-8ae0-b2d9117ae133-', '1717347075.47061', '1717347075.46721', '0', '', '1717347075.46721', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('498b61f4-004d-4518-8ae0-b2d9117ae133-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.50173.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '4e6732bb5bc507258e111f0272476480', '498b61f4-004d-4518-8ae0-b2d9117ae133-', '1717347075.47061', '1717347075.46721', '0', '', '1717347075.46721', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347075.48005', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347075.48005', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347075.48005', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347075.50212', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347075.50212', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347075.50173', 'epoch': '1717347075.50173', 'reported': 0, 'tombstones': -1}], '498b61f4-004d-4518-8ae0-b2d9117ae133-') test DEBUG: /tmp/tmpv4pq21t4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.50173.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpv4pq21t4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.50173.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4566a8499532e70b58aa36c5678dd48e', 'db7e1c2f-2130-48df-bc54-e96d5d271625-', '1717347075.74908', '1717347075.74421', '0', '', '1717347075.74421', 3, 0) test WARNING: /tmp/tmp0w54blj6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.77215.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmp0w54blj6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.77215.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4566a8499532e70b58aa36c5678dd48e', 'db7e1c2f-2130-48df-bc54-e96d5d271625-', '1717347075.74908', '1717347075.90172', '0', '', '1717347075.74421', 3, 0) test WARNING: /tmp/tmp0w54blj6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.77215.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmp0w54blj6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347075.77215.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4566a8499532e70b58aa36c5678dd48e', '24860922-2804-4ef2-ad06-26f90d997956-', '1717347075.99038', '1717347075.98617', '0', '', '1717347075.98617', 3, 0) test WARNING: /tmp/tmp8hvftqhw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347076.01907.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347060.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txac9ecd1efa384c239ddf4-00665ca2eb) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri ... ok proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp5sedbfqw/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp5sedbfqw/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp5sedbfqw/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp5sedbfqw/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:17 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.1221 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:17 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0184 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:17 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0372 "-" 3610637 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:17 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0149 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:17 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx0bfa8405bce24abaae243-00665ca305" "proxy-server 3610637" 0.0018 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:53741/sda1/1/a/c" "tx0bfa8405bce24abaae243-00665ca305" "container-server 3610637" 0.0016 "-" 3610637 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:37569/sdb1/1/a/c" "tx0bfa8405bce24abaae243-00665ca305" "container-server 3610637" 0.0016 "-" 3610637 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx0bfa8405bce24abaae243-00665ca305" "proxy-server 3610637" 0.0434 "-" 3610637 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx0bfa8405bce24abaae243-00665ca305" "proxy-server 3610637" 0.0223 "-" 3610637 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/18 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx0bfa8405bce24abaae243-00665ca305 - 0.0567 - - 1717347077.967787743 1717347078.024443865 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx4604b03f9bf842b7afb4d-00665ca306" "proxy-server 3610637" 0.0017 "-" 3610637 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:37569/sdb1/1/a1/c1" "tx4604b03f9bf842b7afb4d-00665ca306" "container-server 3610637" 0.0016 "-" 3610637 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:53741/sda1/1/a1/c1" "tx4604b03f9bf842b7afb4d-00665ca306" "container-server 3610637" 0.0013 "-" 3610637 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx4604b03f9bf842b7afb4d-00665ca306" "proxy-server 3610637" 0.0398 "-" 3610637 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx4604b03f9bf842b7afb4d-00665ca306" "proxy-server 3610637" 0.0234 "-" 3610637 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/18 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx4604b03f9bf842b7afb4d-00665ca306 - 0.0513 - - 1717347078.026281834 1717347078.077554464 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx123c4cb6ab9a4b06a1915-00665ca306" "proxy-server 3610637" 0.0106 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:53741/sda1/2/a/c1" "tx123c4cb6ab9a4b06a1915-00665ca306" "container-server 3610637" 0.0016 "-" 3610637 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:37569/sdb1/2/a/c1" "tx123c4cb6ab9a4b06a1915-00665ca306" "container-server 3610637" 0.0090 "-" 3610637 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx123c4cb6ab9a4b06a1915-00665ca306" "proxy-server 3610637" 0.0280 "-" 3610637 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx123c4cb6ab9a4b06a1915-00665ca306" "proxy-server 3610637" 0.0429 "-" 3610637 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/18 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx123c4cb6ab9a4b06a1915-00665ca306 - 0.0626 - - 1717347078.079435110 1717347078.142007113 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx87180f1fcf34475f945f1-00665ca306" "proxy-server 3610637" 0.0109 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:53741/sda1/1/a/c2" "tx87180f1fcf34475f945f1-00665ca306" "container-server 3610637" 0.0021 "-" 3610637 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:37569/sdb1/1/a/c2" "tx87180f1fcf34475f945f1-00665ca306" "container-server 3610637" 0.0196 "-" 3610637 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx87180f1fcf34475f945f1-00665ca306" "proxy-server 3610637" 0.0556 "-" 3610637 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:18 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx87180f1fcf34475f945f1-00665ca306" "proxy-server 3610637" 0.0425 "-" 3610637 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/18 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx87180f1fcf34475f945f1-00665ca306 - 0.0773 - - 1717347078.144057035 1717347078.221313238 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347069.56586', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347069.58236', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Expect: 100-continue on /a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx61fe86d119394667b1656-00665ca2ff) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.2:1002/sdc (txn: txdd17469c41894fe0a0dcc-00665ca303) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx0a5f16f590f04f258d0e9-00665ca303) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/a/c/o.jpg: Timeout (Nones) (txn: tx58f7611cd92e4a3caa53a-00665ca303) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to get final status of PUT to /v1/a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception (txn: tx8e11985672d34081bb4f3-00665ca303) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: txe8749e96939e4e41b28f8-00665ca303) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: tx7f71b0413e444b10ac1c4-00665ca307) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx27a02267f388426c948b0-00665ca307) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx27a02267f388426c948b0-00665ca307) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx27a02267f388426c948b0-00665ca307) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: tx27a02267f388426c948b0-00665ca307) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: Object returning 503 for [] (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Ignoring 404 from primary (txn: tx65680ef1158f4dcca18ec-00665ca2d4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: tx191e5a1c2ff749dd95a51-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: tx191e5a1c2ff749dd95a51-00665ca2f8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 0, Lengths: [490, 490, 490, 490, 490, 490, 490, 490, 490, 490]: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx62bfb34666324a6d9cd30-00665ca2f8) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347069.90248', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347069.91910', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission ... ok test WARNING: /tmp/tmp8hvftqhw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347076.01907.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '4566a8499532e70b58aa36c5678dd48e', '24860922-2804-4ef2-ad06-26f90d997956-', '1717347075.99038', '1717347076.08470', '0', '', '1717347075.98617', 3, 0) test WARNING: /tmp/tmp8hvftqhw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347076.01907.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmp8hvftqhw/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347076.01907.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, 'c33957eaeec4ad752eb7766d6eeec485', 'a4c7ef81-66a0-438a-87ac-fd858debd1a5-', '1717347077.86087', '1717347077.85634', '0', '', '1717347077.85634', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347078.00493', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.00493', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347078.00493', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347078.03792', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.03792', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347077.89728', 'epoch': '1717347077.89728', 'reported': 0, 'tombstones': -1}], 'a4c7ef81-66a0-438a-87ac-fd858debd1a5-') test DEBUG: /tmp/tmpk9kj465j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347077.89728.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpk9kj465j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347077.89728.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, 'c33957eaeec4ad752eb7766d6eeec485', 'a4c7ef81-66a0-438a-87ac-fd858debd1a5-', '1717347077.86087', '1717347077.98706', '0', '', '1717347077.85634', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347078.00493', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.00493', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347078.00493', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347078.03792', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.03792', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347077.89728', 'epoch': '1717347077.89728', 'reported': 0, 'tombstones': -1}], 'a4c7ef81-66a0-438a-87ac-fd858debd1a5-') test DEBUG: /tmp/tmpk9kj465j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347077.89728.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpk9kj465j/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347077.89728.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '7689ed73bbce6424ed37ea0c2df069f1', 'e5589e02-70bf-45d3-b9af-aa69c5b87966-', '1717347078.21174', '1717347078.20508', '0', '', '1717347078.20508', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347078.33632', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.33632', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347078.33632', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347078.34579', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.34579', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347078.24824', 'epoch': '1717347078.24824', 'reported': 0, 'tombstones': -1}], 'e5589e02-70bf-45d3-b9af-aa69c5b87966-') test DEBUG: /tmp/tmpp85yd2lx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.24824.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpp85yd2lx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.24824.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '7689ed73bbce6424ed37ea0c2df069f1', 'e5589e02-70bf-45d3-b9af-aa69c5b87966-', '1717347078.21174', '1717347078.32374', '0', '', '1717347078.20508', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347078.33632', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.33632', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347078.33632', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347078.34579', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.34579', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347078.24824', 'epoch': '1717347078.24824', 'reported': 0, 'tombstones': -1}], 'e5589e02-70bf-45d3-b9af-aa69c5b87966-') test DEBUG: /tmp/tmpp85yd2lx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.24824.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpp85yd2lx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.24824.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '41b6a15db5f6e5a27b6009fa9ca5f2be', 'a85da82e-42e9-414e-a185-e4ae9004df5d-', '1717347078.52658', '1717347078.52143', '0', '', '1717347078.52143', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347078.53989', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.53989', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347078.53989', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347078.54982', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.54982', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347078.54919', 'epoch': '1717347078.54919', 'reported': 0, 'tombstones': -1}], 'a85da82e-42e9-414e-a185-e4ae9004df5d-') test DEBUG: /tmp/tmpkgjuuc0o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.54919.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpkgjuuc0o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.54919.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '41b6a15db5f6e5a27b6009fa9ca5f2be', 'a85da82e-42e9-414e-a185-e4ae9004df5d-', '1717347078.52658', '1717347081.38421', '0', '', '1717347078.52143', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded ... ok proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txff18f035be5e49c1838fc-00665ca2f8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 400 Bad Request test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read ... ok proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1020, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1662, in read raise Exception('exception message') Exception: exception message (txn: tx87bdcdc6608c4097959e1-00665ca307) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception: boom (txn: tx7ea56166fd484f788b37a-00665ca30b) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Timeout (Nones) (txn: txdefc278c92554209878ad-00665ca30b) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txb5c1987d121746e2a9663-00665ca30b) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx3d8ebdf35d9544029b1f2-00665ca30b) proxy-server ERROR: ERROR 500 b'' Trying to PUT /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89 From Object Server 10.0.0.1:1001/sdb (txn: tx3d8ebdf35d9544029b1f2-00665ca30b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx41c44472bf9e4b3d9fb65-00665ca30b) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx83f82cca9a2142e196100-00665ca30b) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.2:1002/sdc (txn: tx83f82cca9a2142e196100-00665ca30b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [412] (txn: tx7363764c26a8405cbcf82-00665ca30b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending enough data (txn: tx00ea4c47c154421c80642-00665ca30f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data ... /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347079.07919', 'Content-Type': 'binary/octet-stream'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:14 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347074.91210', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:51:14 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347074.93552', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347078.53989', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.53989', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347078.53989', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347078.54982', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347078.54982', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347078.54919', 'epoch': '1717347078.54919', 'reported': 0, 'tombstones': -1}], 'a85da82e-42e9-414e-a185-e4ae9004df5d-') test DEBUG: /tmp/tmpkgjuuc0o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.54919.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpkgjuuc0o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347078.54919.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'f05777634b81ba9b5cc2599706f196ed', '22f4e851-40d5-42cd-a57e-492ac2a314c3-', '1717347083.79618', '1717347083.78988', '0', '', '1717347083.78988', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347083.80437', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347083.80437', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347083.80437', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347083.81622', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347083.81622', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347083.81522', 'epoch': '1717347083.81522', 'reported': 0, 'tombstones': -1}], '22f4e851-40d5-42cd-a57e-492ac2a314c3-') test DEBUG: /tmp/tmp25f4sirr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347083.81522.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp25f4sirr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347083.81522.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'f05777634b81ba9b5cc2599706f196ed', '22f4e851-40d5-42cd-a57e-492ac2a314c3-', '1717347083.79618', '1717347083.88418', '0', '', '1717347083.78988', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347083.80437', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347083.80437', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347083.80437', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347083.81622', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347083.81622', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347083.81522', 'epoch': '1717347083.81522', 'reported': 0, 'tombstones': -1}], '22f4e851-40d5-42cd-a57e-492ac2a314c3-') test DEBUG: /tmp/tmp25f4sirr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347083.81522.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp25f4sirr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347083.81522.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '627e069d30cb4886e82a101cc95e8735', '57a815b6-fcbc-4b95-b7a1-41b365c69ca8-sda', '1717347084.01050', '1717347084.00220', '0', '', '1717347084.00220', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('57a815b6-fcbc-4b95-b7a1-41b365c69ca8-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '64b76b0f6d925d79c46bfa4ab703a3b5', '5100155d-e8f6-4124-80ff-b4bddfb0a0e3-sda', '1717347084.08222', '1717347084.07298', '0', '', '1717347084.07298', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347084.12336', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347084.12336', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347084.12336', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347084.13214', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347084.13214', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347084.13085', 'epoch': '1717347084.13085', 'reported': 0, 'tombstones': -1}], '5100155d-e8f6-4124-80ff-b4bddfb0a0e3-sda') test DEBUG: /tmp/tmpqivq9_gx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpqivq9_gx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '64b76b0f6d925d79c46bfa4ab703a3b5', '5100155d-e8f6-4124-80ff-b4bddfb0a0e3-sda', '1717347084.08222', '1717347084.10941', '0', '', '1717347084.07298', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347084.12336', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347084.12336', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347084.12336', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347084.13214', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347084.13214', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347084.13085', 'epoch': '1717347084.13085', 'reported': 0, 'tombstones': -1}], '5100155d-e8f6-4124-80ff-b4bddfb0a0e3-sda') test DEBUG: /tmp/tmpqivq9_gx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpqivq9_gx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '64b76b0f6d925d79c46bfa4ab703a3b5', '32e97ac5-75cb-463b-8700-27effea18a41-sda', '1717347084.30213', '1717347084.29491', '0', '', '1717347084.29491', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347084.34110', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347084.34110', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347084.34110', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347084.35121', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347084.35121', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347084.35062', 'epoch': '1717347084.35062', 'reported': 0, 'tombstones': -1}], '32e97ac5-75cb-463b-8700-27effea18a41-sda') test DEBUG: /tmp/tmpq4109j3k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines ... ok proxy-server ERROR: ERROR Unhandled exception in request: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/server.py", line 598, in handle_request return handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 375, in wrapped return func(*a, **kw) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 275, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 263, in GETorHEAD resp = self._get_or_head_response(req, node_iter, partition, policy) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2930, in _get_or_head_response app_iter.kickoff(req, resp) File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1184, in kickoff self.stashed_iter = reiterate(self._real_iter(req, resp.headers)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3749, in reiterate chunk = next(iterator) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4452, in _get_next_item return super(StringAlong, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1470, in _iter_one_range for i, next_seg in enumerate(segment_iter): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1549, in _decode_segments_from_fragments segment = self.policy.pyeclib_driver.decode(fragments) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/ec_iface.py", line 300, in decode return self.ec_lib_reference.decode(fragment_payloads, ranges, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/core.py", line 91, in decode return pyeclib_c.decode(self.handle, _fragment_payloads, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pyeclib.ec_iface.ECInvalidFragmentMetadata: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx62bfb34666324a6d9cd30-00665ca2f8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Ignoring 404 from handoff (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server ERROR: Object returning 503 for [] (txn: txd94b8a4c354842d7b556b-00665ca2fe) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx3317894dc26e41cd9cdc1-00665ca31a) test INFO: Client disconnected on read of '/a/c' (txn: tx3317894dc26e41cd9cdc1-00665ca31a) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: tx4af14e1b9c6b4d5eb4bd0-00665ca31a) test INFO: Client disconnected on read of '/a/c' (txn: tx4af14e1b9c6b4d5eb4bd0-00665ca31a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: txbd063833b6334cf7862b1-00665ca31b) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: txbd063833b6334cf7862b1-00665ca31b) test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1795, in _send_chunk self.conn.send(to_send) File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 926, in send self.give_send(self, data) File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1426, in capture_send raise exc Exception: Exception while sending data on connection (txn: txe7727df64fe7433ca45fd-00665ca30f) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: ChunkWriteTimeout (Nones) (txn: tx4d4fa63dfcf74d299a1ba-00665ca312) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: tx3d7f63904e8f4996a7274-00665ca314) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347092.75768 <= '409 (1717347092.75768), 409 (1717347092.75768), 409 (1717347092.75768)' (txn: tx1029d0c4b3dc4a39a8ee1-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347092.76240 <= '409 (1717347092.76240), 409 (1717347092.76240), 409 (1717347092.76240)' (txn: tx177ed02197984b458fb7d-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347092.76612 <= '409 (1717347092.76612), 409 (1717347092.76612), 409 (1717347092.76612)' (txn: txd99504dd99414e6092afc-00665ca314) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347092.00000 <= '409 (1717347093.00000), 409 (1717347093.00000), 409 (1717347093.00000)' (txn: txae1b04378127443085f9a-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347094.00000 <= '409 (1717347095.00000), 409 (1717347095.00000), 409 (1717347095.00000)' (txn: txec8244fc9d26482e9d4f7-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347096.00000 <= '409 (1717347097.00000), 409 (1717347097.00000), 409 (1717347097.00000)' (txn: txc322b2c28fc2439390ca9-00665ca314) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347092.00000 <= '409 (1717347092.00000), 409 (1717347092.00000), 409 (1717347092.00000)' (txn: tx15e2af5e0abd48fbbf5d5-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347093.00000 <= '409 (1717347093.00000), 409 (1717347093.00000), 409 (1717347093.00000)' (txn: txe1e2712b46e7425d8f901-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347094.00000 <= '409 (1717347094.00000), 409 (1717347094.00000), 409 (1717347094.00000)' (txn: tx2092122f5ea6418fb6caf-00665ca314) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl ... ok proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx3b2f006327cb46bbbd42e-00665ca312) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx3b2f006327cb46bbbd42e-00665ca312) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx3b2f006327cb46bbbd42e-00665ca312) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: tx3b2f006327cb46bbbd42e-00665ca312) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission ... ok test WARNING: /tmp/tmpq4109j3k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '64b76b0f6d925d79c46bfa4ab703a3b5', '32e97ac5-75cb-463b-8700-27effea18a41-sda', '1717347084.30213', '1717347084.32152', '0', '', '1717347084.29491', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347084.34110', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347084.34110', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347084.34110', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347084.35121', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347084.35121', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347084.35062', 'epoch': '1717347084.35062', 'reported': 0, 'tombstones': -1}], '32e97ac5-75cb-463b-8700-27effea18a41-sda') test DEBUG: /tmp/tmpq4109j3k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpq4109j3k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7ad0575f88736279f61e97ac43627ced', '6eb26ca8-6a51-4a60-9bc1-cd749ebd233f-sda', '1717347094.13258', '1717347094.12019', '0', '', '1717347094.12019', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('6eb26ca8-6a51-4a60-9bc1-cd749ebd233f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmpxhmypr7g/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7ad0575f88736279f61e97ac43627ced', 'c9e0844d-aa87-44a5-aca4-c3332ca53680-sda', '1717347094.33457', '1717347094.32998', '0', '', '1717347094.32998', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('c9e0844d-aa87-44a5-aca4-c3332ca53680-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7ad0575f88736279f61e97ac43627ced', '9475d41d-0c10-4de4-aefd-a77303247e5b-sda', '1717347094.54134', '1717347094.53422', '0', '', '1717347094.53422', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('9475d41d-0c10-4de4-aefd-a77303247e5b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmprl0tz2d8/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7ad0575f88736279f61e97ac43627ced', '16be9e67-1ce4-42d9-b388-70797387ee66-sda', '1717347094.67735', '1717347094.67169', '0', '', '1717347094.67169', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('16be9e67-1ce4-42d9-b388-70797387ee66-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c810ae55-0fe9-41c0-8217-3636a9a66185-sda', '1717347094.92358', '1717347094.91841', '0', '', '1717347094.91841', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1rtrpcme/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b1621cc2-5433-44b0-926a-b109146d7c01-sdb', '1717347095.06601', '1717347096.00000', '0', '', '1717347096.00000', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b1621cc2-5433-44b0-926a-b109146d7c01-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b1621cc2-5433-44b0-926a-b109146d7c01-sdb', '1717347095.06601', '1717347096.00000', '0', '', '1717347096.00000', 0, 1) test ERROR: Quarantining DB /tmp/tmp1rtrpcme/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b1621cc2-5433-44b0-926a-b109146d7c01-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07593 seconds (13.17032/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5p54743n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '25719c99-43cb-46dc-8343-3c4d016d7fd9-sda', '1717347095.17403', '1717347095', '0', '', '1717347095', 0, 1) test DEBUG: /tmp/tmp5p54743n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '25719c99-43cb-46dc-8343-3c4d016d7fd9-sda', '1717347095.17403', '1717347095', '0', '', '1717347095', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('25719c99-43cb-46dc-8343-3c4d016d7fd9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07528 seconds (13.28394/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxc6915hl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'fdc46d2f1a27fe26355942ab086117bd', 'fce45655-09da-423f-861c-779b52256b86-sda', '1717347095.29978', '1717347095', '0', '', '1717347095', 1, 0) test DEBUG: /tmp/tmpxc6915hl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347097.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'fce45655-09da-423f-861c-779b52256b86-sda') test DEBUG: /tmp/tmpxc6915hl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0132573s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'fdc46d2f1a27fe26355942ab086117bd', 'fce45655-09da-423f-861c-779b52256b86-sda', '1717347095.29978', '1717347095', '0', '', '1717347095', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fce45655-09da-423f-861c-779b52256b86-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.16434 seconds (6.08512/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1vploz2r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2539fad3-3b8c-47d4-a056-db6d14128f5b-sda', '1717347095.51016', '1717347095', '0', '', '1717347095', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.75956', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.76397', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.76738', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.77016', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.77335', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.77744', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.78091', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.78403', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.78703', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.80284', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347087.80638', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission ... ok test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: txbd063833b6334cf7862b1-00665ca31b) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: txbd063833b6334cf7862b1-00665ca31b) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server ERROR: Object returning 503 for [] (txn: tx3f6860f53d194225b5df6-00665ca2fe) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from primary (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from primary (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from primary (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server ERROR: Object returning 503 for [] (txn: txa716e82652f3480cb08c9-00665ca305) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from primary (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from primary (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from primary (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx773188287c3842fd84d5a-00665ca305) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from primary (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from primary (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from primary (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from primary (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx1fbfeb85de9349a490b57-00665ca306) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx714d914e0e8941d690bf4-00665ca306) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx66c6f43b5a474904b4024-00665ca306) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body ... ok proxy-server DEBUG: Object PUT returning 202 for 409: 1717347092.00000 <= '409 (1717347092.00000)' (txn: txdf5982da3bd34c759118e-00665ca314) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347093.00000 <= '409 (1717347093.00000)' (txn: tx520493ed5ad64bbd86d46-00665ca319) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347094.00000 <= '409 (1717347094.00000)' (txn: tx8b48f91f49064b2288589-00665ca319) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx1a3f06ceee474305a64bc-00665ca31d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347101.00000 <= '409 (1717347102.00000)' (txn: tx663ad241698c4befae855-00665ca31d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [409, 412, 412] (txn: tx242480b165be4bc3a73ba-00665ca31d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347101.00000 <= '409 (unknown), 409 (unknown), 409 (unknown)' (txn: txba2008610651417c8944b-00665ca31d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347101.00000 <= '409 (1717347102.00000), 201 (unknown), 201 (unknown)' (txn: tx6ee5fbe9938c4989aebaa-00665ca31d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl ... ok test DEBUG: /tmp/tmp1vploz2r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2539fad3-3b8c-47d4-a056-db6d14128f5b-sda', '1717347095.51016', '1717347095', '0', '', '1717347095', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2539fad3-3b8c-47d4-a056-db6d14128f5b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 3.83969 seconds (0.26044/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpc_0ydria/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd8839beb5e8a566ee6536c1b63ffa7ba', '78a273d0-9d6b-4f24-8312-9d60cc79896e-sda', '1717347099.39895', '1717347099', '0', '', '1717347099', 1, 1) test DEBUG: /tmp/tmpc_0ydria/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd8839beb5e8a566ee6536c1b63ffa7ba', '78a273d0-9d6b-4f24-8312-9d60cc79896e-sda', '1717347099.39895', '1717347099', '0', '', '1717347099', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('78a273d0-9d6b-4f24-8312-9d60cc79896e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08836 seconds (11.31713/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpulb3ju20/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3367299c-4fd9-41bb-8a2c-59de7ee78c82-sda', '1717347103.10252', '1717347103', '0', '', '1717347103', 0, 0) test DEBUG: /tmp/tmpulb3ju20/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3367299c-4fd9-41bb-8a2c-59de7ee78c82-sda', '1717347103.10252', '1717347103', '0', '', '1717347103', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('3367299c-4fd9-41bb-8a2c-59de7ee78c82-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.83650 seconds (0.35255/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpd6mc3eo6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '73561727b00457686b416443bf522a87', '0a141532-29ae-44b5-9b4f-5e47a8c5237a-sda', '1717347107.57629', '1717347107', '0', '', '1717347107', 1, 1) test DEBUG: /tmp/tmpd6mc3eo6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347110.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '0a141532-29ae-44b5-9b4f-5e47a8c5237a-sda') test DEBUG: /tmp/tmpd6mc3eo6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0131161s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '73561727b00457686b416443bf522a87', '0a141532-29ae-44b5-9b4f-5e47a8c5237a-sda', '1717347107.57629', '1717347107', '0', '', '1717347107', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('0a141532-29ae-44b5-9b4f-5e47a8c5237a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13380 seconds (7.47385/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmppc2fr3_d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b3ff7921-ae5b-4ba2-91bb-7ca588c08dcd-sda', '1717347107.75989', '1717347107', '0', '', '1717347107', 0, 1) test DEBUG: /tmp/tmppc2fr3_d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b3ff7921-ae5b-4ba2-91bb-7ca588c08dcd-sda', '1717347107.75989', '1717347107', '0', '', '1717347107', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b3ff7921-ae5b-4ba2-91bb-7ca588c08dcd-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08520 seconds (11.73723/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptorx0b3r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '73561727b00457686b416443bf522a87', '17994dda-0db8-49ed-8c0b-00609bc4a937-sda', '1717347107.88790', '1717347107', '0', '', '1717347107', 1, 0) test DEBUG: /tmp/tmptorx0b3r/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '73561727b00457686b416443bf522a87', '17994dda-0db8-49ed-8c0b-00609bc4a937-sda', '1717347107.88790', '1717347107', '0', '', '1717347107', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('17994dda-0db8-49ed-8c0b-00609bc4a937-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08928 seconds (11.20108/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpiu27uz5x/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '06f1d3ea-2ad6-4131-9704-049707e82228-sda', '1717347108.04596', '1717347110', '0', '', '1717347110', 0, 1) test DEBUG: /tmp/tmpiu27uz5x/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '06f1d3ea-2ad6-4131-9704-049707e82228-sda', '1717347108.04596', '1717347110', '0', '', '1717347110', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('06f1d3ea-2ad6-4131-9704-049707e82228-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05701 seconds (17.54180/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create ... ok test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpiglp4b1j/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpiglp4b1j/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpiglp4b1j/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpiglp4b1j/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:46 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.6023 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:46 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.3221 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:46 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.1819 "-" 3610637 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.6373 "-" 3610637 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txfdb26aeba24549edbce1d-00665ca323" "proxy-server 3610637" 0.0021 "-" 3610637 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:34679/sdb1/1/a/c" "txfdb26aeba24549edbce1d-00665ca323" "container-server 3610637" 0.0033 "-" 3610637 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:40665/sda1/1/a/c" "txfdb26aeba24549edbce1d-00665ca323" "container-server 3610637" 0.0035 "-" 3610637 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txfdb26aeba24549edbce1d-00665ca323" "proxy-server 3610637" 0.1938 "-" 3610637 0 test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.25469', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.25844', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.26209', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.26528', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.26995', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.27505', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.27880', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.28226', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.28599', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.30180', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347085.30537', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Ignoring 404 from handoff (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server ERROR: Object returning 503 for [] (txn: txc1cf3dd28f9c4e68b50f2-00665ca30e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.11:1011/sdl (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.10:1010/sdk (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from primary (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server ERROR: Exception fetching fragments for '/a/c/o': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1496, in put_fragments_in_queue for fragment in frag_iter: File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2518, in _iter_bytes_from_response_part six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2490, in _iter_bytes_from_response_part chunk = part_file.read(self.app.object_chunk_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1133, in read raise ShortReadError( swift.common.exceptions.ShortReadError: Too few bytes; read 415, expecting 490 (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx5005f72c23a441be9f53b-00665ca312) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.6:1006/sdg (txn: tx18d90976d55b478796d05-00665ca312) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347090.45686', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165130Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347090.48347', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash ... ok cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txfdb26aeba24549edbce1d-00665ca323" "proxy-server 3610637" 0.0282 "-" 3610637 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/47 PUT /v1/a/c HTTP/1.0 201 - - t - - - txfdb26aeba24549edbce1d-00665ca323 - 0.2059 - - 1717347107.399473906 1717347107.605352402 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx1c17b9d4a8264567bc8a8-00665ca323" "proxy-server 3610637" 0.0021 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:40665/sda1/1/a1/c1" "tx1c17b9d4a8264567bc8a8-00665ca323" "container-server 3610637" 0.0018 "-" 3610637 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:34679/sdb1/1/a1/c1" "tx1c17b9d4a8264567bc8a8-00665ca323" "container-server 3610637" 0.0018 "-" 3610637 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1c17b9d4a8264567bc8a8-00665ca323" "proxy-server 3610637" 0.0554 "-" 3610637 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1c17b9d4a8264567bc8a8-00665ca323" "proxy-server 3610637" 0.0382 "-" 3610637 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/47 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx1c17b9d4a8264567bc8a8-00665ca323 - 0.0687 - - 1717347107.607583523 1717347107.676309347 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd500d4018e504ea79667a-00665ca323" "proxy-server 3610637" 0.0289 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:34679/sdb1/2/a/c1" "txd500d4018e504ea79667a-00665ca323" "container-server 3610637" 0.0107 "-" 3610637 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:40665/sda1/2/a/c1" "txd500d4018e504ea79667a-00665ca323" "container-server 3610637" 0.0019 "-" 3610637 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd500d4018e504ea79667a-00665ca323" "proxy-server 3610637" 0.0539 "-" 3610637 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd500d4018e504ea79667a-00665ca323" "proxy-server 3610637" 0.0372 "-" 3610637 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/47 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txd500d4018e504ea79667a-00665ca323 - 0.0941 - - 1717347107.678465366 1717347107.772570372 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txbe2e203ad99842b780a9f-00665ca323" "proxy-server 3610637" 0.0105 "-" 3610637 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:40665/sda1/1/a/c2" "txbe2e203ad99842b780a9f-00665ca323" "container-server 3610637" 0.0022 "-" 3610637 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:34679/sdb1/1/a/c2" "txbe2e203ad99842b780a9f-00665ca323" "container-server 3610637" 0.0107 "-" 3610637 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txbe2e203ad99842b780a9f-00665ca323" "proxy-server 3610637" 0.0557 "-" 3610637 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:47 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txbe2e203ad99842b780a9f-00665ca323" "proxy-server 3610637" 0.0391 "-" 3610637 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/47 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txbe2e203ad99842b780a9f-00665ca323 - 0.0783 - - 1717347107.774656057 1717347107.852906942 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': True, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 4}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.2, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source ... ok sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717347051.00000, (0, 0) as of 1717347051.00000, active as of 1717347051.00000> to ShardRange<'a' to 'b' as of 1717347051.00000, (0, 0) as of 1717347051.00000, sharding as of 1717347056.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108_1717347051.00000.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108_1717347051.00000.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108_1717347051.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108_1717347051.00000.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717347051.00000, (0, 0) as of 1717347051.00000, sharding as of 1717347057.00000> to ShardRange<'a' to 'b' as of 1717347051.00000, (0, 0) as of 1717347051.00000, sharded as of 1717347060.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347051.00000-2, db: /tmp/tmpf4z34uhq/sda/containers/0/108/0300d519f596390f8e42673705c41108/0300d519f596390f8e42673705c41108_1717347051.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347054.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/690/a62d3e7a5dca358a5e4fbfa8e1d1d690/a62d3e7a5dca358a5e4fbfa8e1d1d690.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347056.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c27/6669b4dbad7e57bb5a2d40f5e98b0c27/6669b4dbad7e57bb5a2d40f5e98b0c27.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347055.00000, (0, 0) as of 1717347055.00000, found as of 1717347055.00000> to ShardRange<'k' to 't' as of 1717347056.00000, (0, 0) as of 1717347056.00000, created as of 1717347056.00000>, path: .shards_a/shard_c_1717347056.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c27/6669b4dbad7e57bb5a2d40f5e98b0c27/6669b4dbad7e57bb5a2d40f5e98b0c27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347058.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/369/cc52094b338256a4783bf420509b8369/cc52094b338256a4783bf420509b8369.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347057.00000, (0, 0) as of 1717347057.00000, found as of 1717347057.00000> to ShardRange<'k' to 't' as of 1717347058.00000, (0, 0) as of 1717347058.00000, cleaved as of 1717347058.00000>, path: .shards_a/shard_c_1717347058.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/369/cc52094b338256a4783bf420509b8369/cc52094b338256a4783bf420509b8369.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347060.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d8a/8dc27574fc211945da808401f1691d8a/8dc27574fc211945da808401f1691d8a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347059.00000, (0, 0) as of 1717347059.00000, found as of 1717347059.00000> to ShardRange<'k' to 't' as of 1717347060.00000, (0, 0) as of 1717347060.00000, active as of 1717347060.00000>, path: .shards_a/shard_c_1717347060.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d8a/8dc27574fc211945da808401f1691d8a/8dc27574fc211945da808401f1691d8a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347062.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fae/dcefeba524ca022b04304ad762ca2fae/dcefeba524ca022b04304ad762ca2fae.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347061.00000, (0, 0) as of 1717347061.00000, found as of 1717347061.00000> to ShardRange<'k' to 't' as of 1717347062.00000, (0, 0) as of 1717347062.00000, shrinking as of 1717347062.00000>, path: .shards_a/shard_c_1717347062.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fae/dcefeba524ca022b04304ad762ca2fae/dcefeba524ca022b04304ad762ca2fae.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347062.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fae/dcefeba524ca022b04304ad762ca2fae/dcefeba524ca022b04304ad762ca2fae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347064.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f83/00d44deda34d77389fa078a1e2646f83/00d44deda34d77389fa078a1e2646f83.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347063.00000, (0, 0) as of 1717347063.00000, found as of 1717347063.00000> to ShardRange<'k' to 't' as of 1717347064.00000, (0, 0) as of 1717347064.00000, sharding as of 1717347064.00000>, path: .shards_a/shard_c_1717347064.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f83/00d44deda34d77389fa078a1e2646f83/00d44deda34d77389fa078a1e2646f83.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347064.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f83/00d44deda34d77389fa078a1e2646f83/00d44deda34d77389fa078a1e2646f83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347066.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f87/c9773b9e95cce7abafd6d3075914ef87/c9773b9e95cce7abafd6d3075914ef87.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347065.00000, (0, 0) as of 1717347065.00000, found as of 1717347065.00000> to ShardRange<'k' to 't' as of 1717347066.00000, (0, 0) as of 1717347066.00000, sharded as of 1717347066.00000>, path: .shards_a/shard_c_1717347066.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f87/c9773b9e95cce7abafd6d3075914ef87/c9773b9e95cce7abafd6d3075914ef87.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347066.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f87/c9773b9e95cce7abafd6d3075914ef87/c9773b9e95cce7abafd6d3075914ef87.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347068.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/884/f3e30a93426fcf16dee46158c58ef884/f3e30a93426fcf16dee46158c58ef884.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347067.00000, (0, 0) as of 1717347067.00000, found as of 1717347067.00000> to ShardRange<'k' to 't' as of 1717347068.00000, (0, 0) as of 1717347068.00000, shrunk as of 1717347068.00000>, path: .shards_a/shard_c_1717347068.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/884/f3e30a93426fcf16dee46158c58ef884/f3e30a93426fcf16dee46158c58ef884.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347068.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/884/f3e30a93426fcf16dee46158c58ef884/f3e30a93426fcf16dee46158c58ef884.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347070.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/13b/3faed025583b1436472eef27034af13b/3faed025583b1436472eef27034af13b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347069.00000, (0, 0) as of 1717347069.00000, created as of 1717347069.00000> to ShardRange<'k' to 't' as of 1717347070.00000, (0, 0) as of 1717347070.00000, found as of 1717347070.00000>, path: .shards_a/shard_c_1717347070.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/13b/3faed025583b1436472eef27034af13b/3faed025583b1436472eef27034af13b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347072.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/854/40b126d0f114bbaa285735632fbd3854/40b126d0f114bbaa285735632fbd3854.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347074.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1b8/40208c20d1c03368b4f91f174f9d51b8/40208c20d1c03368b4f91f174f9d51b8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347073.00000, (0, 0) as of 1717347073.00000, created as of 1717347073.00000> to ShardRange<'k' to 't' as of 1717347074.00000, (0, 0) as of 1717347074.00000, cleaved as of 1717347074.00000>, path: .shards_a/shard_c_1717347074.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1b8/40208c20d1c03368b4f91f174f9d51b8/40208c20d1c03368b4f91f174f9d51b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347076.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/67e/ff427f28aeca0c5ebe102ca610a1f67e/ff427f28aeca0c5ebe102ca610a1f67e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347075.00000, (0, 0) as of 1717347075.00000, created as of 1717347075.00000> to ShardRange<'k' to 't' as of 1717347076.00000, (0, 0) as of 1717347076.00000, active as of 1717347076.00000>, path: .shards_a/shard_c_1717347076.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/67e/ff427f28aeca0c5ebe102ca610a1f67e/ff427f28aeca0c5ebe102ca610a1f67e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347078.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/7a0/2e24b44784890694876558828d2db7a0/2e24b44784890694876558828d2db7a0.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347077.00000, (0, 0) as of 1717347077.00000, created as of 1717347077.00000> to ShardRange<'k' to 't' as of 1717347078.00000, (0, 0) as of 1717347078.00000, shrinking as of 1717347078.00000>, path: .shards_a/shard_c_1717347078.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/7a0/2e24b44784890694876558828d2db7a0/2e24b44784890694876558828d2db7a0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347078.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/7a0/2e24b44784890694876558828d2db7a0/2e24b44784890694876558828d2db7a0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347080.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/674/8c4ff932331b17b8d6b294913d00f674/8c4ff932331b17b8d6b294913d00f674.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347079.00000, (0, 0) as of 1717347079.00000, created as of 1717347079.00000> to ShardRange<'k' to 't' as of 1717347080.00000, (0, 0) as of 1717347080.00000, sharding as of 1717347080.00000>, path: .shards_a/shard_c_1717347080.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/674/8c4ff932331b17b8d6b294913d00f674/8c4ff932331b17b8d6b294913d00f674.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347080.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/674/8c4ff932331b17b8d6b294913d00f674/8c4ff932331b17b8d6b294913d00f674.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347082.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/653/22dd04a079c9c73286fcd327ffd59653/22dd04a079c9c73286fcd327ffd59653.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347081.00000, (0, 0) as of 1717347081.00000, created as of 1717347081.00000> to ShardRange<'k' to 't' as of 1717347082.00000, (0, 0) as of 1717347082.00000, sharded as of 1717347082.00000>, path: .shards_a/shard_c_1717347082.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/653/22dd04a079c9c73286fcd327ffd59653/22dd04a079c9c73286fcd327ffd59653.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347082.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/653/22dd04a079c9c73286fcd327ffd59653/22dd04a079c9c73286fcd327ffd59653.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347084.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/528/eb8e19f4369587c82c63766dd20bb528/eb8e19f4369587c82c63766dd20bb528.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347083.00000, (0, 0) as of 1717347083.00000, created as of 1717347083.00000> to ShardRange<'k' to 't' as of 1717347084.00000, (0, 0) as of 1717347084.00000, shrunk as of 1717347084.00000>, path: .shards_a/shard_c_1717347084.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/528/eb8e19f4369587c82c63766dd20bb528/eb8e19f4369587c82c63766dd20bb528.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347084.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/528/eb8e19f4369587c82c63766dd20bb528/eb8e19f4369587c82c63766dd20bb528.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347086.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/6c8/ab3e3aaa93bf298fc902a6ba9a2856c8/ab3e3aaa93bf298fc902a6ba9a2856c8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347085.00000, (0, 0) as of 1717347085.00000, cleaved as of 1717347085.00000> to ShardRange<'k' to 't' as of 1717347086.00000, (0, 0) as of 1717347086.00000, found as of 1717347086.00000>, path: .shards_a/shard_c_1717347086.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/6c8/ab3e3aaa93bf298fc902a6ba9a2856c8/ab3e3aaa93bf298fc902a6ba9a2856c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347088.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/4a2/e1f03f7ab73490ec83cf32b1b52824a2/e1f03f7ab73490ec83cf32b1b52824a2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347087.00000, (0, 0) as of 1717347087.00000, cleaved as of 1717347087.00000> to ShardRange<'k' to 't' as of 1717347088.00000, (0, 0) as of 1717347088.00000, created as of 1717347088.00000>, path: .shards_a/shard_c_1717347088.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/4a2/e1f03f7ab73490ec83cf32b1b52824a2/e1f03f7ab73490ec83cf32b1b52824a2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347090.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/441/5704168f2c5587131cac6ae15fb23441/5704168f2c5587131cac6ae15fb23441.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347092.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5b4/304cd83d07bc46824751aa55676f85b4/304cd83d07bc46824751aa55676f85b4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347091.00000, (0, 0) as of 1717347091.00000, cleaved as of 1717347091.00000> to ShardRange<'k' to 't' as of 1717347092.00000, (0, 0) as of 1717347092.00000, active as of 1717347092.00000>, path: .shards_a/shard_c_1717347092.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5b4/304cd83d07bc46824751aa55676f85b4/304cd83d07bc46824751aa55676f85b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347094.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/144/eb903e9211d5731adefe17da76094144/eb903e9211d5731adefe17da76094144.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347093.00000, (0, 0) as of 1717347093.00000, cleaved as of 1717347093.00000> to ShardRange<'k' to 't' as of 1717347094.00000, (0, 0) as of 1717347094.00000, shrinking as of 1717347094.00000>, path: .shards_a/shard_c_1717347094.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/144/eb903e9211d5731adefe17da76094144/eb903e9211d5731adefe17da76094144.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347094.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/144/eb903e9211d5731adefe17da76094144/eb903e9211d5731adefe17da76094144.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347096.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1e3/33eb9a0a1d014330e64543132c7d01e3/33eb9a0a1d014330e64543132c7d01e3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347095.00000, (0, 0) as of 1717347095.00000, cleaved as of 1717347095.00000> to ShardRange<'k' to 't' as of 1717347096.00000, (0, 0) as of 1717347096.00000, sharding as of 1717347096.00000>, path: .shards_a/shard_c_1717347096.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1e3/33eb9a0a1d014330e64543132c7d01e3/33eb9a0a1d014330e64543132c7d01e3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347096.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1e3/33eb9a0a1d014330e64543132c7d01e3/33eb9a0a1d014330e64543132c7d01e3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347098.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5c2/92915f28064587614d6a2440ad2705c2/92915f28064587614d6a2440ad2705c2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347097.00000, (0, 0) as of 1717347097.00000, cleaved as of 1717347097.00000> to ShardRange<'k' to 't' as of 1717347098.00000, (0, 0) as of 1717347098.00000, sharded as of 1717347098.00000>, path: .shards_a/shard_c_1717347098.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5c2/92915f28064587614d6a2440ad2705c2/92915f28064587614d6a2440ad2705c2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347098.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5c2/92915f28064587614d6a2440ad2705c2/92915f28064587614d6a2440ad2705c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347100.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/07a/caaa8e58b0c03221afdcf9bd9e23e07a/caaa8e58b0c03221afdcf9bd9e23e07a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347099.00000, (0, 0) as of 1717347099.00000, cleaved as of 1717347099.00000> to ShardRange<'k' to 't' as of 1717347100.00000, (0, 0) as of 1717347100.00000, shrunk as of 1717347100.00000>, path: .shards_a/shard_c_1717347100.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/07a/caaa8e58b0c03221afdcf9bd9e23e07a/caaa8e58b0c03221afdcf9bd9e23e07a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347100.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/07a/caaa8e58b0c03221afdcf9bd9e23e07a/caaa8e58b0c03221afdcf9bd9e23e07a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347102.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f5f/a8880dd7be0a60dbeab4af5a4d4def5f/a8880dd7be0a60dbeab4af5a4d4def5f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347101.00000, (0, 0) as of 1717347101.00000, active as of 1717347101.00000> to ShardRange<'k' to 't' as of 1717347102.00000, (0, 0) as of 1717347102.00000, found as of 1717347102.00000>, path: .shards_a/shard_c_1717347102.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f5f/a8880dd7be0a60dbeab4af5a4d4def5f/a8880dd7be0a60dbeab4af5a4d4def5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347104.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5f5/848fc08e41d8908cfc40b21cc58eb5f5/848fc08e41d8908cfc40b21cc58eb5f5.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347103.00000, (0, 0) as of 1717347103.00000, active as of 1717347103.00000> to ShardRange<'k' to 't' as of 1717347104.00000, (0, 0) as of 1717347104.00000, created as of 1717347104.00000>, path: .shards_a/shard_c_1717347104.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5f5/848fc08e41d8908cfc40b21cc58eb5f5/848fc08e41d8908cfc40b21cc58eb5f5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347106.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/26a/b249a460332cc7708e9aa09dbae4b26a/b249a460332cc7708e9aa09dbae4b26a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347105.00000, (0, 0) as of 1717347105.00000, active as of 1717347105.00000> to ShardRange<'k' to 't' as of 1717347106.00000, (0, 0) as of 1717347106.00000, cleaved as of 1717347106.00000>, path: .shards_a/shard_c_1717347106.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/26a/b249a460332cc7708e9aa09dbae4b26a/b249a460332cc7708e9aa09dbae4b26a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347108.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f61/3fd2a172bba97ffbb9957ee34e893f61/3fd2a172bba97ffbb9957ee34e893f61.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347110.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c33/25aad360c67f08e3120bac24d0880c33/25aad360c67f08e3120bac24d0880c33.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347109.00000, (0, 0) as of 1717347109.00000, active as of 1717347109.00000> to ShardRange<'k' to 't' as of 1717347110.00000, (0, 0) as of 1717347110.00000, shrinking as of 1717347110.00000>, path: .shards_a/shard_c_1717347110.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c33/25aad360c67f08e3120bac24d0880c33/25aad360c67f08e3120bac24d0880c33.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347110.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c33/25aad360c67f08e3120bac24d0880c33/25aad360c67f08e3120bac24d0880c33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347112.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fd5/6e239c9845067369a7cd671c6e516fd5/6e239c9845067369a7cd671c6e516fd5.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347111.00000, (0, 0) as of 1717347111.00000, active as of 1717347111.00000> to ShardRange<'k' to 't' as of 1717347112.00000, (0, 0) as of 1717347112.00000, sharding as of 1717347112.00000>, path: .shards_a/shard_c_1717347112.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fd5/6e239c9845067369a7cd671c6e516fd5/6e239c9845067369a7cd671c6e516fd5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347112.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fd5/6e239c9845067369a7cd671c6e516fd5/6e239c9845067369a7cd671c6e516fd5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347114.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/421/0e02b6ffc1e652655f3007353eabe421/0e02b6ffc1e652655f3007353eabe421.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347113.00000, (0, 0) as of 1717347113.00000, active as of 1717347113.00000> to ShardRange<'k' to 't' as of 1717347114.00000, (0, 0) as of 1717347114.00000, sharded as of 1717347114.00000>, path: .shards_a/shard_c_1717347114.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/421/0e02b6ffc1e652655f3007353eabe421/0e02b6ffc1e652655f3007353eabe421.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347114.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/421/0e02b6ffc1e652655f3007353eabe421/0e02b6ffc1e652655f3007353eabe421.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347116.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e88/f32d7c6905639e17aa70ae7755846e88/f32d7c6905639e17aa70ae7755846e88.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347115.00000, (0, 0) as of 1717347115.00000, active as of 1717347115.00000> to ShardRange<'k' to 't' as of 1717347116.00000, (0, 0) as of 1717347116.00000, shrunk as of 1717347116.00000>, path: .shards_a/shard_c_1717347116.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e88/f32d7c6905639e17aa70ae7755846e88/f32d7c6905639e17aa70ae7755846e88.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347116.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e88/f32d7c6905639e17aa70ae7755846e88/f32d7c6905639e17aa70ae7755846e88.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347118.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/b9f/12f7a9fb020772ea4c969115378afb9f/12f7a9fb020772ea4c969115378afb9f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347117.00000, (0, 0) as of 1717347117.00000, shrinking as of 1717347117.00000> to ShardRange<'k' to 't' as of 1717347118.00000, (0, 0) as of 1717347118.00000, found as of 1717347118.00000>, path: .shards_a/shard_c_1717347118.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/b9f/12f7a9fb020772ea4c969115378afb9f/12f7a9fb020772ea4c969115378afb9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347120.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/322/935a4e31bfa139f34c6179586f6c5322/935a4e31bfa139f34c6179586f6c5322.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347119.00000, (0, 0) as of 1717347119.00000, shrinking as of 1717347119.00000> to ShardRange<'k' to 't' as of 1717347120.00000, (0, 0) as of 1717347120.00000, created as of 1717347120.00000>, path: .shards_a/shard_c_1717347120.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/322/935a4e31bfa139f34c6179586f6c5322/935a4e31bfa139f34c6179586f6c5322.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347122.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/2b2/e4081cfe2bcbbd6b5149ad3530a5a2b2/e4081cfe2bcbbd6b5149ad3530a5a2b2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347121.00000, (0, 0) as of 1717347121.00000, shrinking as of 1717347121.00000> to ShardRange<'k' to 't' as of 1717347122.00000, (0, 0) as of 1717347122.00000, cleaved as of 1717347122.00000>, path: .shards_a/shard_c_1717347122.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/2b2/e4081cfe2bcbbd6b5149ad3530a5a2b2/e4081cfe2bcbbd6b5149ad3530a5a2b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347124.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ff9/de592809d70a6ee766811a4fc4b77ff9/de592809d70a6ee766811a4fc4b77ff9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347123.00000, (0, 0) as of 1717347123.00000, shrinking as of 1717347123.00000> to ShardRange<'k' to 't' as of 1717347124.00000, (0, 0) as of 1717347124.00000, active as of 1717347124.00000>, path: .shards_a/shard_c_1717347124.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ff9/de592809d70a6ee766811a4fc4b77ff9/de592809d70a6ee766811a4fc4b77ff9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347126.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/98f/a64016e57f03775afed1320f3741d98f/a64016e57f03775afed1320f3741d98f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347126.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/98f/a64016e57f03775afed1320f3741d98f/a64016e57f03775afed1320f3741d98f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347128.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ae1/577aeb1bdb4d8598271a0536a7e88ae1/577aeb1bdb4d8598271a0536a7e88ae1.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347127.00000, (0, 0) as of 1717347127.00000, shrinking as of 1717347127.00000> to ShardRange<'k' to 't' as of 1717347128.00000, (0, 0) as of 1717347128.00000, sharding as of 1717347128.00000>, path: .shards_a/shard_c_1717347128.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ae1/577aeb1bdb4d8598271a0536a7e88ae1/577aeb1bdb4d8598271a0536a7e88ae1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347128.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ae1/577aeb1bdb4d8598271a0536a7e88ae1/577aeb1bdb4d8598271a0536a7e88ae1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347130.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/314/ae5b578a86e7e0005d0a8e4528cb5314/ae5b578a86e7e0005d0a8e4528cb5314.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347129.00000, (0, 0) as of 1717347129.00000, shrinking as of 1717347129.00000> to ShardRange<'k' to 't' as of 1717347130.00000, (0, 0) as of 1717347130.00000, sharded as of 1717347130.00000>, path: .shards_a/shard_c_1717347130.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/314/ae5b578a86e7e0005d0a8e4528cb5314/ae5b578a86e7e0005d0a8e4528cb5314.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347130.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/314/ae5b578a86e7e0005d0a8e4528cb5314/ae5b578a86e7e0005d0a8e4528cb5314.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347132.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e26/dd5631c5f1218a75460df6316d5a3e26/dd5631c5f1218a75460df6316d5a3e26.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347131.00000, (0, 0) as of 1717347131.00000, shrinking as of 1717347131.00000> to ShardRange<'k' to 't' as of 1717347132.00000, (0, 0) as of 1717347132.00000, shrunk as of 1717347132.00000>, path: .shards_a/shard_c_1717347132.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e26/dd5631c5f1218a75460df6316d5a3e26/dd5631c5f1218a75460df6316d5a3e26.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347132.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e26/dd5631c5f1218a75460df6316d5a3e26/dd5631c5f1218a75460df6316d5a3e26.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347134.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/742/61c0a7cea612896205f10bbe3de8d742/61c0a7cea612896205f10bbe3de8d742.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347133.00000, (0, 0) as of 1717347133.00000, sharding as of 1717347133.00000> to ShardRange<'k' to 't' as of 1717347134.00000, (0, 0) as of 1717347134.00000, found as of 1717347134.00000>, path: .shards_a/shard_c_1717347134.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/742/61c0a7cea612896205f10bbe3de8d742/61c0a7cea612896205f10bbe3de8d742.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347136.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ee8/ab5051b8e3e9f0dbad1f22bf71feeee8/ab5051b8e3e9f0dbad1f22bf71feeee8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347135.00000, (0, 0) as of 1717347135.00000, sharding as of 1717347135.00000> to ShardRange<'k' to 't' as of 1717347136.00000, (0, 0) as of 1717347136.00000, created as of 1717347136.00000>, path: .shards_a/shard_c_1717347136.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ee8/ab5051b8e3e9f0dbad1f22bf71feeee8/ab5051b8e3e9f0dbad1f22bf71feeee8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347138.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/522/1439d31956a550f9be4d6b6a5cf9d522/1439d31956a550f9be4d6b6a5cf9d522.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347137.00000, (0, 0) as of 1717347137.00000, sharding as of 1717347137.00000> to ShardRange<'k' to 't' as of 1717347138.00000, (0, 0) as of 1717347138.00000, cleaved as of 1717347138.00000>, path: .shards_a/shard_c_1717347138.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/522/1439d31956a550f9be4d6b6a5cf9d522/1439d31956a550f9be4d6b6a5cf9d522.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347140.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/20a/94452f1292a9de1c663e74da16ea020a/94452f1292a9de1c663e74da16ea020a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347139.00000, (0, 0) as of 1717347139.00000, sharding as of 1717347139.00000> to ShardRange<'k' to 't' as of 1717347140.00000, (0, 0) as of 1717347140.00000, active as of 1717347140.00000>, path: .shards_a/shard_c_1717347140.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/20a/94452f1292a9de1c663e74da16ea020a/94452f1292a9de1c663e74da16ea020a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347142.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c82/eafe43082233985bcf2779ad5c697c82/eafe43082233985bcf2779ad5c697c82.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347141.00000, (0, 0) as of 1717347141.00000, sharding as of 1717347141.00000> to ShardRange<'k' to 't' as of 1717347142.00000, (0, 0) as of 1717347142.00000, shrinking as of 1717347142.00000>, path: .shards_a/shard_c_1717347142.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c82/eafe43082233985bcf2779ad5c697c82/eafe43082233985bcf2779ad5c697c82.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347142.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c82/eafe43082233985bcf2779ad5c697c82/eafe43082233985bcf2779ad5c697c82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347144.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/7b5/f056c015dd3d5f298687b7a5cf1d57b5/f056c015dd3d5f298687b7a5cf1d57b5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347144.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/7b5/f056c015dd3d5f298687b7a5cf1d57b5/f056c015dd3d5f298687b7a5cf1d57b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347146.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/35d/e1330ac65257ae1a30258ba9d854f35d/e1330ac65257ae1a30258ba9d854f35d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347145.00000, (0, 0) as of 1717347145.00000, sharding as of 1717347145.00000> to ShardRange<'k' to 't' as of 1717347146.00000, (0, 0) as of 1717347146.00000, sharded as of 1717347146.00000>, path: .shards_a/shard_c_1717347146.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/35d/e1330ac65257ae1a30258ba9d854f35d/e1330ac65257ae1a30258ba9d854f35d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347146.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/35d/e1330ac65257ae1a30258ba9d854f35d/e1330ac65257ae1a30258ba9d854f35d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347148.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c83/4eb84a680b9ab37581e9d5e5e88b7c83/4eb84a680b9ab37581e9d5e5e88b7c83.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347147.00000, (0, 0) as of 1717347147.00000, sharding as of 1717347147.00000> to ShardRange<'k' to 't' as of 1717347148.00000, (0, 0) as of 1717347148.00000, shrunk as of 1717347148.00000>, path: .shards_a/shard_c_1717347148.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c83/4eb84a680b9ab37581e9d5e5e88b7c83/4eb84a680b9ab37581e9d5e5e88b7c83.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347148.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c83/4eb84a680b9ab37581e9d5e5e88b7c83/4eb84a680b9ab37581e9d5e5e88b7c83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347150.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/67b/78f9704902103b3a34bcfad7bede367b/78f9704902103b3a34bcfad7bede367b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347149.00000, (0, 0) as of 1717347149.00000, sharded as of 1717347149.00000> to ShardRange<'k' to 't' as of 1717347150.00000, (0, 0) as of 1717347150.00000, found as of 1717347150.00000>, path: .shards_a/shard_c_1717347150.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/67b/78f9704902103b3a34bcfad7bede367b/78f9704902103b3a34bcfad7bede367b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347152.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/3ab/72fed473af5fc1b9a057a5a9b3f213ab/72fed473af5fc1b9a057a5a9b3f213ab.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347151.00000, (0, 0) as of 1717347151.00000, sharded as of 1717347151.00000> to ShardRange<'k' to 't' as of 1717347152.00000, (0, 0) as of 1717347152.00000, created as of 1717347152.00000>, path: .shards_a/shard_c_1717347152.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/3ab/72fed473af5fc1b9a057a5a9b3f213ab/72fed473af5fc1b9a057a5a9b3f213ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347154.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c0c/d34473e1f78c999ca57c538776eb2c0c/d34473e1f78c999ca57c538776eb2c0c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347153.00000, (0, 0) as of 1717347153.00000, sharded as of 1717347153.00000> to ShardRange<'k' to 't' as of 1717347154.00000, (0, 0) as of 1717347154.00000, cleaved as of 1717347154.00000>, path: .shards_a/shard_c_1717347154.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c0c/d34473e1f78c999ca57c538776eb2c0c/d34473e1f78c999ca57c538776eb2c0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347156.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5da/e7ba79cc7ba7abc6d4d2354ecb9795da/e7ba79cc7ba7abc6d4d2354ecb9795da.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347155.00000, (0, 0) as of 1717347155.00000, sharded as of 1717347155.00000> to ShardRange<'k' to 't' as of 1717347156.00000, (0, 0) as of 1717347156.00000, active as of 1717347156.00000>, path: .shards_a/shard_c_1717347156.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5da/e7ba79cc7ba7abc6d4d2354ecb9795da/e7ba79cc7ba7abc6d4d2354ecb9795da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347158.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8cf/36493e613811a4a2df615c3de11d98cf/36493e613811a4a2df615c3de11d98cf.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347157.00000, (0, 0) as of 1717347157.00000, sharded as of 1717347157.00000> to ShardRange<'k' to 't' as of 1717347158.00000, (0, 0) as of 1717347158.00000, shrinking as of 1717347158.00000>, path: .shards_a/shard_c_1717347158.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8cf/36493e613811a4a2df615c3de11d98cf/36493e613811a4a2df615c3de11d98cf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347158.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8cf/36493e613811a4a2df615c3de11d98cf/36493e613811a4a2df615c3de11d98cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347160.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ad2/d38752972df8e697063f68e2c586ead2/d38752972df8e697063f68e2c586ead2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347159.00000, (0, 0) as of 1717347159.00000, sharded as of 1717347159.00000> to ShardRange<'k' to 't' as of 1717347160.00000, (0, 0) as of 1717347160.00000, sharding as of 1717347160.00000>, path: .shards_a/shard_c_1717347160.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ad2/d38752972df8e697063f68e2c586ead2/d38752972df8e697063f68e2c586ead2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347160.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ad2/d38752972df8e697063f68e2c586ead2/d38752972df8e697063f68e2c586ead2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347162.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/10c/b4eb362fff1836d0c7b8d346b3dd310c/b4eb362fff1836d0c7b8d346b3dd310c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347162.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/10c/b4eb362fff1836d0c7b8d346b3dd310c/b4eb362fff1836d0c7b8d346b3dd310c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347164.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cd5/5adbaf6316d51a6d433f1ca7bfa90cd5/5adbaf6316d51a6d433f1ca7bfa90cd5.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347163.00000, (0, 0) as of 1717347163.00000, sharded as of 1717347163.00000> to ShardRange<'k' to 't' as of 1717347164.00000, (0, 0) as of 1717347164.00000, shrunk as of 1717347164.00000>, path: .shards_a/shard_c_1717347164.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cd5/5adbaf6316d51a6d433f1ca7bfa90cd5/5adbaf6316d51a6d433f1ca7bfa90cd5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347164.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cd5/5adbaf6316d51a6d433f1ca7bfa90cd5/5adbaf6316d51a6d433f1ca7bfa90cd5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347166.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/bd0/5483300273f28a88feb92f0e62ddebd0/5483300273f28a88feb92f0e62ddebd0.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347165.00000, (0, 0) as of 1717347165.00000, shrunk as of 1717347165.00000> to ShardRange<'k' to 't' as of 1717347166.00000, (0, 0) as of 1717347166.00000, found as of 1717347166.00000>, path: .shards_a/shard_c_1717347166.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/bd0/5483300273f28a88feb92f0e62ddebd0/5483300273f28a88feb92f0e62ddebd0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347168.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/31a/3c0f0aa0d7e9702d1e16a6470aeaf31a/3c0f0aa0d7e9702d1e16a6470aeaf31a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347167.00000, (0, 0) as of 1717347167.00000, shrunk as of 1717347167.00000> to ShardRange<'k' to 't' as of 1717347168.00000, (0, 0) as of 1717347168.00000, created as of 1717347168.00000>, path: .shards_a/shard_c_1717347168.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/31a/3c0f0aa0d7e9702d1e16a6470aeaf31a/3c0f0aa0d7e9702d1e16a6470aeaf31a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347170.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c71/5fe666decf41fedeebe167c9d4b06c71/5fe666decf41fedeebe167c9d4b06c71.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347169.00000, (0, 0) as of 1717347169.00000, shrunk as of 1717347169.00000> to ShardRange<'k' to 't' as of 1717347170.00000, (0, 0) as of 1717347170.00000, cleaved as of 1717347170.00000>, path: .shards_a/shard_c_1717347170.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c71/5fe666decf41fedeebe167c9d4b06c71/5fe666decf41fedeebe167c9d4b06c71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347172.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/0c2/a1a6454f4038ab21cf00ff56c0e960c2/a1a6454f4038ab21cf00ff56c0e960c2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347171.00000, (0, 0) as of 1717347171.00000, shrunk as of 1717347171.00000> to ShardRange<'k' to 't' as of 1717347172.00000, (0, 0) as of 1717347172.00000, active as of 1717347172.00000>, path: .shards_a/shard_c_1717347172.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/0c2/a1a6454f4038ab21cf00ff56c0e960c2/a1a6454f4038ab21cf00ff56c0e960c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347174.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d45/d7a587f9104f99492e8e0a7a916a8d45/d7a587f9104f99492e8e0a7a916a8d45.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347173.00000, (0, 0) as of 1717347173.00000, shrunk as of 1717347173.00000> to ShardRange<'k' to 't' as of 1717347174.00000, (0, 0) as of 1717347174.00000, shrinking as of 1717347174.00000>, path: .shards_a/shard_c_1717347174.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d45/d7a587f9104f99492e8e0a7a916a8d45/d7a587f9104f99492e8e0a7a916a8d45.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347174.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d45/d7a587f9104f99492e8e0a7a916a8d45/d7a587f9104f99492e8e0a7a916a8d45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347176.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f2e/cd82c9bd7f8c4bee9b591df8d2299f2e/cd82c9bd7f8c4bee9b591df8d2299f2e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347175.00000, (0, 0) as of 1717347175.00000, shrunk as of 1717347175.00000> to ShardRange<'k' to 't' as of 1717347176.00000, (0, 0) as of 1717347176.00000, sharding as of 1717347176.00000>, path: .shards_a/shard_c_1717347176.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f2e/cd82c9bd7f8c4bee9b591df8d2299f2e/cd82c9bd7f8c4bee9b591df8d2299f2e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347176.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f2e/cd82c9bd7f8c4bee9b591df8d2299f2e/cd82c9bd7f8c4bee9b591df8d2299f2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347178.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/0bc/26092811b84b8d802bf7ed550c4600bc/26092811b84b8d802bf7ed550c4600bc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347177.00000, (0, 0) as of 1717347177.00000, shrunk as of 1717347177.00000> to ShardRange<'k' to 't' as of 1717347178.00000, (0, 0) as of 1717347178.00000, sharded as of 1717347178.00000>, path: .shards_a/shard_c_1717347178.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/0bc/26092811b84b8d802bf7ed550c4600bc/26092811b84b8d802bf7ed550c4600bc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347178.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/0bc/26092811b84b8d802bf7ed550c4600bc/26092811b84b8d802bf7ed550c4600bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347180.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f87/d8fd7fd1875ff1622077ffb8b2006f87/d8fd7fd1875ff1622077ffb8b2006f87.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347180.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/f87/d8fd7fd1875ff1622077ffb8b2006f87/d8fd7fd1875ff1622077ffb8b2006f87.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347181.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/e4a/4ff52a4f0b8d105c4b061e6bd8f46e4a/4ff52a4f0b8d105c4b061e6bd8f46e4a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347183.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/0c9/6765f1d56d4cc2d69b82045edc1ce0c9/6765f1d56d4cc2d69b82045edc1ce0c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347185.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/61b/7422b6dece8da72045f762e62e12e61b/7422b6dece8da72045f762e62e12e61b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347187.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/632/efd2821d5d594a8bfbd14c39865f9632/efd2821d5d594a8bfbd14c39865f9632.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347189.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/14b/4769539ea5b078a53bccc806f614914b/4769539ea5b078a53bccc806f614914b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347191.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/77c/7a76ee61f8cd601f78f48b6bca7da77c/7a76ee61f8cd601f78f48b6bca7da77c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347193.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/079/3f6d98fe9831a9511c49d0c344f27079/3f6d98fe9831a9511c49d0c344f27079.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347195.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ada/b7a8e1317dc5fc5adf23a2b73dad2ada/b7a8e1317dc5fc5adf23a2b73dad2ada.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347197.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1bb/34b15d57ad92b60dff1c9f9daa7651bb/34b15d57ad92b60dff1c9f9daa7651bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347199.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/65f/b216fed295bf092710bd07ed6dfc065f/b216fed295bf092710bd07ed6dfc065f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347201.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/6da/5ddf682320c190bdb9ce673e929bb6da/5ddf682320c190bdb9ce673e929bb6da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347203.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/4aa/e9f7890b43e9758a60e0fa08dccd24aa/e9f7890b43e9758a60e0fa08dccd24aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347205.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ade/701daeeb47c391acdaa3cd54396e2ade/701daeeb47c391acdaa3cd54396e2ade.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347207.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/4f4/b40800d4b71fe79c0b9804f93af704f4/b40800d4b71fe79c0b9804f93af704f4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347209.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/efa/7dedbff713e66c465cda56a91c80defa/7dedbff713e66c465cda56a91c80defa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347211.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cb5/bae881495856038a1730493c12ee9cb5/bae881495856038a1730493c12ee9cb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347213.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/19f/a96e69dca35960f91df4068e3cac319f/a96e69dca35960f91df4068e3cac319f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347215.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/7ac/266fe037f36680583245fa60e936f7ac/266fe037f36680583245fa60e936f7ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347217.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/691/40e627a43d33aa5ebcaba87a71e11691/40e627a43d33aa5ebcaba87a71e11691.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347219.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fce/9bf54304dcf20721f84bc45498514fce/9bf54304dcf20721f84bc45498514fce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347221.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/eb5/301f9d2faceb6eff2d152d1a5ad17eb5/301f9d2faceb6eff2d152d1a5ad17eb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347223.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/a58/b656bf7d775d16eeec211dd0b0c01a58/b656bf7d775d16eeec211dd0b0c01a58.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347225.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ddd/334462a4f78242867b3a7fde015d6ddd/334462a4f78242867b3a7fde015d6ddd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347227.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/10f/12e85ce5d11d183df09f8f1dcaf9110f/12e85ce5d11d183df09f8f1dcaf9110f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347229.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cec/9b6f9708b2b4ff829cab7ca1a1142cec/9b6f9708b2b4ff829cab7ca1a1142cec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347231.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/af0/d93454ed13b6bc652ddc77ce7dd25af0/d93454ed13b6bc652ddc77ce7dd25af0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347233.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d70/e699c24afb20bf297b488506230fad70/e699c24afb20bf297b488506230fad70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347235.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/dea/fd8161d3a164b9b754f0bc77656cedea/fd8161d3a164b9b754f0bc77656cedea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347237.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/48b/a79a6112e0880d201af87ecbfbfd648b/a79a6112e0880d201af87ecbfbfd648b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347239.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fa6/f52108ef3a312b58f908af5c938aafa6/f52108ef3a312b58f908af5c938aafa6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347241.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/9a3/7a5af9f701107bfb55b7504dfb84b9a3/7a5af9f701107bfb55b7504dfb84b9a3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347243.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/033/f860046d5584b872e9da7b3ab62bb033/f860046d5584b872e9da7b3ab62bb033.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347245.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c47/7b11d15586cfb4cdf065caa707599c47/7b11d15586cfb4cdf065caa707599c47.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347245.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c47/7b11d15586cfb4cdf065caa707599c47/7b11d15586cfb4cdf065caa707599c47.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347247.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8b5/92f336efc4de250896854e9c5c2738b5/92f336efc4de250896854e9c5c2738b5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347247.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8b5/92f336efc4de250896854e9c5c2738b5/92f336efc4de250896854e9c5c2738b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347249.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5f0/1980987a7ef7d8c6fb33379f1d84f5f0/1980987a7ef7d8c6fb33379f1d84f5f0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347249.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/5f0/1980987a7ef7d8c6fb33379f1d84f5f0/1980987a7ef7d8c6fb33379f1d84f5f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347251.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/bb1/e69927ce2df29f3860e7b1cfa741abb1/e69927ce2df29f3860e7b1cfa741abb1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347251.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/bb1/e69927ce2df29f3860e7b1cfa741abb1/e69927ce2df29f3860e7b1cfa741abb1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347253.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/510/3a9349456c909c4a79ba1495bde35510/3a9349456c909c4a79ba1495bde35510.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347253.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/510/3a9349456c909c4a79ba1495bde35510/3a9349456c909c4a79ba1495bde35510.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347255.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/2ab/67f8d3ed2718ce7aa912a1be42c342ab/67f8d3ed2718ce7aa912a1be42c342ab.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347255.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/2ab/67f8d3ed2718ce7aa912a1be42c342ab/67f8d3ed2718ce7aa912a1be42c342ab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347257.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/6ee/1efc1189f762ecc514532e6116c006ee/1efc1189f762ecc514532e6116c006ee.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347257.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/6ee/1efc1189f762ecc514532e6116c006ee/1efc1189f762ecc514532e6116c006ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347259.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/065/1c9bea19c9cbfdc57ef6ef6359dfd065/1c9bea19c9cbfdc57ef6ef6359dfd065.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347259.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/065/1c9bea19c9cbfdc57ef6ef6359dfd065/1c9bea19c9cbfdc57ef6ef6359dfd065.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347261.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cdd/81b3468307c3bcfa546d6c41d69b4cdd/81b3468307c3bcfa546d6c41d69b4cdd.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347261.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/cdd/81b3468307c3bcfa546d6c41d69b4cdd/81b3468307c3bcfa546d6c41d69b4cdd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347263.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/07f/0e646aa308e01eb9f16954aeef6ad07f/0e646aa308e01eb9f16954aeef6ad07f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347263.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/07f/0e646aa308e01eb9f16954aeef6ad07f/0e646aa308e01eb9f16954aeef6ad07f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347265.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/eb2/1d68cdb46f6d583dbb4d1082a7355eb2/1d68cdb46f6d583dbb4d1082a7355eb2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347265.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/eb2/1d68cdb46f6d583dbb4d1082a7355eb2/1d68cdb46f6d583dbb4d1082a7355eb2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347267.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/a8e/df0db05d02cbf25f4b5f905d3512da8e/df0db05d02cbf25f4b5f905d3512da8e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347267.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/a8e/df0db05d02cbf25f4b5f905d3512da8e/df0db05d02cbf25f4b5f905d3512da8e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347269.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/78d/f8a7105dc80ae21fe5a1028a8aae378d/f8a7105dc80ae21fe5a1028a8aae378d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347269.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/78d/f8a7105dc80ae21fe5a1028a8aae378d/f8a7105dc80ae21fe5a1028a8aae378d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347271.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c8b/c50cba939e5c2a5de2d1f3d61b1e9c8b/c50cba939e5c2a5de2d1f3d61b1e9c8b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347271.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c8b/c50cba939e5c2a5de2d1f3d61b1e9c8b/c50cba939e5c2a5de2d1f3d61b1e9c8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347273.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/034/0ee012f6d4d0e2a64466386776bc3034/0ee012f6d4d0e2a64466386776bc3034.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347273.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/034/0ee012f6d4d0e2a64466386776bc3034/0ee012f6d4d0e2a64466386776bc3034.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347275.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/807/4e8a5da619c650c1eabab23ac73bb807/4e8a5da619c650c1eabab23ac73bb807.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347275.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/807/4e8a5da619c650c1eabab23ac73bb807/4e8a5da619c650c1eabab23ac73bb807.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347277.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/53b/716b4010d3d8d356f58b98e5f54eb53b/716b4010d3d8d356f58b98e5f54eb53b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347277.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/53b/716b4010d3d8d356f58b98e5f54eb53b/716b4010d3d8d356f58b98e5f54eb53b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347279.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d0b/145067981bc43daf3bc68f39e0ae4d0b/145067981bc43daf3bc68f39e0ae4d0b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347279.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/d0b/145067981bc43daf3bc68f39e0ae4d0b/145067981bc43daf3bc68f39e0ae4d0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347281.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8b9/e9c81f11ee5bddc40b2dcdb282f608b9/e9c81f11ee5bddc40b2dcdb282f608b9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347281.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8b9/e9c81f11ee5bddc40b2dcdb282f608b9/e9c81f11ee5bddc40b2dcdb282f608b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347283.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/9e8/a06a7753b0b9065539a3ddd824fd79e8/a06a7753b0b9065539a3ddd824fd79e8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347283.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/9e8/a06a7753b0b9065539a3ddd824fd79e8/a06a7753b0b9065539a3ddd824fd79e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347285.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/55c/d849158e8c49906f32badee9945b655c/d849158e8c49906f32badee9945b655c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347285.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/55c/d849158e8c49906f32badee9945b655c/d849158e8c49906f32badee9945b655c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347287.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fbc/6157a553550fcfcb61d70269a9479fbc/6157a553550fcfcb61d70269a9479fbc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347287.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/fbc/6157a553550fcfcb61d70269a9479fbc/6157a553550fcfcb61d70269a9479fbc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347289.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/bbf/12c39c12c863bfe05623a310dc816bbf/12c39c12c863bfe05623a310dc816bbf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347289.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/bbf/12c39c12c863bfe05623a310dc816bbf/12c39c12c863bfe05623a310dc816bbf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347291.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1c1/23336bec3d161d1e35161b6d336881c1/23336bec3d161d1e35161b6d336881c1.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347291.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/1c1/23336bec3d161d1e35161b6d336881c1/23336bec3d161d1e35161b6d336881c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347293.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c6f/45e33c852d116535e6549d78287ccc6f/45e33c852d116535e6549d78287ccc6f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347293.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/c6f/45e33c852d116535e6549d78287ccc6f/45e33c852d116535e6549d78287ccc6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347295.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/30e/d9959fe1493d97f6237462e66cbb430e/d9959fe1493d97f6237462e66cbb430e.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpw5x85ytz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '17100c813c2cfe7a9853a00b98d9407a', '98b49724-4102-4387-9751-d6663ce361a6-sda', '1717347108.14198', '1717347110', '0', '', '1717347110', 1, 0) test DEBUG: /tmp/tmpw5x85ytz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347111.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '98b49724-4102-4387-9751-d6663ce361a6-sda') test DEBUG: /tmp/tmpw5x85ytz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0121632s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '17100c813c2cfe7a9853a00b98d9407a', '98b49724-4102-4387-9751-d6663ce361a6-sda', '1717347108.14198', '1717347110', '0', '', '1717347110', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('98b49724-4102-4387-9751-d6663ce361a6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09689 seconds (10.32047/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpx66qevp6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '17100c813c2cfe7a9853a00b98d9407a', '304eca55-14a1-4bc6-9e05-951f3506e40f-sda', '1717347108.28482', '1717347108', '1717347109', '', '1717347109', 1, 1) test DEBUG: /tmp/tmpx66qevp6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347111.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '304eca55-14a1-4bc6-9e05-951f3506e40f-sda') test DEBUG: /tmp/tmpx66qevp6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0139093s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '17100c813c2cfe7a9853a00b98d9407a', '304eca55-14a1-4bc6-9e05-951f3506e40f-sda', '1717347108.28482', '1717347108', '1717347109', '', '1717347109', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('304eca55-14a1-4bc6-9e05-951f3506e40f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.62465 seconds (1.60089/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpmwdih3j6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e4d32c959999cb65d9f098fea0ba1dec', 'b4955914-f46e-46f8-b0fd-5651a00b3536-sda', '1717347109.93434', '1717347109', '1717347110', '', '1717347110', 1, 1) test DEBUG: /tmp/tmpmwdih3j6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e4d32c959999cb65d9f098fea0ba1dec', 'b4955914-f46e-46f8-b0fd-5651a00b3536-sda', '1717347109.93434', '1717347109', '1717347110', '', '1717347110', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b4955914-f46e-46f8-b0fd-5651a00b3536-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10040 seconds (9.95999/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptxix2wzv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0485db2c-5d8d-42f7-be6d-d5d73a787f21-sda', '1717347110.09026', '1717347112', '1717347111', '', '1717347112', 0, 1) test DEBUG: /tmp/tmptxix2wzv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0485db2c-5d8d-42f7-be6d-d5d73a787f21-sda', '1717347110.09026', '1717347112', '1717347111', '', '1717347112', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('0485db2c-5d8d-42f7-be6d-d5d73a787f21-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07214 seconds (13.86160/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpmbdik2if/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd3652df7f2eb095ccc8eb7cd5e429ad6', '737298f4-654c-48ee-9a34-c16ce66e869d-sda', '1717347110.21825', '1717347112', '1717347111', '', '1717347112', 1, 0) test DEBUG: /tmp/tmpmbdik2if/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347114.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '737298f4-654c-48ee-9a34-c16ce66e869d-sda') test DEBUG: /tmp/tmpmbdik2if/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.222128s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd3652df7f2eb095ccc8eb7cd5e429ad6', '737298f4-654c-48ee-9a34-c16ce66e869d-sda', '1717347110.21825', '1717347112', '1717347111', '', '1717347112', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('737298f4-654c-48ee-9a34-c16ce66e869d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.47049 seconds (2.12546/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmppfy7553n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9c5f1936-847d-41ea-a2bf-0bc030434298-sda', '1717347110.75350', '1717347112', '1717347111', '', '1717347112', 0, 0) test DEBUG: /tmp/tmppfy7553n/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347095.21756', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165135Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347095.24902', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash ... ok proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpvw722ez3/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpvw722ez3/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpvw722ez3/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpvw722ez3/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:52 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0169 "-" 3610586 - HTTP PERF: 0.02170 seconds to PUT 127.0.0.1:56403 /sda1/0/a) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:52 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0135 "-" 3610586 - HTTP PERF: 0.01585 seconds to PUT 127.0.0.1:47023 /sdb1/0/a) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:52 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0127 "-" 3610586 - HTTP PERF: 0.01576 seconds to PUT 127.0.0.1:56403 /sda1/2/a1) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:55 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.5698 "-" 3610586 - HTTP PERF: 0.57245 seconds to PUT 127.0.0.1:47023 /sdb1/2/a1) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:55 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx10d1d7abf31940a3ae434-00665ca32b" "proxy-server 3610586" 0.0019 "-" 3610586 - HTTP PERF: 0.00393 seconds to HEAD 127.0.0.1:47023 /sdb1/0/a?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:57 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:57809/sda1/3/a/c" "tx10d1d7abf31940a3ae434-00665ca32b" "container-server 3610586" 0.0016 "-" 3610586 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:57 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:44233/sdb1/3/a/c" "tx10d1d7abf31940a3ae434-00665ca32b" "container-server 3610586" 0.0014 "-" 3610586 0 HTTP PERF: 0.00663 seconds to PUT 127.0.0.1:56403 /sda1/0/a/c) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:57 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx10d1d7abf31940a3ae434-00665ca32b" "proxy-server 3610586" 0.9494 "-" 3610586 0 HTTP PERF: 0.95126 seconds to PUT 127.0.0.1:47023 /sdb1/0/a/c) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:57 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "tx10d1d7abf31940a3ae434-00665ca32b" "proxy-server 3610586" 1.5314 "-" 3610586 0 HTTP PERF: 1.53474 seconds to PUT 127.0.0.1:57809 /sda1/3/a/c) HTTP PERF: 1.53580 seconds to PUT 127.0.0.1:44233 /sdb1/3/a/c) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/57 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx10d1d7abf31940a3ae434-00665ca32b - 1.5441 - - 1717347115.849647522 1717347117.393716812 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:57 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx10e3f4fdfe454a568936e-00665ca32d" "proxy-server 3610586" 0.0018 "-" 3610586 - HTTP PERF: 0.00359 seconds to HEAD 127.0.0.1:47023 /sdb1/2/a1?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:44233/sdb1/0/a1/c1" "tx10e3f4fdfe454a568936e-00665ca32d" "container-server 3610586" 0.1513 "-" 3610586 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:57809/sda1/0/a1/c1" "tx10e3f4fdfe454a568936e-00665ca32d" "container-server 3610586" 0.0020 "-" 3610586 0 HTTP PERF: 1.01903 seconds to PUT 127.0.0.1:56403 /sda1/2/a1/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx10e3f4fdfe454a568936e-00665ca32d" "proxy-server 3610586" 1.7930 "-" 3610586 0 HTTP PERF: 0.15824 seconds to PUT 127.0.0.1:47023 /sdb1/2/a1/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx10e3f4fdfe454a568936e-00665ca32d" "proxy-server 3610586" 1.0194 "-" 3610586 0 HTTP PERF: 1.79621 seconds to PUT 127.0.0.1:44233 /sdb1/0/a1/c1) HTTP PERF: 1.79730 seconds to PUT 127.0.0.1:57809 /sda1/0/a1/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/59 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx10e3f4fdfe454a568936e-00665ca32d - 1.8046 - - 1717347117.395703793 1717347119.200333357 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txc920cf2abb5d46ce98867-00665ca32f" "proxy-server 3610586" 0.5701 "-" 3610586 - HTTP PERF: 0.57234 seconds to HEAD 127.0.0.1:56403 /sda1/0/a?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:44233/sdb1/3/a/c1" "txc920cf2abb5d46ce98867-00665ca32f" "container-server 3610586" 0.0042 "-" 3610586 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:57809/sda1/3/a/c1" "txc920cf2abb5d46ce98867-00665ca32f" "container-server 3610586" 0.0141 "-" 3610586 1 HTTP PERF: 0.02256 seconds to PUT 127.0.0.1:47023 /sdb1/0/a/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txc920cf2abb5d46ce98867-00665ca32f" "proxy-server 3610586" 0.0387 "-" 3610586 1 HTTP PERF: 0.04069 seconds to PUT 127.0.0.1:56403 /sda1/0/a/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:51:59 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txc920cf2abb5d46ce98867-00665ca32f" "proxy-server 3610586" 0.2113 "-" 3610586 1 HTTP PERF: 0.21363 seconds to PUT 127.0.0.1:57809 /sda1/3/a/c1) HTTP PERF: 0.21490 seconds to PUT 127.0.0.1:44233 /sdb1/3/a/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/51/59 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txc920cf2abb5d46ce98867-00665ca32f - 0.7915 - - 1717347119.202225685 1717347119.993686676 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:00 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1e8507f34d9348fa8d8ee-00665ca32f" "proxy-server 3610586" 0.0122 "-" 3610586 - HTTP PERF: 0.01514 seconds to HEAD 127.0.0.1:56403 /sda1/0/a?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:00 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:44233/sdb1/0/a/c2" "tx1e8507f34d9348fa8d8ee-00665ca32f" "container-server 3610586" 0.0024 "-" 3610586 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:00 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:57809/sda1/0/a/c2" "tx1e8507f34d9348fa8d8ee-00665ca32f" "container-server 3610586" 0.0095 "-" 3610586 2 HTTP PERF: 0.03384 seconds to PUT 127.0.0.1:47023 /sdb1/0/a/c2) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:00 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx1e8507f34d9348fa8d8ee-00665ca32f" "proxy-server 3610586" 0.0554 "-" 3610586 2 HTTP PERF: 0.01826 seconds to PUT 127.0.0.1:56403 /sda1/0/a/c2) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:00 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx1e8507f34d9348fa8d8ee-00665ca32f" "proxy-server 3610586" 0.0349 "-" 3610586 2 HTTP PERF: 0.05994 seconds to PUT 127.0.0.1:57809 /sda1/0/a/c2) HTTP PERF: 0.06127 seconds to PUT 127.0.0.1:44233 /sdb1/0/a/c2) test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/greenpool.py", line 87, in _spawn_n_impl func(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/x_profile/profile_model.py", line 274, in dump_profile profiler.dump_stats(tmpfn) File "/usr/lib/python3.12/profile.py", line 394, in dump_stats with open(file, 'wb') as f: ^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/tmp/log/swift/profile/default.profile3610486.tmp' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_valueTraceback (most recent call last): ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' File "/usr/lib/python3/dist-packages/eventlet/greenpool.py", line 87, in _spawn_n_impl func(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/x_profile/profile_model.py", line 274, in dump_profile profiler.dump_stats(tmpfn) File "/usr/lib/python3.12/profile.py", line 394, in dump_stats with open(file, 'wb') as f: ^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/tmp/log/swift/profile/default.profile3610486.tmp' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/00 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx1e8507f34d9348fa8d8ee-00665ca32f - 0.0815 - - 1717347119.996113777 1717347120.077589512 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:51] "GET https://cluster.domain/bucket/key HTTP/1.0" 200 106 0.000137 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:51] "GET http://cluster.domain/v1/acct/cont/obj HTTP/1.0" 200 112 0.000088 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:51] "GET ftp://cluster.domain/bucket/key HTTP/1.0" 200 126 0.000049 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:51] "GET https://cluster.domain HTTP/1.0" 200 117 0.000045 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:51] "GET http:omg//wtf/bbq HTTP/1.0" 200 112 0.000056 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: tx08e0632494d54a6ca3b61-00665ca327) proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) test ERROR: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD') (txn: test-trans-id) proxy INFO: wsgi exiting (txn: test-trans-id) proxy INFO: (3610488) wsgi exited, is_accepting=True (txn: test-trans-id) proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: test-trans-id) proxy INFO: wsgi exiting test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple ... ok test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star test.unit.common.test_swob.TestConditionalIfMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match ... ok test.unit.common.test_swob.TestEtag.test_normalize_bytes test.unit.common.test_swob.TestEtag.test_normalize_bytes ... ok test.unit.common.test_swob.TestEtag.test_normalize_etag test.unit.common.test_swob.TestEtag.test_normalize_etag ... ok test.unit.common.test_swob.TestRequest.test_401_invalid_path test.unit.common.test_swob.TestRequest.test_401_invalid_path ... ok test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path ... ok test.unit.common.test_swob.TestRequest.test_401_unauthorized test.unit.common.test_swob.TestRequest.test_401_unauthorized ... ok test.unit.common.test_swob.TestRequest.test_401_valid_account_path test.unit.common.test_swob.TestRequest.test_401_valid_account_path ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted ... ok test.unit.common.test_swob.TestRequest.test_accept_header test.unit.common.test_swob.TestRequest.test_accept_header ... ok test.unit.common.test_swob.TestRequest.test_allow_reserved_names test.unit.common.test_swob.TestRequest.test_allow_reserved_names ... ok test.unit.common.test_swob.TestRequest.test_as_referer test.unit.common.test_swob.TestRequest.test_as_referer ... ok test.unit.common.test_swob.TestRequest.test_bad_path_info_pop test.unit.common.test_swob.TestRequest.test_bad_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_bad_range test.unit.common.test_swob.TestRequest.test_bad_range ... ok test.unit.common.test_swob.TestRequest.test_blank test.unit.common.test_swob.TestRequest.test_blank ... ok test.unit.common.test_swob.TestRequest.test_blank_body_precedence test.unit.common.test_swob.TestRequest.test_blank_body_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_parsing test.unit.common.test_swob.TestRequest.test_blank_parsing ... ok test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args ... ok test.unit.common.test_swob.TestRequest.test_copy_get test.unit.common.test_swob.TestRequest.test_copy_get ... ok test.unit.common.test_swob.TestRequest.test_datetime_properties test.unit.common.test_swob.TestRequest.test_datetime_properties ... ok test.unit.common.test_swob.TestRequest.test_get_response test.unit.common.test_swob.TestRequest.test_get_response ... ok test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args ... ok test.unit.common.test_swob.TestRequest.test_message_length_just_content_length test.unit.common.test_swob.TestRequest.test_message_length_just_content_length ... ok test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding ... ok test.unit.common.test_swob.TestRequest.test_not_401 test.unit.common.test_swob.TestRequest.test_not_401 ... ok test.unit.common.test_swob.TestRequest.test_params test.unit.common.test_swob.TestRequest.test_params ... ok test.unit.common.test_swob.TestRequest.test_path test.unit.common.test_swob.TestRequest.test_path ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop test.unit.common.test_swob.TestRequest.test_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_last test.unit.common.test_swob.TestRequest.test_path_info_pop_last ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_none test.unit.common.test_swob.TestRequest.test_path_info_pop_none ... ok test.unit.common.test_swob.TestRequest.test_path_qs test.unit.common.test_swob.TestRequest.test_path_qs ... ok test.unit.common.test_swob.TestRequest.test_path_question_mark test.unit.common.test_swob.TestRequest.test_path_question_mark ... ok test.unit.common.test_swob.TestRequest.test_properties test.unit.common.test_swob.TestRequest.test_properties ... ok test.unit.common.test_swob.TestRequest.test_split_path test.unit.common.test_swob.TestRequest.test_split_path ... ok test.unit.common.test_swob.TestRequest.test_swift_entity_path test.unit.common.test_swob.TestRequest.test_swift_entity_path ... ok test.unit.common.test_swob.TestRequest.test_timestamp test.unit.common.test_swob.TestRequest.test_timestamp ... ok test.unit.common.test_swob.TestRequest.test_timestamp_invalid test.unit.common.test_swob.TestRequest.test_timestamp_invalid ... ok test.unit.common.test_swob.TestRequest.test_timestamp_missing test.unit.common.test_swob.TestRequest.test_timestamp_missing ... ok test.unit.common.test_swob.TestRequest.test_unicode_params test.unit.common.test_swob.TestRequest.test_unicode_params ... ok test.unit.common.test_swob.TestRequest.test_unicode_path test.unit.common.test_swob.TestRequest.test_unicode_path ... ok test.unit.common.test_swob.TestRequest.test_unicode_query test.unit.common.test_swob.TestRequest.test_unicode_query ... ok test.unit.common.test_swob.TestRequest.test_url test.unit.common.test_swob.TestRequest.test_url ... ok test.unit.common.test_swob.TestRequest.test_url2 test.unit.common.test_swob.TestRequest.test_url2 ... ok test.unit.common.test_swob.TestRequest.test_wsgify test.unit.common.test_swob.TestRequest.test_wsgify ... ok test.unit.common.test_swob.TestRequest.test_wsgify_method test.unit.common.test_swob.TestRequest.test_wsgify_method ... ok test.unit.common.test_swob.TestRequest.test_wsgify_raise test.unit.common.test_swob.TestRequest.test_wsgify_raise ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.10:1010/sdk (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server ERROR: ERROR Unhandled exception in request: ChunkReadTimeout (0.01s after 0.02s) (txn: tx02fcf79579f44b28a5a62-00665ca31a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: tx4557e9ccf4064f07987df-00665ca31a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9370f736966c498c9e34b-00665ca31e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165140Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347100.85456', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165143Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347103.26052', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165143Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347103.27649', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9c5f1936-847d-41ea-a2bf-0bc030434298-sda', '1717347110.75350', '1717347112', '1717347111', '', '1717347112', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('9c5f1936-847d-41ea-a2bf-0bc030434298-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10284 seconds (9.72379/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp3ou9o5xo/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd3652df7f2eb095ccc8eb7cd5e429ad6', '23f42671-8eb0-43b0-9cce-735933417a89-sda', '1717347110.91694', '1717347112', '1717347111', '', '1717347112', 1, 1) test DEBUG: /tmp/tmp3ou9o5xo/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd3652df7f2eb095ccc8eb7cd5e429ad6', '23f42671-8eb0-43b0-9cce-735933417a89-sda', '1717347110.91694', '1717347112', '1717347111', '', '1717347112', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('23f42671-8eb0-43b0-9cce-735933417a89-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.11065 seconds (9.03738/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp9h48btx7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c070c2d4-febb-4e6f-8529-94559dd10df9-sda', '1717347112.58157', '1717347115', '1717347114', '', '1717347115', 0, 0) test DEBUG: /tmp/tmp9h48btx7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c070c2d4-febb-4e6f-8529-94559dd10df9-sda', '1717347112.58157', '1717347115', '1717347114', '', '1717347115', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c070c2d4-febb-4e6f-8529-94559dd10df9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06760 seconds (14.79387/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmppxkjokph/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7ee0b55ebea57c2a1b6cd66fb3430855', '89473da9-00f3-46e9-b321-6168c4ce73c4-sda', '1717347112.71531', '1717347115', '1717347114', '', '1717347115', 1, 0) test DEBUG: /tmp/tmppxkjokph/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347117.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '89473da9-00f3-46e9-b321-6168c4ce73c4-sda') test DEBUG: /tmp/tmppxkjokph/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0108099s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7ee0b55ebea57c2a1b6cd66fb3430855', '89473da9-00f3-46e9-b321-6168c4ce73c4-sda', '1717347112.71531', '1717347115', '1717347114', '', '1717347115', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('89473da9-00f3-46e9-b321-6168c4ce73c4-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09835 seconds (10.16736/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpblo6zdy2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '34e7f682-576b-41e1-a4b8-d9c9632e0e65-sda', '1717347112.88794', '1717347115', '1717347114', '', '1717347115', 0, 0) test DEBUG: /tmp/tmpblo6zdy2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '34e7f682-576b-41e1-a4b8-d9c9632e0e65-sda', '1717347112.88794', '1717347115', '1717347114', '', '1717347115', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('34e7f682-576b-41e1-a4b8-d9c9632e0e65-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07334 seconds (13.63596/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkjxbn43o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '99053d185519858cbf809f13312400b9', '5060774a-013f-4107-a8c8-16423c83b8c3-sda', '1717347113.02167', '1717347116', '1717347115', '', '1717347116', 1, 1) test DEBUG: /tmp/tmpkjxbn43o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '99053d185519858cbf809f13312400b9', '5060774a-013f-4107-a8c8-16423c83b8c3-sda', '1717347113.02167', '1717347116', '1717347115', '', '1717347116', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5060774a-013f-4107-a8c8-16423c83b8c3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08131 seconds (12.29823/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpg3de7ad_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c1e266e4-b594-4cf6-aeef-f5093ec4f6b6-sda', '1717347113.17137', '1717347116', '1717347115', '', '1717347116', 0, 0) test DEBUG: /tmp/tmpg3de7ad_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c1e266e4-b594-4cf6-aeef-f5093ec4f6b6-sda', '1717347113.17137', '1717347116', '1717347115', '', '1717347116', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c1e266e4-b594-4cf6-aeef-f5093ec4f6b6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04972 seconds (20.11209/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create ... ok Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx31ff62a0006d4a2aae5f4-00665ca31e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Ignoring 404 from primary (txn: tx11ed59bc51a2497f9babc-00665ca328) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx2692e6d105b948139f86e-00665ca328) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txdf938ac565fc41669893e-00665ca328) proxy-server ERROR: ERROR 500 Trying to GET /a/c/o From Object Server 10.0.0.10:1010/sdk (txn: txdf938ac565fc41669893e-00665ca328) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from primary (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from primary (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from primary (txn: tx150acec170c247999607a-00665ca329) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165140Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:51:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347100.87282', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347104.63504', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347109.46646', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347109.48161', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp58wbuawg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '4f6123e8651b093ae49910a04efb4a86', 'ebfeb3dc-cb2c-4d78-9104-5034659cb78e-sda', '1717347116.59370', '1717347118', '1717347117', '', '1717347118', 1, 1) test DEBUG: /tmp/tmp58wbuawg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347119.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'ebfeb3dc-cb2c-4d78-9104-5034659cb78e-sda') test DEBUG: /tmp/tmp58wbuawg/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0225642s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '4f6123e8651b093ae49910a04efb4a86', 'ebfeb3dc-cb2c-4d78-9104-5034659cb78e-sda', '1717347116.59370', '1717347118', '1717347117', '', '1717347118', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ebfeb3dc-cb2c-4d78-9104-5034659cb78e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.68689 seconds (1.45584/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpz_3jpnpz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '37e0d8b7-3975-40d8-a579-6e2350be1ee5-sda', '1717347120.10410', '1717347123', '1717347122', '', '1717347123', 0, 0) test DEBUG: /tmp/tmpz_3jpnpz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '37e0d8b7-3975-40d8-a579-6e2350be1ee5-sda', '1717347120.10410', '1717347123', '1717347122', '', '1717347123', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('37e0d8b7-3975-40d8-a579-6e2350be1ee5-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07901 seconds (12.65611/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpdv234hy9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2b74c776a12f9ae4a9c9d3b2a360e7bc', '30ac229e-75ea-46e2-bddb-68bbdfb8527d-sda', '1717347120.28315', '1717347123', '1717347122', '', '1717347123', 1, 1) test DEBUG: /tmp/tmpdv234hy9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2b74c776a12f9ae4a9c9d3b2a360e7bc', '30ac229e-75ea-46e2-bddb-68bbdfb8527d-sda', '1717347120.28315', '1717347123', '1717347122', '', '1717347123', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('30ac229e-75ea-46e2-bddb-68bbdfb8527d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09728 seconds (10.27987/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_w221ua1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '80067d2f-988b-405f-a60c-313601048189-sda', '1717347120.44051', '1717347124', '1717347123', '', '1717347124', 0, 1) test DEBUG: /tmp/tmp_w221ua1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '80067d2f-988b-405f-a60c-313601048189-sda', '1717347120.44051', '1717347124', '1717347123', '', '1717347124', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('80067d2f-988b-405f-a60c-313601048189-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06662 seconds (15.01100/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpe5vfntya/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0946fca5947d6561670989bf01f1b3ee', '5e5ff0b1-b841-4db7-9106-2fb2f513d34b-sda', '1717347120.58927', '1717347124', '1717347123', '', '1717347124', 1, 1) test DEBUG: /tmp/tmpe5vfntya/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347125.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '5e5ff0b1-b841-4db7-9106-2fb2f513d34b-sda') test DEBUG: /tmp/tmpe5vfntya/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0139749s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0946fca5947d6561670989bf01f1b3ee', '5e5ff0b1-b841-4db7-9106-2fb2f513d34b-sda', '1717347120.58927', '1717347124', '1717347123', '', '1717347124', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5e5ff0b1-b841-4db7-9106-2fb2f513d34b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10879 seconds (9.19200/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpcmk1jwjq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '00f05cb3-356e-4f11-8c1c-f23a1b180ba1-sda', '1717347120.78395', '1717347124', '1717347123', '', '1717347124', 0, 1) test DEBUG: /tmp/tmpcmk1jwjq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '00f05cb3-356e-4f11-8c1c-f23a1b180ba1-sda', '1717347120.78395', '1717347124', '1717347123', '', '1717347124', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('00f05cb3-356e-4f11-8c1c-f23a1b180ba1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07464 seconds (13.39753/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:44 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347104.65900', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347110.50396', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:50 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347110.51779', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:full_control FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347117.51575', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347117.52984', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpnrn6wc8m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0946fca5947d6561670989bf01f1b3ee', 'd91bd3d9-9d61-44a8-a921-a337f268bb74-sda', '1717347120.93501', '1717347124', '1717347123', '', '1717347124', 1, 1) test DEBUG: /tmp/tmpnrn6wc8m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0946fca5947d6561670989bf01f1b3ee', 'd91bd3d9-9d61-44a8-a921-a337f268bb74-sda', '1717347120.93501', '1717347124', '1717347123', '', '1717347124', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d91bd3d9-9d61-44a8-a921-a337f268bb74-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10929 seconds (9.14959/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpvpi_u7f_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '345d984e-34cf-4b8a-9db7-15b2caa5932b-sda', '1717347123.80536', '1717347128', '1717347127', '', '1717347128', 0, 0) test DEBUG: /tmp/tmpvpi_u7f_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '345d984e-34cf-4b8a-9db7-15b2caa5932b-sda', '1717347123.80536', '1717347128', '1717347127', '', '1717347128', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('345d984e-34cf-4b8a-9db7-15b2caa5932b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.42568 seconds (2.34921/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp14652t_3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '35d4acc3927e04a17f707c04a3a2b1d6', 'e5e338b7-4f35-4b60-87be-ab5a9cc004ca-sda', '1717347129.00774', '1717347133', '1717347132', '', '1717347133', 1, 1) test DEBUG: /tmp/tmp14652t_3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347134.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'e5e338b7-4f35-4b60-87be-ab5a9cc004ca-sda') test DEBUG: /tmp/tmp14652t_3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0163801s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '35d4acc3927e04a17f707c04a3a2b1d6', 'e5e338b7-4f35-4b60-87be-ab5a9cc004ca-sda', '1717347129.00774', '1717347133', '1717347132', '', '1717347133', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e5e338b7-4f35-4b60-87be-ab5a9cc004ca-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12742 seconds (7.84787/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpw5gpu9xz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aa30e98b-4414-417c-9831-04403efbaf61-sda', '1717347129.26370', '1717347134', '1717347133', '', '1717347134', 0, 0) test DEBUG: /tmp/tmpw5gpu9xz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aa30e98b-4414-417c-9831-04403efbaf61-sda', '1717347129.26370', '1717347134', '1717347133', '', '1717347134', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('aa30e98b-4414-417c-9831-04403efbaf61-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12460 seconds (8.02560/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpx2zhx01m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7e86503d623e0e246c3eaa01f99d2649', 'acc38e38-f259-4c17-9f0a-7669ef7562f8-sda', '1717347129.48192', '1717347134', '1717347133', '', '1717347134', 1, 1) test DEBUG: /tmp/tmpx2zhx01m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7e86503d623e0e246c3eaa01f99d2649', 'acc38e38-f259-4c17-9f0a-7669ef7562f8-sda', '1717347129.48192', '1717347134', '1717347133', '', '1717347134', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('acc38e38-f259-4c17-9f0a-7669ef7562f8-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.20274 seconds (4.93242/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c9352481-94c7-458a-b835-32503c38ef16-sda', '1717347129.78105', '1717347129.00000', '0', '', '1717347129.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c9352481-94c7-458a-b835-32503c38ef16-sda', '1717347129.78105', '1717347130.00000', '0', '', '1717347129.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'd3cd7cd1949b6ae4283a28d6d75e41bc', '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda', '1717347129.85141', '1717347129.83905', '0', '', '1717347129.83905', 11, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up ... ok sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347295.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/30e/d9959fe1493d97f6237462e66cbb430e/d9959fe1493d97f6237462e66cbb430e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347297.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/9bb/ca598b985f273577056079bf12fe99bb/ca598b985f273577056079bf12fe99bb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347297.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/9bb/ca598b985f273577056079bf12fe99bb/ca598b985f273577056079bf12fe99bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347299.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/802/10afd13c8aac2320c852e8d79021e802/10afd13c8aac2320c852e8d79021e802.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347299.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/802/10afd13c8aac2320c852e8d79021e802/10afd13c8aac2320c852e8d79021e802.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347301.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8ed/a5b7351ed811d2291228d1f72db278ed/a5b7351ed811d2291228d1f72db278ed.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347301.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/8ed/a5b7351ed811d2291228d1f72db278ed/a5b7351ed811d2291228d1f72db278ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347303.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/013/18a90ce78f8f8ae2ae46e2f209045013/18a90ce78f8f8ae2ae46e2f209045013.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347303.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/013/18a90ce78f8f8ae2ae46e2f209045013/18a90ce78f8f8ae2ae46e2f209045013.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347305.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ce0/b5fcd84d41be6507e4ed50eb11ecbce0/b5fcd84d41be6507e4ed50eb11ecbce0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347305.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ce0/b5fcd84d41be6507e4ed50eb11ecbce0/b5fcd84d41be6507e4ed50eb11ecbce0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347307.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ddf/29ec0081cda1d2409fb3712cea939ddf/29ec0081cda1d2409fb3712cea939ddf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347307.00000, db: /tmp/tmpxj3x76h7/sda/containers/0/ddf/29ec0081cda1d2409fb3712cea939ddf/29ec0081cda1d2409fb3712cea939ddf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpzh42y6b6/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347101.00000, (0, 0) as of 1717347101.00000, active as of 1717347101.00000> to ShardRange<'k' to 't' as of 1717347102.00000, (0, 0) as of 1717347102.00000, sharded as of 1717347103.00000>, path: .shards_a/shard_c, db: /tmp/tmpzh42y6b6/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpzh42y6b6/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpe_gr3au5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347103.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/f8d/c2bb74575efbacf03799b31f74c83f8d/c2bb74575efbacf03799b31f74c83f8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347106.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/4e9/fae6c6257108f3dc87c192e2792394e9/fae6c6257108f3dc87c192e2792394e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347109.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/825/63bf95f274bba71ac69579c67f6ea825/63bf95f274bba71ac69579c67f6ea825.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347112.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/5eb/918fda536b8f5e37f0f0984cd6d9f5eb/918fda536b8f5e37f0f0984cd6d9f5eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347115.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/a63/2a7c1f65bd293d8ffaa8d4a4809b2a63/2a7c1f65bd293d8ffaa8d4a4809b2a63.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347118.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/111/0c6a1fbd7dc7cb7f315bd6743d592111/0c6a1fbd7dc7cb7f315bd6743d592111.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347121.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/fe6/e487b014f5ac96e42769378fd5752fe6/e487b014f5ac96e42769378fd5752fe6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347124.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/028/243a7e4c651c9a998b5bd7a4fa785028/243a7e4c651c9a998b5bd7a4fa785028.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347127.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/25d/bb1ebc90516bf77d260234640a1e025d/bb1ebc90516bf77d260234640a1e025d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347130.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/944/4699f694668984e38d09d0d30829c944/4699f694668984e38d09d0d30829c944.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347133.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/365/d81f853e7a5d2e82e9e9f02ecc33f365/d81f853e7a5d2e82e9e9f02ecc33f365.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347136.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/274/b65ff33cb20798b522e03f6dd7268274/b65ff33cb20798b522e03f6dd7268274.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347139.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/d7b/9aacaa5bc1170d16aaf92e9913295d7b/9aacaa5bc1170d16aaf92e9913295d7b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347142.00000-0, db: /tmp/tmpse62yb7e/sda/containers/0/db6/fb440c4c4b3d82b11fb0a50d1e515db6/fb440c4c4b3d82b11fb0a50d1e515db6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347103.00000-0, db: /tmp/tmpj3t4afd6/sda/containers/0/f8d/c2bb74575efbacf03799b31f74c83f8d/c2bb74575efbacf03799b31f74c83f8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347107.00000-0, db: /tmp/tmpj3t4afd6/sda/containers/0/2f3/3fa96a1e9a4566a5b5a6ac74f2ecf2f3/3fa96a1e9a4566a5b5a6ac74f2ecf2f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347108.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/881/0924b0c231e85a1a884ff3c149436881/0924b0c231e85a1a884ff3c149436881.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347112.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5eb/918fda536b8f5e37f0f0984cd6d9f5eb/918fda536b8f5e37f0f0984cd6d9f5eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347116.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/ba0/2a1f84ef79830f46517498ce51263ba0/2a1f84ef79830f46517498ce51263ba0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347120.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/321/cfdf803ea6b1ec38ad6492e458552321/cfdf803ea6b1ec38ad6492e458552321.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347120.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/321/cfdf803ea6b1ec38ad6492e458552321/cfdf803ea6b1ec38ad6492e458552321.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347124.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/028/243a7e4c651c9a998b5bd7a4fa785028/243a7e4c651c9a998b5bd7a4fa785028.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347128.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4da/feac3339f62e310808289ff3fff6e4da/feac3339f62e310808289ff3fff6e4da.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347132.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/0c6/c6cdd20e0f6be85f4b11531102a7a0c6/c6cdd20e0f6be85f4b11531102a7a0c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347136.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/274/b65ff33cb20798b522e03f6dd7268274/b65ff33cb20798b522e03f6dd7268274.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347140.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5e2/529595a7e786dacec64c76cbd7c715e2/529595a7e786dacec64c76cbd7c715e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347144.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/9f5/d63dbd9dd994944ce0e026d382d1e9f5/d63dbd9dd994944ce0e026d382d1e9f5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347148.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/eb9/6740f253099f2f4202929aca9eaa9eb9/6740f253099f2f4202929aca9eaa9eb9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347152.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/013/792a9cef604065d81c4f98975b3d6013/792a9cef604065d81c4f98975b3d6013.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347152.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/013/792a9cef604065d81c4f98975b3d6013/792a9cef604065d81c4f98975b3d6013.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347156.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/71f/d52a6a6ad8082a975da4513a0c3bc71f/d52a6a6ad8082a975da4513a0c3bc71f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347160.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/c30/6afefcb559d21e780fffa5567d9f2c30/6afefcb559d21e780fffa5567d9f2c30.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347164.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/345/b18496cc46d2034705bf6ecbedebf345/b18496cc46d2034705bf6ecbedebf345.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347168.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/9e7/6eeb000df831432e17b94c558081c9e7/6eeb000df831432e17b94c558081c9e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347172.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4f0/a897279f1a117184e57901c9444f54f0/a897279f1a117184e57901c9444f54f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347176.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/343/5e2d0efa8baa83468002dedd31db9343/5e2d0efa8baa83468002dedd31db9343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347180.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b9a/edb8119cd4d2630bc865f52f81544b9a/edb8119cd4d2630bc865f52f81544b9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347184.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e03/4a04f7b013c2437a9d912b67daa69e03/4a04f7b013c2437a9d912b67daa69e03.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347184.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e03/4a04f7b013c2437a9d912b67daa69e03/4a04f7b013c2437a9d912b67daa69e03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347188.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/43c/79bb61b3280e9d0fff484d614f94543c/79bb61b3280e9d0fff484d614f94543c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347192.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/6fa/158fd7385158515875860fc8a69966fa/158fd7385158515875860fc8a69966fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347196.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/af5/2722d14adf6006e454f1bd1f56ae9af5/2722d14adf6006e454f1bd1f56ae9af5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347200.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/839/ca32f0bad11a353bc320bc462f679839/ca32f0bad11a353bc320bc462f679839.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347204.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4d3/fc8f5bf8f660cb78da09caf6faac54d3/fc8f5bf8f660cb78da09caf6faac54d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347208.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/030/987dbb3f91948969290126a3a4e19030/987dbb3f91948969290126a3a4e19030.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347212.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/72a/d4851979b9f8fe82c57c8808a422572a/d4851979b9f8fe82c57c8808a422572a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347220.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/71e/6de3f330f9e8d84055274ee652dbf71e/6de3f330f9e8d84055274ee652dbf71e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347224.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a8d/0523f61807c65a5176e6ba2db5f11a8d/0523f61807c65a5176e6ba2db5f11a8d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347228.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/906/f58109f54c63f9a53b744430e1dba906/f58109f54c63f9a53b744430e1dba906.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347232.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/23b/d4be284624ac64b24c804dab5c62b23b/d4be284624ac64b24c804dab5c62b23b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347236.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a8f/ebfb13dc304bbd03a49c50d07f4e8a8f/ebfb13dc304bbd03a49c50d07f4e8a8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347240.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/d07/8462b1d830f66070f109037b39029d07/8462b1d830f66070f109037b39029d07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347244.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/896/3f022372527e997d8a5822f80d1bf896/3f022372527e997d8a5822f80d1bf896.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347248.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e93/285e35ebc4bcd6e9d873e195d3a2ae93/285e35ebc4bcd6e9d873e195d3a2ae93.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347248.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e93/285e35ebc4bcd6e9d873e195d3a2ae93/285e35ebc4bcd6e9d873e195d3a2ae93.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347252.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/bd3/b6280feb989c820da91430dd7f1d0bd3/b6280feb989c820da91430dd7f1d0bd3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347256.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/8a1/99e66f00d4f64cf385e7c517ec9a28a1/99e66f00d4f64cf385e7c517ec9a28a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347260.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/1f7/bb0f04c926f9e652ed0f14e8bdfcb1f7/bb0f04c926f9e652ed0f14e8bdfcb1f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347264.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/ec7/bac4bc06c3b9745b2e448f664b31cec7/bac4bc06c3b9745b2e448f664b31cec7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347268.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a31/63588ba1a47a5d32808e745e2f5f1a31/63588ba1a47a5d32808e745e2f5f1a31.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347272.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/dd2/122f305573fc87ab221fe67ef953cdd2/122f305573fc87ab221fe67ef953cdd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347280.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e9f/ade24454704b1adac4bcd7f8f7c35e9f/ade24454704b1adac4bcd7f8f7c35e9f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347280.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e9f/ade24454704b1adac4bcd7f8f7c35e9f/ade24454704b1adac4bcd7f8f7c35e9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347284.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/97e/9e8bcf6f868aa05f28fa8d49d87fc97e/9e8bcf6f868aa05f28fa8d49d87fc97e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347288.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/370/22fb67f5943b6195582d689c6d5b9370/22fb67f5943b6195582d689c6d5b9370.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347292.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/ac4/7468d55d914d6a24804619a20fa04ac4/7468d55d914d6a24804619a20fa04ac4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347296.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/ede/fb883c1b75b874242cbae9762c9daede/fb883c1b75b874242cbae9762c9daede.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347300.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/621/eb0ffcb557d2dc1350343916fadc4621/eb0ffcb557d2dc1350343916fadc4621.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347304.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b43/fa0a6b08aaa815cbe399709bc72c2b43/fa0a6b08aaa815cbe399709bc72c2b43.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347308.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5af/88066f2f868d4658f8d629d2007a15af/88066f2f868d4658f8d629d2007a15af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347312.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/f74/89b43ec01c98425c5d156d89116cdf74/89b43ec01c98425c5d156d89116cdf74.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347312.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/f74/89b43ec01c98425c5d156d89116cdf74/89b43ec01c98425c5d156d89116cdf74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347316.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/d98/ab6feeba2011355fccdf84c60aa17d98/ab6feeba2011355fccdf84c60aa17d98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347320.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b1d/fd64a339b288c727dff01fbd2315cb1d/fd64a339b288c727dff01fbd2315cb1d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347324.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/01c/979f99f4861d3c8354b062b9cc55b01c/979f99f4861d3c8354b062b9cc55b01c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347328.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/3b7/fe857db5b133c0e8119e4682d20653b7/fe857db5b133c0e8119e4682d20653b7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347332.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e5e/dc425f49306f325db4bd22f093a62e5e/dc425f49306f325db4bd22f093a62e5e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347336.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/6a9/cb56574485967632113a25de82a576a9/cb56574485967632113a25de82a576a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347340.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/cc1/3e2b067e92c4411a4a17a5b85b65ccc1/3e2b067e92c4411a4a17a5b85b65ccc1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347344.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4c2/55404ff55d82c8adfc804c7c0f1d34c2/55404ff55d82c8adfc804c7c0f1d34c2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347344.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4c2/55404ff55d82c8adfc804c7c0f1d34c2/55404ff55d82c8adfc804c7c0f1d34c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347348.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/148/739195e783e4f0c9a637cd182665d148/739195e783e4f0c9a637cd182665d148.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347352.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e08/0bf38a155b7c5a2144ba3594ac1d6e08/0bf38a155b7c5a2144ba3594ac1d6e08.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347356.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/fd9/5e75a8aabafea6a2b2a2950786459fd9/5e75a8aabafea6a2b2a2950786459fd9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347360.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/7b4/d1b6fc527793d618e9f51e76fa8797b4/d1b6fc527793d618e9f51e76fa8797b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347364.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/c7d/6d49104b86901158743b16b6c7f40c7d/6d49104b86901158743b16b6c7f40c7d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347368.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/50d/de5294262a2b29cab5977a828c41950d/de5294262a2b29cab5977a828c41950d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347372.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/58f/a37c2e0d7a9f26fdc96d58e28422b58f/a37c2e0d7a9f26fdc96d58e28422b58f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347376.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/11a/1fd1ed0006e3b44abafa1ab44bc7011a/1fd1ed0006e3b44abafa1ab44bc7011a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347380.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/1af/d9e7cd285264117f161f347374dde1af/d9e7cd285264117f161f347374dde1af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347384.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/263/c6c0178de65bc2a438034b0147990263/c6c0178de65bc2a438034b0147990263.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347388.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/8a6/6b5449040101d43445fc59d5fa1918a6/6b5449040101d43445fc59d5fa1918a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347392.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/204/3b482036f0fec6005957eaef4793f204/3b482036f0fec6005957eaef4793f204.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347396.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/57c/65b8a04cfc9b201d45fbeba64040e57c/65b8a04cfc9b201d45fbeba64040e57c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347400.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/0f7/aaf78a8b6d9e4b05d1c0dbf319c170f7/aaf78a8b6d9e4b05d1c0dbf319c170f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347404.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/60d/2f3a90522ec8b9a434a3b2002dabc60d/2f3a90522ec8b9a434a3b2002dabc60d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347408.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/9b5/505b1538f7ba78461db9e228f56f79b5/505b1538f7ba78461db9e228f56f79b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347412.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b96/f20f3c68f9bce6952bdb8f879eac4b96/f20f3c68f9bce6952bdb8f879eac4b96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347416.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5c8/49cdec80e148f9846e74797f86acf5c8/49cdec80e148f9846e74797f86acf5c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347420.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/3c0/fadc2da3334ddd65eaac1ec0ade323c0/fadc2da3334ddd65eaac1ec0ade323c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347424.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/7b9/6004ab0cbf36bccf665980f0b86ee7b9/6004ab0cbf36bccf665980f0b86ee7b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347428.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/aa5/6b2466472aef47d6fdb45db4d1eefaa5/6b2466472aef47d6fdb45db4d1eefaa5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347432.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/769/020cd6fb1f6144ba61bb4091e7110769/020cd6fb1f6144ba61bb4091e7110769.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347436.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/508/6f7bc76dab03d7cfd8769c0a0a9cc508/6f7bc76dab03d7cfd8769c0a0a9cc508.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347440.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5c2/6a8390ad17934708462b5c2180b7c5c2/6a8390ad17934708462b5c2180b7c5c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347444.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/d94/10035c45a87bbb3271c83ec4d0a03d94/10035c45a87bbb3271c83ec4d0a03d94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347448.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/450/b6b9265c14beb103102286b859a55450/b6b9265c14beb103102286b859a55450.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347452.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/6ed/05392282e35c976f43b7a0d3a31516ed/05392282e35c976f43b7a0d3a31516ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347456.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/df2/93c5c28d0484465f57318ba1e6c2adf2/93c5c28d0484465f57318ba1e6c2adf2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347460.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/ea3/fd75aa72ca18e32d6b8d1fbcd4158ea3/fd75aa72ca18e32d6b8d1fbcd4158ea3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347464.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/351/c8636461c6ab6941c5ef4df61bac7351/c8636461c6ab6941c5ef4df61bac7351.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347468.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5a8/b114e5157990d179b1aa30894e39a5a8/b114e5157990d179b1aa30894e39a5a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347472.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/cd4/cbe3b96962677d7e5e88aaa0b3933cd4/cbe3b96962677d7e5e88aaa0b3933cd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347476.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/96b/02634527f89f260bcea4612470ee396b/02634527f89f260bcea4612470ee396b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347480.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/27f/740f69bdc9854043a20e92f23f94d27f/740f69bdc9854043a20e92f23f94d27f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347484.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5d8/1e82f0c91d72019072b16e493753d5d8/1e82f0c91d72019072b16e493753d5d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347488.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/5fd/2a038280e5cd7faef45348022ef7d5fd/2a038280e5cd7faef45348022ef7d5fd.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile ... ok test.unit.common.test_daemon.TestDaemon.test_create test.unit.common.test_daemon.TestDaemon.test_create ... ok test.unit.common.test_daemon.TestDaemon.test_stubs test.unit.common.test_daemon.TestDaemon.test_stubs ... ok test.unit.common.test_daemon.TestRunDaemon.test_fork_workers test.unit.common.test_daemon.TestRunDaemon.test_fork_workers ... ok test.unit.common.test_daemon.TestRunDaemon.test_forked_worker test.unit.common.test_daemon.TestRunDaemon.test_forked_worker ... ok test.unit.common.test_daemon.TestRunDaemon.test_restart_workers test.unit.common.test_daemon.TestRunDaemon.test_restart_workers ... ok test.unit.common.test_daemon.TestRunDaemon.test_run test.unit.common.test_daemon.TestRunDaemon.test_run ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon test.unit.common.test_daemon.TestRunDaemon.test_run_daemon ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file ... ok test.unit.common.test_daemon.TestRunDaemon.test_signal test.unit.common.test_daemon.TestRunDaemon.test_signal ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_init test.unit.common.test_internal_client.TestCompressingfileReader.test_init ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_read test.unit.common.test_internal_client.TestCompressingfileReader.test_read ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek test.unit.common.test_internal_client.TestCompressingfileReader.test_seek ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'AUTH_a.example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86bd00>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o_0', 'created_at': '1717347129.87411', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o_1', 'created_at': '1717347129.87614', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o_2', 'created_at': '1717347129.87641', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o_3', 'created_at': '1717347129.87667', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o_4', 'created_at': '1717347129.87690', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o_5', 'created_at': '1717347129.87713', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o_6', 'created_at': '1717347129.87736', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o_7', 'created_at': '1717347129.87759', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o_8', 'created_at': '1717347129.87782', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o_9', 'created_at': '1717347129.87804', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing_0', 'created_at': '1717347129.87833', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 12, 'name': '/a/c/o_missing_1', 'created_at': '1717347129.93576', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 13, 'name': '/a/c/o_missing_2', 'created_at': '1717347129.95660', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 14, 'name': '/a/c/o_missing_3', 'created_at': '1717347130.01180', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 15, 'name': '/a/c/o_missing_4', 'created_at': '1717347130.03980', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 16, 'name': '/a/c/o_missing_5', 'created_at': '1717347130.06405', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 17, 'name': '/a/c/o_missing_6', 'created_at': '1717347130.09584', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 18, 'name': '/a/c/o_missing_7', 'created_at': '1717347130.12266', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 19, 'name': '/a/c/o_missing_8', 'created_at': '1717347130.14721', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 20, 'name': '/a/c/o_missing_9', 'created_at': '1717347130.19628', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (21, 'bec086b18de7b43cf26ccda43b630495', '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda', '1717347129.85141', '1717347129.83905', '0', '', '1717347129.83905', 21, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': '/a/c/o_missing_10', 'created_at': '1717347130.21917', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '180caa88-4da3-42cb-b0b2-5a68bc77b651-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpggyrc_kx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '56a0693c-95f5-43b0-b9bf-d0b8dc6a2493-sda', '1717347130.27647', '1717347131', '0', '', '1717347131', 0, 0) test DEBUG: /tmp/tmpggyrc_kx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '56a0693c-95f5-43b0-b9bf-d0b8dc6a2493-sda', '1717347130.27647', '1717347131', '0', '', '1717347131', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('56a0693c-95f5-43b0-b9bf-d0b8dc6a2493-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06611 seconds (15.12614/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkud3izvp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9cc40cd8576ce797fe1a87669f37d28a', '7ac9d562-c311-41ec-a9d7-131cbd62ee62-sda', '1717347130.39152', '1717347131', '0', '', '1717347131', 1, 0) test DEBUG: /tmp/tmpkud3izvp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347132.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '7ac9d562-c311-41ec-a9d7-131cbd62ee62-sda') test DEBUG: /tmp/tmpkud3izvp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0183203s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9cc40cd8576ce797fe1a87669f37d28a', '7ac9d562-c311-41ec-a9d7-131cbd62ee62-sda', '1717347130.39152', '1717347131', '0', '', '1717347131', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7ac9d562-c311-41ec-a9d7-131cbd62ee62-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpkud3izvp/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15430 seconds (6.48100/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ Sun Jun 2 04:52:14 2024 /tmp/log/swift/profile/default.profile3610486 Sun Jun 2 04:52:14 2024 /tmp/log/swift/profile/default.profile123 Sun Jun 2 04:52:14 2024 /tmp/log/swift/profile/default.profile456 45 function calls in 0.003 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 9 0.002 0.000 0.003 0.000 profile:0(import os;os.getcwd();) 9 0.001 0.000 0.001 0.000 :0(exec) 9 0.000 0.000 0.000 0.000 :1() 9 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 9 0.000 0.000 0.000 0.000 :0(setprofile) my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Worker mock-pid-0 exited my-daemon DEBUG: Worker mock-pid-1 exited my-daemon DEBUG: Worker mock-pid-2 exited my-daemon NOTICE: Finished 3610486 my-daemon DEBUG: Forked worker 3610486 finished my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Daemon needs to change options, aborting workers my-daemon DEBUG: Cleaned up worker mock-pid-0 my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-3 with {} my-daemon DEBUG: Spawned worker mock-pid-4 with {} my-daemon DEBUG: Spawned worker mock-pid-5 with {} my-daemon DEBUG: Cleaned up worker mock-pid-3 my-daemon DEBUG: Cleaned up worker mock-pid-4 my-daemon DEBUG: Cleaned up worker mock-pid-5 my-daemon NOTICE: SIGTERM received () my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Worker mock-pid exited my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon NOTICE: Worker mock-pid died my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10048794746398926 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.1024179458618164 test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_connection_pooling test.unit.common.test_memcached.TestMemcached.test_connection_pooling ... ok test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect ... ok test.unit.common.test_memcached.TestMemcached.test_decr test.unit.common.test_memcached.TestMemcached.test_decr ... ok test.unit.common.test_memcached.TestMemcached.test_delete test.unit.common.test_memcached.TestMemcached.test_delete ... ok test.unit.common.test_memcached.TestMemcached.test_error_disabled test.unit.common.test_memcached.TestMemcached.test_error_disabled ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86bd00>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/dontchange', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.aexample.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1dc69240>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1dc69240>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/ä½\xa0好', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/FOO_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/PREFIX_uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86bd00>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com:8080', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage1.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params ... ok test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir ... ok test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration ... ok test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_chexor test.unit.container.test_backend.TestContainerBroker.test_chexor ... ok test.unit.container.test_backend.TestContainerBroker.test_create_broker test.unit.container.test_backend.TestContainerBroker.test_create_broker ... ok test.unit.container.test_backend.TestContainerBroker.test_creation test.unit.container.test_backend.TestContainerBroker.test_creation ... ok test.unit.container.test_backend.TestContainerBroker.test_db_file test.unit.container.test_backend.TestContainerBroker.test_db_file ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_object test.unit.container.test_backend.TestContainerBroker.test_delete_object ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBroker.test_empty test.unit.container.test_backend.TestContainerBroker.test_empty ... ok test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_enable_sharding test.unit.container.test_backend.TestContainerBroker.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBroker.test_exception test.unit.container.test_backend.TestContainerBroker.test_exception ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_brokers test.unit.container.test_backend.TestContainerBroker.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBroker.test_get_db_state test.unit.container.test_backend.TestContainerBroker.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info test.unit.container.test_backend.TestContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBroker.test_get_items_since test.unit.container.test_backend.TestContainerBroker.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces test.unit.container.test_backend.TestContainerBroker.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects test.unit.container.test_backend.TestContainerBroker.test_get_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBroker.test_get_policy_info test.unit.container.test_backend.TestContainerBroker.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_replication_info test.unit.container.test_backend.TestContainerBroker.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_report_info test.unit.container.test_backend.TestContainerBroker.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBroker.test_is_deleted test.unit.container.test_backend.TestContainerBroker.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_last_modified_time test.unit.container.test_backend.TestContainerBroker.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items test.unit.container.test_backend.TestContainerBroker.test_merge_items ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBroker.test_newid test.unit.container.test_backend.TestContainerBroker.test_newid ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest ... ok test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches ... ok test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.container.test_backend.TestContainerBroker.test_path test.unit.container.test_backend.TestContainerBroker.test_path ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download ... ok test.unit.container.test_backend.TestContainerBroker.test_pending_file_name test.unit.container.test_backend.TestContainerBroker.test_pending_file_name ... ok test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval ... ok test.unit.common.test_container_sync_realms.TestUtils.test_empty test.unit.common.test_container_sync_realms.TestUtils.test_empty ... ok test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm ... ok test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing ... ok test.unit.common.test_container_sync_realms.TestUtils.test_get_sig test.unit.common.test_container_sync_realms.TestUtils.test_get_sig ... ok test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object ... ok test.unit.common.test_container_sync_realms.TestUtils.test_one_realm test.unit.common.test_container_sync_realms.TestUtils.test_one_realm ... ok test.unit.common.test_container_sync_realms.TestUtils.test_os_error test.unit.common.test_container_sync_realms.TestUtils.test_os_error ... ok test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default ... ok test.unit.common.test_db.TestHelperFunctions.test_zero_like test.unit.common.test_db.TestHelperFunctions.test_zero_like ... ok test.unit.common.test_swob.TestMatch.test_match test.unit.common.test_swob.TestMatch.test_match ... ok test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_noquote test.unit.common.test_swob.TestMatch.test_match_noquote ... ok test.unit.common.test_swob.TestMatch.test_match_star test.unit.common.test_swob.TestMatch.test_match_star ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value ... ok test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:51:59 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_chrome test.unit.common.middleware.test_formpost.TestFormPost.test_chrome ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_explorer test.unit.common.middleware.test_formpost.TestFormPost.test_explorer ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_firefox test.unit.common.middleware.test_formpost.TestFormPost.test_firefox ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero ... ok test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container ... ok test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers ... ok test.unit.common.test_utils.TestShardRangeList.test_filter test.unit.common.test_utils.TestShardRangeList.test_filter ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start ... ok test.unit.common.test_utils.TestShardRangeList.test_find_lower test.unit.common.test_utils.TestShardRangeList.test_find_lower ... ok test.unit.common.test_utils.TestShardRangeList.test_includes test.unit.common.test_utils.TestShardRangeList.test_includes ... ok test.unit.common.test_utils.TestShardRangeList.test_init test.unit.common.test_utils.TestShardRangeList.test_init ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_container test.unit.common.middleware.test_formpost.TestFormPost.test_no_container ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_redirect ... ok test.unit.common.test_utils.TestShardRangeList.test_init_with_list test.unit.common.test_utils.TestShardRangeList.test_init_with_list ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_safari test.unit.common.middleware.test_formpost.TestFormPost.test_safari ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 ... ok test.unit.common.test_utils.TestShardRangeList.test_pop test.unit.common.test_utils.TestShardRangeList.test_pop ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after ... ok test.unit.common.test_utils.TestShardRangeList.test_slice test.unit.common.test_utils.TestShardRangeList.test_slice ... ok test.unit.common.test_utils.TestShardRangeList.test_states test.unit.common.test_utils.TestShardRangeList.test_states ... ok test.unit.common.test_utils.TestShardRangeList.test_timestamps test.unit.common.test_utils.TestShardRangeList.test_timestamps ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int ... ok test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names ... ok test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call ... ok test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates test.unit.common.test_utils.TestStatsdLogging.test_sample_rates ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 ... ok test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation ... ok test.unit.common.test_utils.TestStatsdLogging.test_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_timing_stats ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_delta test.unit.common.utils.test_timestamp.TestTimestamp.test_delta ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_false test.unit.common.utils.test_timestamp.TestTimestamp.test_false ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_float test.unit.common.utils.test_timestamp.TestTimestamp.test_float ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_int test.unit.common.utils.test_timestamp.TestTimestamp.test_int ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object test.unit.container.test_backend.TestContainerBroker.test_put_object ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_raw test.unit.common.utils.test_timestamp.TestTimestamp.test_raw ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_true test.unit.common.utils.test_timestamp.TestTimestamp.test_true ... ok test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing ... ok test.unit.container.test_sharder.TestCleavingContext.test_cursor test.unit.container.test_sharder.TestCleavingContext.test_cursor ... ok test.unit.container.test_sharder.TestCleavingContext.test_delete test.unit.container.test_sharder.TestCleavingContext.test_delete ... ok test.unit.container.test_sharder.TestCleavingContext.test_done test.unit.container.test_sharder.TestCleavingContext.test_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_init test.unit.container.test_sharder.TestCleavingContext.test_init ... ok test.unit.container.test_sharder.TestCleavingContext.test_iter test.unit.container.test_sharder.TestCleavingContext.test_iter ... ok test.unit.container.test_sharder.TestCleavingContext.test_load test.unit.container.test_sharder.TestCleavingContext.test_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_all test.unit.container.test_sharder.TestCleavingContext.test_load_all ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_range_done test.unit.container.test_sharder.TestCleavingContext.test_range_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_reset test.unit.container.test_sharder.TestCleavingContext.test_reset ... ok test.unit.container.test_sharder.TestCleavingContext.test_start test.unit.container.test_sharder.TestCleavingContext.test_start ... ok test.unit.container.test_sharder.TestCleavingContext.test_store test.unit.container.test_sharder.TestCleavingContext.test_store ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestSweepStats.test_copy test.unit.obj.test_updater.TestSweepStats.test_copy ... ok test.unit.obj.test_updater.TestSweepStats.test_reset test.unit.obj.test_updater.TestSweepStats.test_reset ... ok test.unit.obj.test_updater.TestSweepStats.test_since test.unit.obj.test_updater.TestSweepStats.test_since ... ok test.unit.obj.test_updater.TestSweepStats.test_str test.unit.obj.test_updater.TestSweepStats.test_str ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim test.unit.container.test_backend.TestContainerBroker.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim ... ok test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.1026451587677002 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 4.219423055648804 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 4.219799041748047 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 4.2200329303741455 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 4.220252990722656 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 4.22045636177063 test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0002672672271729 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0007915496826172 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0010521411895752 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0013000965118408 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0016324520111084 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0019338130950928 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.002176284790039 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0024123191833496 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout connecting to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10303115844726562 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 4.673004150390625e-05: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test.unit.common.test_memcached.TestMemcached.test_error_limiting test.unit.common.test_memcached.TestMemcached.test_error_limiting ... ok test DEBUG: Replicating db /tmp/tmpkud3izvp/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c34ce7a9d5f2ed4e2483d6d62090b5e1', '53971d72-07da-403e-ad3a-d07267b1bc7a-sda', '1717347130.52923', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('53971d72-07da-403e-ad3a-d07267b1bc7a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c34ce7a9d5f2ed4e2483d6d62090b5e1', '53971d72-07da-403e-ad3a-d07267b1bc7a-sda', '1717347130.52923', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('53971d72-07da-403e-ad3a-d07267b1bc7a-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpse_63yu1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f29ed67c-3c85-40d0-95b8-0e50fc4926a9-sda', '1717347130.67738', '1717347131', '0', '', '1717347131', 0, 0) test DEBUG: /tmp/tmpse_63yu1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f29ed67c-3c85-40d0-95b8-0e50fc4926a9-sda', '1717347130.67738', '1717347131', '0', '', '1717347131', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f29ed67c-3c85-40d0-95b8-0e50fc4926a9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10317 seconds (9.69283/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6da3m1iz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9cc40cd8576ce797fe1a87669f37d28a', '2351a8f2-981b-4e45-87ee-8f03051e3694-sda', '1717347130.82931', '1717347131', '0', '', '1717347131', 1, 0) test DEBUG: /tmp/tmp6da3m1iz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9cc40cd8576ce797fe1a87669f37d28a', '2351a8f2-981b-4e45-87ee-8f03051e3694-sda', '1717347130.82931', '1717347131', '0', '', '1717347131', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2351a8f2-981b-4e45-87ee-8f03051e3694-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp6da3m1iz/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15673 seconds (6.38049/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp6da3m1iz/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c34ce7a9d5f2ed4e2483d6d62090b5e1', '392db75e-0376-4975-b9c1-c647aa88dc77-sda', '1717347130.96650', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('392db75e-0376-4975-b9c1-c647aa88dc77-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c34ce7a9d5f2ed4e2483d6d62090b5e1', '392db75e-0376-4975-b9c1-c647aa88dc77-sda', '1717347130.96650', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('392db75e-0376-4975-b9c1-c647aa88dc77-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp36_359d3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '635d7c97-19e2-4baf-a98f-4e2ebf5aa21a-sda', '1717347131.12176', '1717347132', '1717347133', '', '1717347133', 0, 1) test DEBUG: /tmp/tmp36_359d3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '635d7c97-19e2-4baf-a98f-4e2ebf5aa21a-sda', '1717347131.12176', '1717347132', '1717347133', '', '1717347133', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('635d7c97-19e2-4baf-a98f-4e2ebf5aa21a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08011 seconds (12.48255/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpw3kkl9mq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '35d4acc3927e04a17f707c04a3a2b1d6', '4a1dc7a4-2d46-4a9c-bb53-128ec19e8a46-sda', '1717347131.25717', '1717347132', '1717347133', '', '1717347133', 1, 1) test DEBUG: /tmp/tmpw3kkl9mq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347134.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '4a1dc7a4-2d46-4a9c-bb53-128ec19e8a46-sda') test DEBUG: /tmp/tmpw3kkl9mq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.016135s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '35d4acc3927e04a17f707c04a3a2b1d6', '4a1dc7a4-2d46-4a9c-bb53-128ec19e8a46-sda', '1717347131.25717', '1717347132', '1717347133', '', '1717347133', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('4a1dc7a4-2d46-4a9c-bb53-128ec19e8a46-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpw3kkl9mq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.20300 seconds (4.92622/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpw3kkl9mq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '44199313298dc7af72d0a1f50b46fb92', 'a2c43c67-12cd-4aef-b5c5-a0d291d6c243-sda', '1717347131.42824', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('a2c43c67-12cd-4aef-b5c5-a0d291d6c243-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx150acec170c247999607a-00665ca329) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.7:1007/sdh, error: ERROR Insufficient Storage (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.3:1003/sdd, error: ERROR Insufficient Storage (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.4:1004/sde, error: ERROR Insufficient Storage (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.6:1006/sdg, error: ERROR Insufficient Storage (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server ERROR: Object returning 503 for [507, 507, 507, 507, 507] (txn: tx7599e5dc45ae4363bde2d-00665ca32d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx513917eb853d4337974f6-00665ca32d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ee4a9d596c84d12957be-00665ca32d) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20fc40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20fc40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20fc40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20ef80>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20ed40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20fdc0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f06fd20fdc0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test ERROR: Error in '/tmp/tmpqugqdtkl/container-sync-realms.conf' with mtime_check_interval: could not convert string to float: 'invalid' test ERROR: Could not load '/tmp/tmpqa0uasyj/container-sync-realms.conf': File contains no section headers. file: '/tmp/tmpqa0uasyj/container-sync-realms.conf', line: 1 'invalid' test DEBUG: Could not load 'f199c6d76b5342618416d7c465a804a4': [Errno 2] No such file or directory: 'f199c6d76b5342618416d7c465a804a4' test ERROR: Could not load '/tmp/tmpv6qdoooi/container-sync-realms.conf': [Errno 13] Permission denied: '/tmp/tmpv6qdoooi/container-sync-realms.conf' test WARNING: Error sending UDP message to ('some.host.com', 8125): [Errno 1] test errno 1 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpg79k9ioy/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpg79k9ioy/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpg79k9ioy/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpg79k9ioy/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:29 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0173 "-" 3610583 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:29 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0202 "-" 3610583 - test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:52:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test DEBUG: tenant mismatch: BLAN_a != tenant_id test DEBUG: allow user with role(s) admin,swiftoperator as account admin test DEBUG: allow user with role(s) admin,swiftoperator as account admin test DEBUG: allowing proxy 127.0.0.1 for container-sync test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: Names allowed in acls. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL: allowme authorizing test DEBUG: Names allowed in acls. test DEBUG: user *:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:* allowed in ACL authorizing. test DEBUG: User tenant_name:user_name is not allowed to delete its own account test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Using identity: {'user': ('user_id', 'user_name'), 'tenant': ('test', 'tenant_name'), 'roles': ['reselleradmin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: User test has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head ... ok test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning ... ok test.unit.common.test_constraints.TestConstraints.test_check_account_format test.unit.common.test_constraints.TestConstraints.test_check_account_format ... ok test.unit.common.test_constraints.TestConstraints.test_check_container_format test.unit.common.test_constraints.TestConstraints.test_check_container_format ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount ... ok test.unit.common.test_constraints.TestConstraints.test_check_float test.unit.common.test_constraints.TestConstraints.test_check_float ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_count test.unit.common.test_constraints.TestConstraints.test_check_metadata_count ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_good test.unit.common.test_constraints.TestConstraints.test_check_metadata_good ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_size test.unit.common.test_constraints.TestConstraints.test_check_metadata_size ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8 test.unit.common.test_constraints.TestConstraints.test_check_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical ... ok test.unit.common.test_constraints.TestConstraints.test_valid_api_version test.unit.common.test_constraints.TestConstraints.test_valid_api_version ... ok test.unit.common.test_constraints.TestConstraints.test_valid_timestamp test.unit.common.test_constraints.TestConstraints.test_valid_timestamp ... ok test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta ... ok test.unit.common.test_constraints.TestConstraints.test_validate_constraints test.unit.common.test_constraints.TestConstraints.test_validate_constraints ... ok test.unit.common.test_db.TestGetDBConnection.test_invalid_path test.unit.common.test_db.TestGetDBConnection.test_invalid_path ... ok test.unit.common.test_db.TestGetDBConnection.test_locked_db test.unit.common.test_db.TestGetDBConnection.test_locked_db ... ok test.unit.common.test_db.TestGetDBConnection.test_normal_case test.unit.common.test_db.TestGetDBConnection.test_normal_case ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_creation test.unit.common.test_db_replicator.TestDBReplicator.test_creation ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs ... ok test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.4:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method get: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config ... ok test.unit.common.test_memcached.TestMemcached.test_error_raising test.unit.common.test_memcached.TestMemcached.test_error_raising ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns test.unit.common.test_memcached.TestMemcached.test_get_conns ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname ... ok test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr test.unit.common.test_memcached.TestMemcached.test_incr ... ok test.unit.common.test_memcached.TestMemcached.test_incr_add_expires test.unit.common.test_memcached.TestMemcached.test_incr_add_expires ... ok test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response ... ok test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold ... ok test.unit.common.test_memcached.TestMemcached.test_logger_kwarg test.unit.common.test_memcached.TestMemcached.test_logger_kwarg ... ok test.unit.common.test_memcached.TestMemcached.test_multi test.unit.common.test_memcached.TestMemcached.test_multi ... ok test.unit.common.test_memcached.TestMemcached.test_multi_delete test.unit.common.test_memcached.TestMemcached.test_multi_delete ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout ... ok REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '44199313298dc7af72d0a1f50b46fb92', 'a2c43c67-12cd-4aef-b5c5-a0d291d6c243-sda', '1717347131.42824', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('a2c43c67-12cd-4aef-b5c5-a0d291d6c243-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp0g4wnn05/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '94b10615-b492-45de-8ca4-64d14933788b-sda', '1717347132.03961', '1717347133', '1717347134', '', '1717347134', 0, 1) test DEBUG: /tmp/tmp0g4wnn05/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '94b10615-b492-45de-8ca4-64d14933788b-sda', '1717347132.03961', '1717347133', '1717347134', '', '1717347134', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('94b10615-b492-45de-8ca4-64d14933788b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10202 seconds (9.80174/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpr4gh0vfr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7e86503d623e0e246c3eaa01f99d2649', '7929a414-b863-462d-ac73-a6c4578b0f5e-sda', '1717347132.21551', '1717347133', '1717347134', '', '1717347134', 1, 0) test DEBUG: /tmp/tmpr4gh0vfr/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7e86503d623e0e246c3eaa01f99d2649', '7929a414-b863-462d-ac73-a6c4578b0f5e-sda', '1717347132.21551', '1717347133', '1717347134', '', '1717347134', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7929a414-b863-462d-ac73-a6c4578b0f5e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpr4gh0vfr/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13678 seconds (7.31093/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpr4gh0vfr/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '9a7800c2494644f8a9f71b191309a663', '2875d3a5-1ac6-4ca2-8f26-e4f54d38c387-sda', '1717347132.36076', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('2875d3a5-1ac6-4ca2-8f26-e4f54d38c387-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '9a7800c2494644f8a9f71b191309a663', '2875d3a5-1ac6-4ca2-8f26-e4f54d38c387-sda', '1717347132.36076', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('2875d3a5-1ac6-4ca2-8f26-e4f54d38c387-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpypxvyrq9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '229070a6-227e-4908-b9f4-3c09cd405549-sda', '1717347134.66137', '1717347134', '1717347135', '', '1717347135', 0, 0) test DEBUG: /tmp/tmpypxvyrq9/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '229070a6-227e-4908-b9f4-3c09cd405549-sda', '1717347134.66137', '1717347134', '1717347135', '', '1717347135', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('229070a6-227e-4908-b9f4-3c09cd405549-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05804 seconds (17.22999/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgp98sxet/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c7726de2-2d78-48fb-ba0f-79ee2e6de6c6-sda', '1717347134.76087', '1717347134', '1717347135', '', '1717347135', 0, 0) test DEBUG: /tmp/tmpgp98sxet/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c7726de2-2d78-48fb-ba0f-79ee2e6de6c6-sda', '1717347134.76087', '1717347134', '1717347135', '', '1717347135', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c7726de2-2d78-48fb-ba0f-79ee2e6de6c6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07987 seconds (12.51985/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpq1ckytbm/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '517a1d6a-7a35-4f4c-9125-3896e48e1207-sda', '1717347134.92306', '1717347136', '0', '', '1717347136', 0, 1) test DEBUG: /tmp/tmpq1ckytbm/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '517a1d6a-7a35-4f4c-9125-3896e48e1207-sda', '1717347134.92306', '1717347136', '0', '', '1717347136', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('517a1d6a-7a35-4f4c-9125-3896e48e1207-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06890 seconds (14.51314/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpj_89xuaf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9ca1c34590b96a5ef2fe93b0c9c07355', '0435588e-99cc-4a08-9ec0-15111a6c277c-sda', '1717347135.04512', '1717347137', '0', '', '1717347137', 1, 0) test DEBUG: /tmp/tmpj_89xuaf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:29 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0164 "-" 3610583 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0282 "-" 3610583 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3e5fc16d535544e2840e4-00665ca351" "proxy-server 3610583" 0.0039 "-" 3610583 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:53469/sdb1/1/a/c" "tx3e5fc16d535544e2840e4-00665ca351" "container-server 3610583" 0.0022 "-" 3610583 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:46155/sda1/1/a/c" "tx3e5fc16d535544e2840e4-00665ca351" "container-server 3610583" 0.0020 "-" 3610583 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx3e5fc16d535544e2840e4-00665ca351" "proxy-server 3610583" 0.0276 "-" 3610583 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx3e5fc16d535544e2840e4-00665ca351" "proxy-server 3610583" 0.0466 "-" 3610583 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/33 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx3e5fc16d535544e2840e4-00665ca351 - 0.0677 - - 1717347153.429101229 1717347153.496774673 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx8bb364d18e384cd89a61c-00665ca351" "proxy-server 3610583" 0.0026 "-" 3610583 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:53469/sdb1/1/a1/c1" "tx8bb364d18e384cd89a61c-00665ca351" "container-server 3610583" 0.0053 "-" 3610583 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:46155/sda1/1/a1/c1" "tx8bb364d18e384cd89a61c-00665ca351" "container-server 3610583" 0.0021 "-" 3610583 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx8bb364d18e384cd89a61c-00665ca351" "proxy-server 3610583" 0.0644 "-" 3610583 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx8bb364d18e384cd89a61c-00665ca351" "proxy-server 3610583" 0.0375 "-" 3610583 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/33 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx8bb364d18e384cd89a61c-00665ca351 - 0.0818 - - 1717347153.499672174 1717347153.581440449 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx54590226513843dbae026-00665ca351" "proxy-server 3610583" 0.0122 "-" 3610583 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:53469/sdb1/2/a/c1" "tx54590226513843dbae026-00665ca351" "container-server 3610583" 0.0114 "-" 3610583 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:46155/sda1/2/a/c1" "tx54590226513843dbae026-00665ca351" "container-server 3610583" 0.0027 "-" 3610583 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx54590226513843dbae026-00665ca351" "proxy-server 3610583" 0.0640 "-" 3610583 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx54590226513843dbae026-00665ca351" "proxy-server 3610583" 0.0429 "-" 3610583 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/33 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx54590226513843dbae026-00665ca351 - 0.0925 - - 1717347153.584273577 1717347153.676724672 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx25c9306666864789b3556-00665ca351" "proxy-server 3610583" 0.0133 "-" 3610583 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:53469/sdb1/1/a/c2" "tx25c9306666864789b3556-00665ca351" "container-server 3610583" 0.0092 "-" 3610583 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:46155/sda1/1/a/c2" "tx25c9306666864789b3556-00665ca351" "container-server 3610583" 0.0020 "-" 3610583 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx25c9306666864789b3556-00665ca351" "proxy-server 3610583" 0.0367 "-" 3610583 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:33 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx25c9306666864789b3556-00665ca351" "proxy-server 3610583" 0.0590 "-" 3610583 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/33 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx25c9306666864789b3556-00665ca351 - 0.0862 - - 1717347153.680012226 1717347153.766249657 2 proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.61s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (60.0s after 60.62s) (txn: tx692a4a81be8e417291c7d-00665ca2fe) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5e4a1946d40d4b98b8f6f-00665ca333) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest ... ok test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2 new rows) test-replicator ERROR: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (negative max_row_delta: -1) test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Failed to delete db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py ERROR reading HTTP response from {'replication_ip': '127.0.0.1', 'replication_port': 80, 'device': 'sdb1'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 170, in replicate self.request('REPLICATE', self.path, body, File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 353, in other_req raise Exception('blah') Exception: blah test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '9ca1c34590b96a5ef2fe93b0c9c07355', '0435588e-99cc-4a08-9ec0-15111a6c277c-sda', '1717347135.04512', '1717347137', '0', '', '1717347137', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('0435588e-99cc-4a08-9ec0-15111a6c277c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpj_89xuaf/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.81700 seconds (0.55036/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpj_89xuaf/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c9f98fbc61da7ed577eca6746d8fd73c', 'c8005199-289e-4f04-90ef-6f7cd3e8ecce-sda', '1717347136.84392', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('c8005199-289e-4f04-90ef-6f7cd3e8ecce-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c9f98fbc61da7ed577eca6746d8fd73c', 'c8005199-289e-4f04-90ef-6f7cd3e8ecce-sda', '1717347136.84392', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('c8005199-289e-4f04-90ef-6f7cd3e8ecce-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '91432e2b-cdd0-4b85-9564-03614042e2e1-sda', '1717347139.72050', '1717347139.71136', '0', '', '1717347139.71136', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpcm38i9pa/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '34b0457d-855e-4837-a206-4accd6501d43-sda', '1717347139.76447', '1717347139.75431', '0', '', '1717347139.75431', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('34b0457d-855e-4837-a206-4accd6501d43-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '34b0457d-855e-4837-a206-4accd6501d43-sda', '1717347139.76447', '1717347139.75431', '0', '', '1717347139.75431', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('34b0457d-855e-4837-a206-4accd6501d43-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09583 seconds (10.43490/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '3364e94e4fa8598f7c41d117b90d0d91', '263b37c9-4206-43b6-8e60-b1ee3cc40135-sda', '1717347139.89586', '1717347139.8895', '0', '', '1717347139.8895', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('263b37c9-4206-43b6-8e60-b1ee3cc40135-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '5d12f8bf78c5274f8f552364ca08352a', '04c4321d-2678-4b2b-8a57-3a2144aab081-sda', '1717347139.99494', '1717347139.98901', '0', '', '1717347139.98901', 11, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o_0', 'created_at': '1717347140.02070', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c/o_1', 'created_at': '1717347140.02388', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c/o_2', 'created_at': '1717347140.02556', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c/o_3', 'created_at': '1717347140.02608', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c/o_4', 'created_at': '1717347140.02666', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c/o_5', 'created_at': '1717347140.02705', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c/o_6', 'created_at': '1717347140.02749', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c/o_7', 'created_at': '1717347140.02787', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c/o_8', 'created_at': '1717347140.02819', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c/o_9', 'created_at': '1717347140.02851', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing', 'created_at': '1717347140.02974', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '04c4321d-2678-4b2b-8a57-3a2144aab081-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmppc_epv6h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '1071e86c-c626-4718-b931-657949ea47ff-sda', '1717347140.15736', '1717347143', '0', '', '1717347143', 0, 1) test DEBUG: /tmp/tmppc_epv6h/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '1071e86c-c626-4718-b931-657949ea47ff-sda', '1717347140.15736', '1717347143', '0', '', '1717347143', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('1071e86c-c626-4718-b931-657949ea47ff-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06781 seconds (14.74648/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp86nmwj_y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ebb897e5145d1015b79c22c60763c64a', '97c1f8dc-d444-4948-92ed-ef9d55c33c71-sda', '1717347140.29429', '1717347143', '0', '', '1717347143', 1, 0) test DEBUG: /tmp/tmp86nmwj_y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347144.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '97c1f8dc-d444-4948-92ed-ef9d55c33c71-sda') test DEBUG: /tmp/tmp86nmwj_y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0123353s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create ... ok test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 2.5510787963867188e-05: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0, expired ttl=1.23 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 493, in _incr_or_decr return int(line[0].strip()) ^^^^^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: b'UNEXPECTED' test WARNING: Item size larger than warning threshold: 80 (80) >= 80 (80) test WARNING: Item size larger than warning threshold: 81 (81) >= 80 (80) test WARNING: Item size larger than warning threshold: 99 (99) >= 80 (80) test WARNING: Item size larger than warning threshold: 100 (100) >= 80 (80) test WARNING: Item size larger than warning threshold: 0 (0) >= 0 (0) test WARNING: Item size larger than warning threshold: 1 (1) >= 0 (0) test WARNING: Item size larger than warning threshold: 50 (50) >= 0 (0) test WARNING: Item size larger than warning threshold: 99 (99) >= 0 (0) test WARNING: Item size larger than warning threshold: 100 (100) >= 0 (0) test WARNING: Item size larger than warning threshold: 2048576 (2Mi) >= 1000000 (977Ki) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix multi_key, method get_multi, time_spent 3.981590270996094e-05, incomplete read test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0, failed to connect test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 460, in get sock.sendall(b'get ' + cmd.hash_key + b'\r\n') File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1282, in handle_get raise Exception('get failed') Exception: get failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method get, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix incr_key, method incr, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 531, in incr new_val = self._add(fp, sock, cmd, add_val, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 496, in _add sock.sendall(b' '.join([ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1224, in handle_add raise Exception('add failed') Exception: add failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix nvratelimit/v2/wf, method incr, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix set_key, method set, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 427, in set sock.sendall(set_msg(cmd.hash_key, flags, timeout, value)) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1252, in handle_set raise Exception('set failed') Exception: set failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, config_timeout 0.01, time_spent 1.0 test.unit.common.test_memcached.TestMemcached.test_retry test.unit.common.test_memcached.TestMemcached.test_retry ... ok test.unit.common.test_memcached.TestMemcached.test_serialization test.unit.common.test_memcached.TestMemcached.test_serialization ... ok test.unit.common.test_memcached.TestMemcached.test_set_error test.unit.common.test_memcached.TestMemcached.test_set_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_get_json test.unit.common.test_memcached.TestMemcached.test_set_get_json ... ok test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg ... ok test.unit.common.test_utils.TestFallocate.test_arg_bounds test.unit.common.test_utils.TestFallocate.test_arg_bounds ... ok test.unit.common.test_utils.TestFallocate.test_fallocate test.unit.common.test_utils.TestFallocate.test_fallocate ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_offset test.unit.common.test_utils.TestFallocate.test_fallocate_offset ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors ... ok test.unit.common.test_utils.TestFallocate.test_no_fallocates_available test.unit.common.test_utils.TestFallocate.test_no_fallocates_available ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset ... ok test.unit.common.test_utils.TestLRUCache.test_maxsize test.unit.common.test_utils.TestLRUCache.test_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_maxtime test.unit.common.test_utils.TestLRUCache.test_maxtime ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxsize test.unit.common.test_utils.TestLRUCache.test_set_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxtime test.unit.common.test_utils.TestLRUCache.test_set_maxtime ... ok test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset ... ok test.unit.common.test_utils.TestShardName.test test.unit.common.test_utils.TestShardName.test ... ok test.unit.common.test_utils.TestShardName.test_bad_create test.unit.common.test_utils.TestShardName.test_bad_create ... ok test.unit.common.test_utils.TestShardName.test_bad_parse test.unit.common.test_utils.TestShardName.test_bad_parse ... ok test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names ... ok test.unit.common.test_utils.TestShardName.test_root_has_hyphens test.unit.common.test_utils.TestShardName.test_root_has_hyphens ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body ... ok test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/0 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 1, 'weight': 10.0, 'zone': 1, 'ip': '1.1.1.1', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.1', 'replication_port': 6200, 'region': 1}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 675, in _replicate_object success = self._repl_to_node(node, broker, partition, info, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 481, in _repl_to_node return self._handle_sync_response(node, response, info, broker, http, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 486, in _handle_sync_response if response.status == HTTP_NOT_FOUND: # completely missing, rsync ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 713, in status raise self._status Exception: ugh test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 2, 'weight': 10.0, 'zone': 2, 'ip': '1.1.1.2', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.2', 'replication_port': 6200, 'region': 2}: Timeout (Nones) test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (1/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator ERROR: ERROR Remote drive not mounted {'id': 3, 'weight': 10.0, 'zone': 3, 'ip': '1.1.1.3', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.3', 'replication_port': 6200, 'region': 1} test-replicator ERROR: ERROR Remote drive not mounted {'id': 4, 'weight': 10.0, 'zone': 4, 'ip': '1.1.1.4', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.4', 'replication_port': 6200, 'region': 2} test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.5:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (3/5 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator ERROR: Unable to decode remote metadata 'metadata' test-replicator ERROR: Quarantining DB test-replicator ERROR: ERROR Failed to get my own IPs? test-replicator ERROR: Can't find itself 127.0.0.1 with port 1000 in ring file, not replicating test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00038 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00080 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: Boom! test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00061 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 1 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: /srv/node/sdb is not mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_stats test.unit.common.test_db_replicator.TestDBReplicator.test_stats ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync test.unit.common.test_db_replicator.TestDBReplicator.test_usync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get test.unit.common.test_internal_client.TestSimpleClient.test_get ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param ... ok test.unit.common.test_internal_client.TestSimpleClient.test_head test.unit.common.test_internal_client.TestSimpleClient.test_head ... ok test.unit.common.test_internal_client.TestSimpleClient.test_proxy test.unit.common.test_internal_client.TestSimpleClient.test_proxy ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError ... ok test.unit.common.test_utils.TestParseContentRange.test_bad test.unit.common.test_utils.TestParseContentRange.test_bad ... ok test.unit.common.test_utils.TestParseContentRange.test_good test.unit.common.test_utils.TestParseContentRange.test_good ... ok test.unit.common.test_utils.TestPunchHole.test_arg_bounds test.unit.common.test_utils.TestPunchHole.test_arg_bounds ... ok test.unit.common.test_utils.TestPunchHole.test_error test.unit.common.test_utils.TestPunchHole.test_error ... ok test.unit.common.test_utils.TestPunchHole.test_no_fallocate test.unit.common.test_utils.TestPunchHole.test_no_fallocate ... ok test.unit.common.test_utils.TestPunchHole.test_punch_hole test.unit.common.test_utils.TestPunchHole.test_punch_hole ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path ... ok test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple ... ok proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8d45b45773724f199b287-00665ca33e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx7aee5a75105d4ed78bbcf-00665ca34e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx785a0db918b6436f96c7d-00665ca34e) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpi_8nvyk7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpi_8nvyk7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpi_8nvyk7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpi_8nvyk7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:30 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0187 "-" 3610486 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:30 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0162 "-" 3610486 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:30 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0197 "-" 3610486 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:30 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0198 "-" 3610486 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:30 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1c8a55a7fe8a4a9f875a9-00665ca34e" "proxy-server 3610486" 0.0019 "-" 3610486 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:31 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:56015/sdb1/1/a/c" "tx1c8a55a7fe8a4a9f875a9-00665ca34e" "container-server 3610486" 0.0024 "-" 3610486 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:31 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:47599/sda1/1/a/c" "tx1c8a55a7fe8a4a9f875a9-00665ca34e" "container-server 3610486" 0.0015 "-" 3610486 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:31 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx1c8a55a7fe8a4a9f875a9-00665ca34e" "proxy-server 3610486" 0.0458 "-" 3610486 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:31 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx1c8a55a7fe8a4a9f875a9-00665ca34e" "proxy-server 3610486" 0.0295 "-" 3610486 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/31 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx1c8a55a7fe8a4a9f875a9-00665ca34e - 0.0587 - - 1717347150.978522778 1717347151.037236691 0 test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.test_locale.test_locale.TestTranslations.test_translations test.unit.test_locale.test_locale.TestTranslations.test_translations ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ebb897e5145d1015b79c22c60763c64a', '97c1f8dc-d444-4948-92ed-ef9d55c33c71-sda', '1717347140.29429', '1717347143', '0', '', '1717347143', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('97c1f8dc-d444-4948-92ed-ef9d55c33c71-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp86nmwj_y/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.19468 seconds (0.83704/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp86nmwj_y/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '2addffa6c973c32e35d4271daf73dce8', '21666bd5-baad-40fa-b329-725718563dd2-sda', '1717347141.46358', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('21666bd5-baad-40fa-b329-725718563dd2-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '2addffa6c973c32e35d4271daf73dce8', '21666bd5-baad-40fa-b329-725718563dd2-sda', '1717347141.46358', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('21666bd5-baad-40fa-b329-725718563dd2-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgqdl2hp8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '155819b2-e86e-4a1a-96e1-8f5b0be4e7ae-sda', '1717347141.62319', '1717347144', '0', '', '1717347144', 0, 0) test DEBUG: /tmp/tmpgqdl2hp8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '155819b2-e86e-4a1a-96e1-8f5b0be4e7ae-sda', '1717347141.62319', '1717347144', '0', '', '1717347144', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('155819b2-e86e-4a1a-96e1-8f5b0be4e7ae-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07329 seconds (13.64377/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpbwcfejr4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ddb3912f227822e10a2d1c50d079fb13', '8d5b4a84-c206-402e-ada1-c484ae3d064a-sda', '1717347141.75870', '1717347144', '0', '', '1717347144', 1, 0) test DEBUG: /tmp/tmpbwcfejr4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ddb3912f227822e10a2d1c50d079fb13', '8d5b4a84-c206-402e-ada1-c484ae3d064a-sda', '1717347141.75870', '1717347144', '0', '', '1717347144', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8d5b4a84-c206-402e-ada1-c484ae3d064a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpbwcfejr4/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.11940 seconds (8.37535/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpbwcfejr4/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'cf95497bfe6d24f61c5a26f34e41ef43', '4e26de53-f3fb-4e6e-9844-835ee4550ccc-sda', '1717347141.87239', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('4e26de53-f3fb-4e6e-9844-835ee4550ccc-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'cf95497bfe6d24f61c5a26f34e41ef43', '4e26de53-f3fb-4e6e-9844-835ee4550ccc-sda', '1717347141.87239', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('4e26de53-f3fb-4e6e-9844-835ee4550ccc-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpyh29dlrd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fc99be58-fb21-40f0-bd45-4c10a7a2ad22-sda', '1717347143.47338', '1717347143', '1717347147', '', '1717347147', 0, 1) test DEBUG: /tmp/tmpyh29dlrd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fc99be58-fb21-40f0-bd45-4c10a7a2ad22-sda', '1717347143.47338', '1717347143', '1717347147', '', '1717347147', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fc99be58-fb21-40f0-bd45-4c10a7a2ad22-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.17134 seconds (5.83631/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpu09rlmnc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec407087515c8f3f51809d356db8b51d', '2efd9241-6936-4f01-a1d4-6d9cbec3c8db-sda', '1717347143.87611', '1717347143', '1717347147', '', '1717347147', 1, 1) test DEBUG: /tmp/tmpu09rlmnc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347148.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '2efd9241-6936-4f01-a1d4-6d9cbec3c8db-sda') test DEBUG: /tmp/tmpu09rlmnc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0137358s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ec407087515c8f3f51809d356db8b51d', '2efd9241-6936-4f01-a1d4-6d9cbec3c8db-sda', '1717347143.87611', '1717347143', '1717347147', '', '1717347147', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2efd9241-6936-4f01-a1d4-6d9cbec3c8db-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpu09rlmnc/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.17666 seconds (5.66072/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:31 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txcf025533b69c45d0a0ca7-00665ca34f" "proxy-server 3610486" 0.0018 "-" 3610486 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:56015/sdb1/1/a1/c1" "txcf025533b69c45d0a0ca7-00665ca34f" "container-server 3610486" 0.0017 "-" 3610486 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:47599/sda1/1/a1/c1" "txcf025533b69c45d0a0ca7-00665ca34f" "container-server 3610486" 0.0014 "-" 3610486 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txcf025533b69c45d0a0ca7-00665ca34f" "proxy-server 3610486" 0.0183 "-" 3610486 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txcf025533b69c45d0a0ca7-00665ca34f" "proxy-server 3610486" 0.0341 "-" 3610486 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/32 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txcf025533b69c45d0a0ca7-00665ca34f - 1.6681 - - 1717347151.039446592 1717347152.707556009 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx19f9a21d5c1b4a5ea9cac-00665ca350" "proxy-server 3610486" 0.0098 "-" 3610486 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:47599/sda1/2/a/c1" "tx19f9a21d5c1b4a5ea9cac-00665ca350" "container-server 3610486" 0.0109 "-" 3610486 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:56015/sdb1/2/a/c1" "tx19f9a21d5c1b4a5ea9cac-00665ca350" "container-server 3610486" 0.0018 "-" 3610486 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx19f9a21d5c1b4a5ea9cac-00665ca350" "proxy-server 3610486" 0.0498 "-" 3610486 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx19f9a21d5c1b4a5ea9cac-00665ca350" "proxy-server 3610486" 0.0368 "-" 3610486 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/32 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx19f9a21d5c1b4a5ea9cac-00665ca350 - 0.0704 - - 1717347152.709712982 1717347152.780088902 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx7635ddb95754484b9297e-00665ca350" "proxy-server 3610486" 0.0092 "-" 3610486 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:56015/sdb1/1/a/c2" "tx7635ddb95754484b9297e-00665ca350" "container-server 3610486" 0.0017 "-" 3610486 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:47599/sda1/1/a/c2" "tx7635ddb95754484b9297e-00665ca350" "container-server 3610486" 0.0081 "-" 3610486 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx7635ddb95754484b9297e-00665ca350" "proxy-server 3610486" 0.0282 "-" 3610486 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:32 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx7635ddb95754484b9297e-00665ca350" "proxy-server 3610486" 0.0478 "-" 3610486 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/32 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx7635ddb95754484b9297e-00665ca350 - 0.0665 - - 1717347152.782110929 1717347152.848608255 2 test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 5.42999 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 6 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 0 dbs in 0.00000 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 30 dbs in 246.81358 seconds (0.12155/s) test-replicator INFO: Removed 9 dbs test-replicator INFO: 25 successes, 1 failures test-replicator INFO: diff:5 diff_capped:4 empty:7 hashmatch:8 no_change:6 remote_merge:2 rsync:3 ts_repl:10 test-ic DEBUG: -> 2014-05-27T20:54:11 GET http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - test-ic DEBUG: -> 2014-05-27T20:54:11 HEAD http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txfba692ced19e4605900d0-00665ca358) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txfba692ced19e4605900d0-00665ca358) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx369b4abb4ddf4fb0ac930-00665ca359) proxy-server ERROR: Object GET returning 503 for [] (txn: tx369b4abb4ddf4fb0ac930-00665ca359) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout ... ok test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpu09rlmnc/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '77ca08e4a45c154120eb1ef0dc3eb8a3', 'd13c5e00-7259-425f-856e-24515fbb347b-sda', '1717347144.12244', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d13c5e00-7259-425f-856e-24515fbb347b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '77ca08e4a45c154120eb1ef0dc3eb8a3', 'd13c5e00-7259-425f-856e-24515fbb347b-sda', '1717347144.12244', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d13c5e00-7259-425f-856e-24515fbb347b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpm627l0ep/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '80560169-29a4-4486-9cfb-0602feb7d0fa-sda', '1717347147.27096', '1717347147', '1717347151', '', '1717347151', 0, 1) test DEBUG: /tmp/tmpm627l0ep/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '80560169-29a4-4486-9cfb-0602feb7d0fa-sda', '1717347147.27096', '1717347147', '1717347151', '', '1717347151', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('80560169-29a4-4486-9cfb-0602feb7d0fa-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09418 seconds (10.61845/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpcztr5_rj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b53eb5f40d1adf408966603fd1827114', 'f0a51020-6600-4853-88c8-94c993cf5803-sda', '1717347147.47716', '1717347147', '1717347151', '', '1717347151', 1, 1) test DEBUG: /tmp/tmpcztr5_rj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b53eb5f40d1adf408966603fd1827114', 'f0a51020-6600-4853-88c8-94c993cf5803-sda', '1717347147.47716', '1717347147', '1717347151', '', '1717347151', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f0a51020-6600-4853-88c8-94c993cf5803-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpcztr5_rj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13189 seconds (7.58200/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpcztr5_rj/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '5479d72eac074cfeabec0fff1d9232f0', '39bfd65c-cd68-4e40-b0b7-cf9aa897c8ec-sda', '1717347147.65697', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('39bfd65c-cd68-4e40-b0b7-cf9aa897c8ec-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '5479d72eac074cfeabec0fff1d9232f0', '39bfd65c-cd68-4e40-b0b7-cf9aa897c8ec-sda', '1717347147.65697', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('39bfd65c-cd68-4e40-b0b7-cf9aa897c8ec-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwap722b4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b1f83c1a-183e-4afd-9ef7-5ecb87fae005-sda', '1717347148.93507', '1717347148', '0', '', '1717347148', 0, 1) test DEBUG: /tmp/tmpwap722b4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b1f83c1a-183e-4afd-9ef7-5ecb87fae005-sda', '1717347148.93507', '1717347148', '0', '', '1717347148', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b1f83c1a-183e-4afd-9ef7-5ecb87fae005-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08710 seconds (11.48060/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptz2zqc3l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '80f962c22c91a4d0bad03d278b5044d4', 'd461fc50-2253-4b9c-ad3c-00b1ab10dbe6-sda', '1717347149.12107', '1717347149', '0', '', '1717347149', 1, 1) test DEBUG: /tmp/tmptz2zqc3l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347153.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'd461fc50-2253-4b9c-ad3c-00b1ab10dbe6-sda') test DEBUG: /tmp/tmptz2zqc3l/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.020503s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '80f962c22c91a4d0bad03d278b5044d4', 'd461fc50-2253-4b9c-ad3c-00b1ab10dbe6-sda', '1717347149.12107', '1717347149', '0', '', '1717347149', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d461fc50-2253-4b9c-ad3c-00b1ab10dbe6-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmptz2zqc3l/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.19157 seconds (5.21999/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmptz2zqc3l/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '98089ed2867e32f3924cb11199e63a77', '01bfc633-0c39-4a98-88c7-925b543cc0ce-sda', '1717347149.33943', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('01bfc633-0c39-4a98-88c7-925b543cc0ce-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create ... ok proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3c8cab5ea6f4c81b5bb1-00665ca343) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347492.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/982/116da6507c7a70e61082e4d77cbed982/116da6507c7a70e61082e4d77cbed982.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347496.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b29/9f5294e180318fe2c2801f2efcec7b29/9f5294e180318fe2c2801f2efcec7b29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347500.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/dc6/534adff96c9f0b0861e5ce86a5b5ddc6/534adff96c9f0b0861e5ce86a5b5ddc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347504.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/e3a/311a786e16d240ca814b5347309bae3a/311a786e16d240ca814b5347309bae3a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347508.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4d8/f5390b915f32f9df287781f0610ef4d8/f5390b915f32f9df287781f0610ef4d8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347512.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/f21/fc1f85d6ae31202ac168582efae03f21/fc1f85d6ae31202ac168582efae03f21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347516.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/c83/40771d6f040a89cd670ab1e7cd402c83/40771d6f040a89cd670ab1e7cd402c83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347520.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/c63/0fde1523df372dbe620238e245817c63/0fde1523df372dbe620238e245817c63.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347524.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a64/7c7e23d75436ca7d0dbf23dd9040ea64/7c7e23d75436ca7d0dbf23dd9040ea64.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347528.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/c3c/19268222c7cdc5977d87d08d1fb2bc3c/19268222c7cdc5977d87d08d1fb2bc3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347532.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/0df/45513448bd4540679fd352ebaf5180df/45513448bd4540679fd352ebaf5180df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347536.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b2d/a9d22f3355b7608297c18e95f068eb2d/a9d22f3355b7608297c18e95f068eb2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347540.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b17/30d00638031a29f7bb0a65af57374b17/30d00638031a29f7bb0a65af57374b17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347544.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/cfe/58f5c05398554c4cb7cbd2c540896cfe/58f5c05398554c4cb7cbd2c540896cfe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347548.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/6df/70611c0ba4813a9f69ec304e323826df/70611c0ba4813a9f69ec304e323826df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347552.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/52f/e4ae97a50ff6844fb1118e92ea3cd52f/e4ae97a50ff6844fb1118e92ea3cd52f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347556.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/bd4/6131607b2464f36af991062975070bd4/6131607b2464f36af991062975070bd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347560.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/a70/793dd519b079c475c0c10570e6bf7a70/793dd519b079c475c0c10570e6bf7a70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347564.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/b5e/a28cef3eaf6087059d66c7e7a365ab5e/a28cef3eaf6087059d66c7e7a365ab5e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347568.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/dcb/baaf505e2e5a1f06444002d3435d1dcb/baaf505e2e5a1f06444002d3435d1dcb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347572.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/8bf/9161eacaef5cbf66ff805619e7a6e8bf/9161eacaef5cbf66ff805619e7a6e8bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347576.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/4b8/2480e27a7bd428748e78db4a5f9694b8/2480e27a7bd428748e78db4a5f9694b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347580.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/f56/9c89e88ed1382d8157763a98ac59bf56/9c89e88ed1382d8157763a98ac59bf56.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347584.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/8ba/fd67c357747e438b715abc1516b398ba/fd67c357747e438b715abc1516b398ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347588.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/17b/e0b7ae67c3882bcaf08b86ae6011717b/e0b7ae67c3882bcaf08b86ae6011717b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347592.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/9b0/99a535e40c925c8c09f57ce8f52479b0/99a535e40c925c8c09f57ce8f52479b0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347596.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/952/0d04f62a905f37d3008784e8df3f8952/0d04f62a905f37d3008784e8df3f8952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347600.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/02c/c963245de37fe34ad4083a0687d2702c/c963245de37fe34ad4083a0687d2702c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347604.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/3ba/c797fe14a96763a0daa5cfe761f623ba/c797fe14a96763a0daa5cfe761f623ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347608.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/cb5/94096e279e8022b3c12b658150de5cb5/94096e279e8022b3c12b658150de5cb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347612.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/00b/db4fa3e7f8c43ad63e7bd93837a1200b/db4fa3e7f8c43ad63e7bd93837a1200b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmp9jabfnlw/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmp8rd5lsup/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347124.00000.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmp8rd5lsup/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347124.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347132.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/0c6/c6cdd20e0f6be85f4b11531102a7a0c6/c6cdd20e0f6be85f4b11531102a7a0c6_1717347132.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347132.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/0c6/c6cdd20e0f6be85f4b11531102a7a0c6/c6cdd20e0f6be85f4b11531102a7a0c6_1717347132.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347135.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/5db/59fb0027511b26e0637229302e8615db/59fb0027511b26e0637229302e8615db_1717347135.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347135.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/5db/59fb0027511b26e0637229302e8615db/59fb0027511b26e0637229302e8615db_1717347135.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347138.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b90/f508b8c7647d0e0d5fd463b248a67b90/f508b8c7647d0e0d5fd463b248a67b90_1717347138.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347138.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b90/f508b8c7647d0e0d5fd463b248a67b90/f508b8c7647d0e0d5fd463b248a67b90_1717347138.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347141.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/bcf/6e621ee4c21a1808af8634a9d70a0bcf/6e621ee4c21a1808af8634a9d70a0bcf_1717347141.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347141.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/bcf/6e621ee4c21a1808af8634a9d70a0bcf/6e621ee4c21a1808af8634a9d70a0bcf_1717347141.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347144.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/9f5/d63dbd9dd994944ce0e026d382d1e9f5/d63dbd9dd994944ce0e026d382d1e9f5_1717347144.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347144.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/9f5/d63dbd9dd994944ce0e026d382d1e9f5/d63dbd9dd994944ce0e026d382d1e9f5_1717347144.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347147.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/02d/4ebd1c7316d34de4b34dd654ccede02d/4ebd1c7316d34de4b34dd654ccede02d_1717347147.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347147.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/02d/4ebd1c7316d34de4b34dd654ccede02d/4ebd1c7316d34de4b34dd654ccede02d_1717347147.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347150.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/c03/04db923c2025be36f3b7a036af418c03/04db923c2025be36f3b7a036af418c03_1717347150.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347150.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/c03/04db923c2025be36f3b7a036af418c03/04db923c2025be36f3b7a036af418c03_1717347150.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347153.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/4e0/2e1c2cf4c9306f2212b25291b86124e0/2e1c2cf4c9306f2212b25291b86124e0_1717347153.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347153.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/4e0/2e1c2cf4c9306f2212b25291b86124e0/2e1c2cf4c9306f2212b25291b86124e0_1717347153.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347156.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/71f/d52a6a6ad8082a975da4513a0c3bc71f/d52a6a6ad8082a975da4513a0c3bc71f_1717347156.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347156.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/71f/d52a6a6ad8082a975da4513a0c3bc71f/d52a6a6ad8082a975da4513a0c3bc71f_1717347156.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347159.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/481/0d6ea938cc5061abbb9eeeae7e13b481/0d6ea938cc5061abbb9eeeae7e13b481_1717347159.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347159.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/481/0d6ea938cc5061abbb9eeeae7e13b481/0d6ea938cc5061abbb9eeeae7e13b481_1717347159.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347162.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ac5/3400da77a1fd8a51a45c043b42582ac5/3400da77a1fd8a51a45c043b42582ac5_1717347162.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347162.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ac5/3400da77a1fd8a51a45c043b42582ac5/3400da77a1fd8a51a45c043b42582ac5_1717347162.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347165.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/0f6/eeb72de362c380c3796e36edd48b20f6/eeb72de362c380c3796e36edd48b20f6_1717347165.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347165.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/0f6/eeb72de362c380c3796e36edd48b20f6/eeb72de362c380c3796e36edd48b20f6_1717347165.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347168.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/9e7/6eeb000df831432e17b94c558081c9e7/6eeb000df831432e17b94c558081c9e7_1717347168.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347168.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/9e7/6eeb000df831432e17b94c558081c9e7/6eeb000df831432e17b94c558081c9e7_1717347168.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347171.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/e58/64187d1dbb62210e8df0a37eed086e58/64187d1dbb62210e8df0a37eed086e58_1717347171.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347171.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/e58/64187d1dbb62210e8df0a37eed086e58/64187d1dbb62210e8df0a37eed086e58_1717347171.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347174.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/78d/d1b7305c5b0ecd322f04950bfa3d678d/d1b7305c5b0ecd322f04950bfa3d678d_1717347174.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347174.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/78d/d1b7305c5b0ecd322f04950bfa3d678d/d1b7305c5b0ecd322f04950bfa3d678d_1717347174.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347177.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/27b/a23aa3ae6d57c38ca4eceacd51d1627b/a23aa3ae6d57c38ca4eceacd51d1627b_1717347177.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347177.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/27b/a23aa3ae6d57c38ca4eceacd51d1627b/a23aa3ae6d57c38ca4eceacd51d1627b_1717347177.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347180.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b9a/edb8119cd4d2630bc865f52f81544b9a/edb8119cd4d2630bc865f52f81544b9a_1717347180.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347180.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b9a/edb8119cd4d2630bc865f52f81544b9a/edb8119cd4d2630bc865f52f81544b9a_1717347180.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347183.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/7ed/f8c01d775095ff8ec816acfa94a0b7ed/f8c01d775095ff8ec816acfa94a0b7ed_1717347183.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347183.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/7ed/f8c01d775095ff8ec816acfa94a0b7ed/f8c01d775095ff8ec816acfa94a0b7ed_1717347183.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347186.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b68/d1cf8a60f7bf9b0415f9d61e896b3b68/d1cf8a60f7bf9b0415f9d61e896b3b68_1717347186.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347186.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b68/d1cf8a60f7bf9b0415f9d61e896b3b68/d1cf8a60f7bf9b0415f9d61e896b3b68_1717347186.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347189.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/c13/48d5dbfb0a1c2c1bb8eaf812701cdc13/48d5dbfb0a1c2c1bb8eaf812701cdc13_1717347189.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347189.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/c13/48d5dbfb0a1c2c1bb8eaf812701cdc13/48d5dbfb0a1c2c1bb8eaf812701cdc13_1717347189.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347192.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/6fa/158fd7385158515875860fc8a69966fa/158fd7385158515875860fc8a69966fa_1717347192.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347192.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/6fa/158fd7385158515875860fc8a69966fa/158fd7385158515875860fc8a69966fa_1717347192.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347195.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/2dc/11d3fc44d684c2ca9264da909b89e2dc/11d3fc44d684c2ca9264da909b89e2dc_1717347195.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347195.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/2dc/11d3fc44d684c2ca9264da909b89e2dc/11d3fc44d684c2ca9264da909b89e2dc_1717347195.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347198.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/f79/be145ba856acc07f2de1e7e6d684ff79/be145ba856acc07f2de1e7e6d684ff79_1717347198.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347198.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/f79/be145ba856acc07f2de1e7e6d684ff79/be145ba856acc07f2de1e7e6d684ff79_1717347198.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347201.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/a73/ba77b742b64680e0c5d093ec0937fa73/ba77b742b64680e0c5d093ec0937fa73_1717347201.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347201.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/a73/ba77b742b64680e0c5d093ec0937fa73/ba77b742b64680e0c5d093ec0937fa73_1717347201.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347204.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/4d3/fc8f5bf8f660cb78da09caf6faac54d3/fc8f5bf8f660cb78da09caf6faac54d3_1717347204.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347204.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/4d3/fc8f5bf8f660cb78da09caf6faac54d3/fc8f5bf8f660cb78da09caf6faac54d3_1717347204.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa_1717347207.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa_1717347207.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763_1717347210.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763_1717347210.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347213.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/3bc/16db9adf175371ed8712e53cbc2ce3bc/16db9adf175371ed8712e53cbc2ce3bc_1717347213.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347213.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/3bc/16db9adf175371ed8712e53cbc2ce3bc/16db9adf175371ed8712e53cbc2ce3bc_1717347213.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61_1717347216.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61_1717347216.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347219.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/33e/f50715e7e4da888a7a403d93ef5d933e/f50715e7e4da888a7a403d93ef5d933e_1717347219.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347219.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/33e/f50715e7e4da888a7a403d93ef5d933e/f50715e7e4da888a7a403d93ef5d933e_1717347219.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347222.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/482/3bc165f881aedbd2b423553496b14482/3bc165f881aedbd2b423553496b14482_1717347222.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347222.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/482/3bc165f881aedbd2b423553496b14482/3bc165f881aedbd2b423553496b14482_1717347222.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347225.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/570/a189990dd1593c1861c4d54a6925b570/a189990dd1593c1861c4d54a6925b570_1717347225.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347225.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/570/a189990dd1593c1861c4d54a6925b570/a189990dd1593c1861c4d54a6925b570_1717347225.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347228.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/906/f58109f54c63f9a53b744430e1dba906/f58109f54c63f9a53b744430e1dba906_1717347228.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347228.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/906/f58109f54c63f9a53b744430e1dba906/f58109f54c63f9a53b744430e1dba906_1717347228.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4_1717347231.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4_1717347231.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347234.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/c1f/f39933cb0b09682ee8bb0554640c8c1f/f39933cb0b09682ee8bb0554640c8c1f_1717347234.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347234.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/c1f/f39933cb0b09682ee8bb0554640c8c1f/f39933cb0b09682ee8bb0554640c8c1f_1717347234.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347237.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/bb1/09d45f0f99db59e44804209cbe007bb1/09d45f0f99db59e44804209cbe007bb1_1717347237.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347237.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/bb1/09d45f0f99db59e44804209cbe007bb1/09d45f0f99db59e44804209cbe007bb1_1717347237.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347240.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/d07/8462b1d830f66070f109037b39029d07/8462b1d830f66070f109037b39029d07_1717347240.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347240.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/d07/8462b1d830f66070f109037b39029d07/8462b1d830f66070f109037b39029d07_1717347240.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347243.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/df7/c00a00be6de65f597e910ebb4e958df7/c00a00be6de65f597e910ebb4e958df7_1717347243.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347243.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/df7/c00a00be6de65f597e910ebb4e958df7/c00a00be6de65f597e910ebb4e958df7_1717347243.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd_1717347246.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd_1717347246.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347249.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/8d7/96f18f6b8df6044fed25f1de0a07f8d7/96f18f6b8df6044fed25f1de0a07f8d7_1717347249.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347249.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/8d7/96f18f6b8df6044fed25f1de0a07f8d7/96f18f6b8df6044fed25f1de0a07f8d7_1717347249.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347252.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/bd3/b6280feb989c820da91430dd7f1d0bd3/b6280feb989c820da91430dd7f1d0bd3_1717347252.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347252.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/bd3/b6280feb989c820da91430dd7f1d0bd3/b6280feb989c820da91430dd7f1d0bd3_1717347252.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347255.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/04b/521252a77ebf5a7f06960774f821804b/521252a77ebf5a7f06960774f821804b_1717347255.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347255.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/04b/521252a77ebf5a7f06960774f821804b/521252a77ebf5a7f06960774f821804b_1717347255.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347258.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/432/69acb38df4c85b8ea722733317ccb432/69acb38df4c85b8ea722733317ccb432_1717347258.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347258.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/432/69acb38df4c85b8ea722733317ccb432/69acb38df4c85b8ea722733317ccb432_1717347258.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347261.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/8ac/81e1169bc4e5fb82f1f14eb5ad7e78ac/81e1169bc4e5fb82f1f14eb5ad7e78ac_1717347261.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347261.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/8ac/81e1169bc4e5fb82f1f14eb5ad7e78ac/81e1169bc4e5fb82f1f14eb5ad7e78ac_1717347261.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347264.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ec7/bac4bc06c3b9745b2e448f664b31cec7/bac4bc06c3b9745b2e448f664b31cec7_1717347264.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347264.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ec7/bac4bc06c3b9745b2e448f664b31cec7/bac4bc06c3b9745b2e448f664b31cec7_1717347264.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347267.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b69/82e22ad90ffdc866aa2f1d347b2fbb69/82e22ad90ffdc866aa2f1d347b2fbb69_1717347267.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347267.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/b69/82e22ad90ffdc866aa2f1d347b2fbb69/82e22ad90ffdc866aa2f1d347b2fbb69_1717347267.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347270.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/27c/57e1f02445fb7f62caa6b9f66d43a27c/57e1f02445fb7f62caa6b9f66d43a27c_1717347270.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347270.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/27c/57e1f02445fb7f62caa6b9f66d43a27c/57e1f02445fb7f62caa6b9f66d43a27c_1717347270.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347273.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/33d/8c83737c8f7cecbe96e7e3b5bbf2d33d/8c83737c8f7cecbe96e7e3b5bbf2d33d_1717347273.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347273.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/33d/8c83737c8f7cecbe96e7e3b5bbf2d33d/8c83737c8f7cecbe96e7e3b5bbf2d33d_1717347273.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6_1717347276.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6_1717347276.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347279.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/778/d87a1e9d0933d5e425cc6be10d38e778/d87a1e9d0933d5e425cc6be10d38e778_1717347279.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347279.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/778/d87a1e9d0933d5e425cc6be10d38e778/d87a1e9d0933d5e425cc6be10d38e778_1717347279.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347282.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/0b8/cc1af7f7dbec7aae69232417e0b890b8/cc1af7f7dbec7aae69232417e0b890b8_1717347282.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347282.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/0b8/cc1af7f7dbec7aae69232417e0b890b8/cc1af7f7dbec7aae69232417e0b890b8_1717347282.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347285.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/20d/21453becc52d0fe6863eec62ef6ea20d/21453becc52d0fe6863eec62ef6ea20d_1717347285.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347285.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/20d/21453becc52d0fe6863eec62ef6ea20d/21453becc52d0fe6863eec62ef6ea20d_1717347285.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347288.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/370/22fb67f5943b6195582d689c6d5b9370/22fb67f5943b6195582d689c6d5b9370_1717347288.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347288.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/370/22fb67f5943b6195582d689c6d5b9370/22fb67f5943b6195582d689c6d5b9370_1717347288.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104_1717347291.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104_1717347291.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347294.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/fad/c314653a706a109dc59e420aa317efad/c314653a706a109dc59e420aa317efad_1717347294.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347294.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/fad/c314653a706a109dc59e420aa317efad/c314653a706a109dc59e420aa317efad_1717347294.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347297.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/500/22c14b20dc8ecb567b90e51dfc0a4500/22c14b20dc8ecb567b90e51dfc0a4500_1717347297.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347297.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/500/22c14b20dc8ecb567b90e51dfc0a4500/22c14b20dc8ecb567b90e51dfc0a4500_1717347297.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347300.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/621/eb0ffcb557d2dc1350343916fadc4621/eb0ffcb557d2dc1350343916fadc4621_1717347300.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347300.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/621/eb0ffcb557d2dc1350343916fadc4621/eb0ffcb557d2dc1350343916fadc4621_1717347300.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347303.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ab4/7d2af95f451fea4d55b5f950c2efcab4/7d2af95f451fea4d55b5f950c2efcab4_1717347303.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347303.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ab4/7d2af95f451fea4d55b5f950c2efcab4/7d2af95f451fea4d55b5f950c2efcab4_1717347303.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60_1717347306.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60_1717347306.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347309.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ce9/2907c6ce038b9d8b8c50950486ceece9/2907c6ce038b9d8b8c50950486ceece9_1717347309.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347309.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ce9/2907c6ce038b9d8b8c50950486ceece9/2907c6ce038b9d8b8c50950486ceece9_1717347309.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347312.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/f74/89b43ec01c98425c5d156d89116cdf74/89b43ec01c98425c5d156d89116cdf74_1717347312.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347312.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/f74/89b43ec01c98425c5d156d89116cdf74/89b43ec01c98425c5d156d89116cdf74_1717347312.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347315.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/db2/b4bb66fd1dc808f7353ed339a3ea3db2/b4bb66fd1dc808f7353ed339a3ea3db2_1717347315.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347315.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/db2/b4bb66fd1dc808f7353ed339a3ea3db2/b4bb66fd1dc808f7353ed339a3ea3db2_1717347315.00000.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names ... ok proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa3da167ae0084a328ebc7-00665ca359) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb1e416476ffe4cdea14f2-00665ca359) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc4ac32192e640f4ae4b4-00665ca359) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured ... ok REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '98089ed2867e32f3924cb11199e63a77', '01bfc633-0c39-4a98-88c7-925b543cc0ce-sda', '1717347149.33943', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('01bfc633-0c39-4a98-88c7-925b543cc0ce-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpysi2lmdf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '622ab98c-0ab4-425f-b1a5-329cb2d14d2c-sda', '1717347151.03794', '1717347151', '0', '', '1717347151', 0, 0) test DEBUG: /tmp/tmpysi2lmdf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '622ab98c-0ab4-425f-b1a5-329cb2d14d2c-sda', '1717347151.03794', '1717347151', '0', '', '1717347151', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('622ab98c-0ab4-425f-b1a5-329cb2d14d2c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08754 seconds (11.42319/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp4dpfhrnc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2f7c4a57ddce005d35de0f293a41e19d', '96189a0e-0104-43ca-b100-f7090c5c0ba3-sda', '1717347151.20341', '1717347151', '0', '', '1717347151', 1, 0) test DEBUG: /tmp/tmp4dpfhrnc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2f7c4a57ddce005d35de0f293a41e19d', '96189a0e-0104-43ca-b100-f7090c5c0ba3-sda', '1717347151.20341', '1717347151', '0', '', '1717347151', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('96189a0e-0104-43ca-b100-f7090c5c0ba3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp4dpfhrnc/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12385 seconds (8.07433/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp4dpfhrnc/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '2fefccda7ca88fed949338a7fd350ae4', '53fb6756-011c-44fc-8936-eb92a5592fd3-sda', '1717347151.36013', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('53fb6756-011c-44fc-8936-eb92a5592fd3-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '2fefccda7ca88fed949338a7fd350ae4', '53fb6756-011c-44fc-8936-eb92a5592fd3-sda', '1717347151.36013', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('53fb6756-011c-44fc-8936-eb92a5592fd3-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpnyx_i2au/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '68a60d1b-db64-4bb4-b44f-1eac776d8923-sda', '1717347153.29379', '1717347153', '1717347155', '', '1717347155', 0, 0) test DEBUG: /tmp/tmpnyx_i2au/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '68a60d1b-db64-4bb4-b44f-1eac776d8923-sda', '1717347153.29379', '1717347153', '1717347155', '', '1717347155', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('68a60d1b-db64-4bb4-b44f-1eac776d8923-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05548 seconds (18.02587/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5kb9eaww/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '37a6ec7d845fe94c8d71277a83cb72fb', '4dfabd80-18ce-4b49-923b-9771b061f63e-sda', '1717347153.42695', '1717347153', '1717347155', '', '1717347155', 1, 0) test DEBUG: /tmp/tmp5kb9eaww/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347158.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '4dfabd80-18ce-4b49-923b-9771b061f63e-sda') test DEBUG: /tmp/tmp5kb9eaww/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.020612s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '37a6ec7d845fe94c8d71277a83cb72fb', '4dfabd80-18ce-4b49-923b-9771b061f63e-sda', '1717347153.42695', '1717347153', '1717347155', '', '1717347155', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('4dfabd80-18ce-4b49-923b-9771b061f63e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp5kb9eaww/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15496 seconds (6.45323/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp5kb9eaww/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'b20c4b1ebf3cab60ddb48d7277ffe05d', 'a9d88b47-1bee-41d8-ae0a-838bf1b82cde-sda', '1717347153.59083', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('a9d88b47-1bee-41d8-ae0a-838bf1b82cde-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'b20c4b1ebf3cab60ddb48d7277ffe05d', 'a9d88b47-1bee-41d8-ae0a-838bf1b82cde-sda', '1717347153.59083', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('a9d88b47-1bee-41d8-ae0a-838bf1b82cde-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpis4u1rqj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config ... ok proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb8721b4518324054872a9-00665ca344) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a9499b9e-a593-4e3b-80ec-85b5db9b32f1-sda', '1717347153.68599', '1717347153', '1717347155', '', '1717347155', 0, 1) test DEBUG: /tmp/tmpis4u1rqj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a9499b9e-a593-4e3b-80ec-85b5db9b32f1-sda', '1717347153.68599', '1717347153', '1717347155', '', '1717347155', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a9499b9e-a593-4e3b-80ec-85b5db9b32f1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.31629 seconds (0.43173/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpb_5es3t7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b3472af3dbdaee39dbc6c6d08ef6d570', '1e2f66fe-0a13-4e8b-ba8f-9417c1adce3b-sda', '1717347156.08057', '1717347156', '1717347158', '', '1717347158', 1, 1) test DEBUG: /tmp/tmpb_5es3t7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b3472af3dbdaee39dbc6c6d08ef6d570', '1e2f66fe-0a13-4e8b-ba8f-9417c1adce3b-sda', '1717347156.08057', '1717347156', '1717347158', '', '1717347158', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('1e2f66fe-0a13-4e8b-ba8f-9417c1adce3b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpb_5es3t7/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14512 seconds (6.89085/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpb_5es3t7/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '46fceac7f6e3240ee23840aa47e740e7', 'd77a1e7e-3570-41f5-b8c8-8968e254c369-sda', '1717347156.26512', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d77a1e7e-3570-41f5-b8c8-8968e254c369-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '46fceac7f6e3240ee23840aa47e740e7', 'd77a1e7e-3570-41f5-b8c8-8968e254c369-sda', '1717347156.26512', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d77a1e7e-3570-41f5-b8c8-8968e254c369-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpnn0oo1ks/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd892e6e2-bd81-428c-a3a5-897b7ec3caf9-sda', '1717347158.40645', '1717347161', '1717347160', '', '1717347161', 0, 0) test DEBUG: /tmp/tmpnn0oo1ks/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd892e6e2-bd81-428c-a3a5-897b7ec3caf9-sda', '1717347158.40645', '1717347161', '1717347160', '', '1717347161', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d892e6e2-bd81-428c-a3a5-897b7ec3caf9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06446 seconds (15.51329/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp31iradte/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '27b720d1047ce2b4e645cc2697ad1276', '012fc30b-02b1-4c2d-a4cb-b727f885f14a-sda', '1717347158.57995', '1717347161', '1717347160', '', '1717347161', 1, 0) test DEBUG: /tmp/tmp31iradte/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347164.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '012fc30b-02b1-4c2d-a4cb-b727f885f14a-sda') test DEBUG: /tmp/tmp31iradte/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0236945s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '27b720d1047ce2b4e645cc2697ad1276', '012fc30b-02b1-4c2d-a4cb-b727f885f14a-sda', '1717347158.57995', '1717347161', '1717347160', '', '1717347161', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('012fc30b-02b1-4c2d-a4cb-b727f885f14a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp31iradte/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.16199 seconds (6.17321/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp31iradte/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c3a8abbd38c149a45e08e39c5825ec3e', '02c9ae1f-0107-496a-87b5-487ef705ed1e-sda', '1717347158.79162', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('02c9ae1f-0107-496a-87b5-487ef705ed1e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'c3a8abbd38c149a45e08e39c5825ec3e', '02c9ae1f-0107-496a-87b5-487ef705ed1e-sda', '1717347158.79162', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('02c9ae1f-0107-496a-87b5-487ef705ed1e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_dqog5kp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '1e1c336e-4fcf-4690-9b46-e50fd924278e-sda', '1717347158.90021', '1717347161', '1717347160', '', '1717347161', 0, 0) test DEBUG: /tmp/tmp_dqog5kp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '1e1c336e-4fcf-4690-9b46-e50fd924278e-sda', '1717347158.90021', '1717347161', '1717347160', '', '1717347161', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync ... ok proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmpu43e8z64/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpu43e8z64/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmpu43e8z64/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpu43e8z64/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0113 "-" 3610679 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0118 "-" 3610679 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0105 "-" 3610679 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0131 "-" 3610679 - test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_remove_objects test.unit.container.test_backend.TestContainerBroker.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states ... ok test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage2.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage3.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f4f1d86a800>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test DEBUG: User tenant_id with role(s) rdr1,rdr2 has project reader authorizing test INFO: Begin container audit "once" mode test DEBUG: Audit passed for /tmp/tmpzf06sro7/sda/containers/0/0/0/test.db test INFO: Container audit "once" mode completed: 0.00s test ERROR: Problem finding shard upper in '/tmp/tmpr6tto800/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpr6tto800/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpjj06ej3_/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpjj06ej3_/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBroker.test_set_db_states test.unit.container.test_backend.TestContainerBroker.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3da85509c0af463cbf0b7-00665ca35c" "proxy-server 3610679" 0.0026 "-" 3610679 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:41761/sda1/1/a/c" "tx3da85509c0af463cbf0b7-00665ca35c" "container-server 3610679" 0.0026 "-" 3610679 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:54021/sdb1/1/a/c" "tx3da85509c0af463cbf0b7-00665ca35c" "container-server 3610679" 0.0021 "-" 3610679 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx3da85509c0af463cbf0b7-00665ca35c" "proxy-server 3610679" 0.0415 "-" 3610679 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx3da85509c0af463cbf0b7-00665ca35c" "proxy-server 3610679" 0.0269 "-" 3610679 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/44 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx3da85509c0af463cbf0b7-00665ca35c - 0.0575 - - 1717347164.901186228 1717347164.958705664 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:44 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx527ecf7ac0884b959865f-00665ca35c" "proxy-server 3610679" 0.0024 "-" 3610679 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:54021/sdb1/1/a1/c1" "tx527ecf7ac0884b959865f-00665ca35c" "container-server 3610679" 0.0019 "-" 3610679 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:41761/sda1/1/a1/c1" "tx527ecf7ac0884b959865f-00665ca35c" "container-server 3610679" 0.0082 "-" 3610679 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx527ecf7ac0884b959865f-00665ca35c" "proxy-server 3610679" 0.0454 "-" 3610679 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx527ecf7ac0884b959865f-00665ca35c" "proxy-server 3610679" 0.0308 "-" 3610679 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/45 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx527ecf7ac0884b959865f-00665ca35c - 0.0592 - - 1717347164.961434364 1717347165.020599365 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9d66c22bd77d45b5bff9e-00665ca35d" "proxy-server 3610679" 0.0091 "-" 3610679 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:54021/sdb1/2/a/c1" "tx9d66c22bd77d45b5bff9e-00665ca35d" "container-server 3610679" 0.0071 "-" 3610679 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:41761/sda1/2/a/c1" "tx9d66c22bd77d45b5bff9e-00665ca35d" "container-server 3610679" 0.0017 "-" 3610679 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx9d66c22bd77d45b5bff9e-00665ca35d" "proxy-server 3610679" 0.0278 "-" 3610679 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx9d66c22bd77d45b5bff9e-00665ca35d" "proxy-server 3610679" 0.0448 "-" 3610679 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/45 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx9d66c22bd77d45b5bff9e-00665ca35d - 0.0629 - - 1717347165.022617340 1717347165.085536242 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx429565e71f6648d6a155a-00665ca35d" "proxy-server 3610679" 0.0103 "-" 3610679 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:54021/sdb1/1/a/c2" "tx429565e71f6648d6a155a-00665ca35d" "container-server 3610679" 0.0016 "-" 3610679 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:41761/sda1/1/a/c2" "tx429565e71f6648d6a155a-00665ca35d" "container-server 3610679" 0.0072 "-" 3610679 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx429565e71f6648d6a155a-00665ca35d" "proxy-server 3610679" 0.0386 "-" 3610679 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:45 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx429565e71f6648d6a155a-00665ca35d" "proxy-server 3610679" 0.0265 "-" 3610679 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/45 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx429565e71f6648d6a155a-00665ca35d - 0.0590 - - 1717347165.087522030 1717347165.146496296 2 test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBroker.test_sync_merging test.unit.container.test_backend.TestContainerBroker.test_sync_merging ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices ... ok test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347318.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ae7/b62bf68e010614da9cda9b4ff25bcae7/b62bf68e010614da9cda9b4ff25bcae7_1717347318.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347318.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/ae7/b62bf68e010614da9cda9b4ff25bcae7/b62bf68e010614da9cda9b4ff25bcae7_1717347318.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347321.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/287/8358b8edaaf2a01d62c53bf67645e287/8358b8edaaf2a01d62c53bf67645e287_1717347321.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347321.00000-0, db: /tmp/tmphfk1qme2/sda/containers/0/287/8358b8edaaf2a01d62c53bf67645e287/8358b8edaaf2a01d62c53bf67645e287_1717347321.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347146.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/eb6/cf136393d10386a0a341fba0d77e5eb6/cf136393d10386a0a341fba0d77e5eb6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347146.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/eb6/cf136393d10386a0a341fba0d77e5eb6/cf136393d10386a0a341fba0d77e5eb6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347150.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/c03/04db923c2025be36f3b7a036af418c03/04db923c2025be36f3b7a036af418c03.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347150.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/c03/04db923c2025be36f3b7a036af418c03/04db923c2025be36f3b7a036af418c03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347154.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/85d/ad9ecb4095b340274fb6e3d1369d585d/ad9ecb4095b340274fb6e3d1369d585d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347154.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/85d/ad9ecb4095b340274fb6e3d1369d585d/ad9ecb4095b340274fb6e3d1369d585d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347158.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0d4/c96ba56927f32dad541b7a9a32ab20d4/c96ba56927f32dad541b7a9a32ab20d4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347158.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0d4/c96ba56927f32dad541b7a9a32ab20d4/c96ba56927f32dad541b7a9a32ab20d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347162.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/ac5/3400da77a1fd8a51a45c043b42582ac5/3400da77a1fd8a51a45c043b42582ac5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347162.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/ac5/3400da77a1fd8a51a45c043b42582ac5/3400da77a1fd8a51a45c043b42582ac5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347166.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/89b/25a5febe83c1248af555fc682ec0089b/25a5febe83c1248af555fc682ec0089b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347166.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/89b/25a5febe83c1248af555fc682ec0089b/25a5febe83c1248af555fc682ec0089b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347170.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/762/ddbe2034c707dadd06050cba50eda762/ddbe2034c707dadd06050cba50eda762.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347170.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/762/ddbe2034c707dadd06050cba50eda762/ddbe2034c707dadd06050cba50eda762.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347174.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/78d/d1b7305c5b0ecd322f04950bfa3d678d/d1b7305c5b0ecd322f04950bfa3d678d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347174.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/78d/d1b7305c5b0ecd322f04950bfa3d678d/d1b7305c5b0ecd322f04950bfa3d678d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347178.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3c1/ee04a4813833ab2321cf4d781f72c3c1/ee04a4813833ab2321cf4d781f72c3c1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347178.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3c1/ee04a4813833ab2321cf4d781f72c3c1/ee04a4813833ab2321cf4d781f72c3c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347182.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/8ba/8c56235d469e046d744d758170a7d8ba/8c56235d469e046d744d758170a7d8ba.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347182.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/8ba/8c56235d469e046d744d758170a7d8ba/8c56235d469e046d744d758170a7d8ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347186.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/b68/d1cf8a60f7bf9b0415f9d61e896b3b68/d1cf8a60f7bf9b0415f9d61e896b3b68.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347186.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/b68/d1cf8a60f7bf9b0415f9d61e896b3b68/d1cf8a60f7bf9b0415f9d61e896b3b68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347190.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/1b6/a8ee6c196fa08d7fce5891617d1111b6/a8ee6c196fa08d7fce5891617d1111b6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347190.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/1b6/a8ee6c196fa08d7fce5891617d1111b6/a8ee6c196fa08d7fce5891617d1111b6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347194.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/780/1baf663e7054afa9f87d6837bc0e5780/1baf663e7054afa9f87d6837bc0e5780.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347194.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/780/1baf663e7054afa9f87d6837bc0e5780/1baf663e7054afa9f87d6837bc0e5780.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347198.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/f79/be145ba856acc07f2de1e7e6d684ff79/be145ba856acc07f2de1e7e6d684ff79.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347198.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/f79/be145ba856acc07f2de1e7e6d684ff79/be145ba856acc07f2de1e7e6d684ff79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347214.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/10c/fe5158f0241342f968eedb7093cce10c/fe5158f0241342f968eedb7093cce10c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347214.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/10c/fe5158f0241342f968eedb7093cce10c/fe5158f0241342f968eedb7093cce10c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347218.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/9f9/924ae5b5caa5d47274402508bb2739f9/924ae5b5caa5d47274402508bb2739f9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347218.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/9f9/924ae5b5caa5d47274402508bb2739f9/924ae5b5caa5d47274402508bb2739f9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347222.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/482/3bc165f881aedbd2b423553496b14482/3bc165f881aedbd2b423553496b14482.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347222.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/482/3bc165f881aedbd2b423553496b14482/3bc165f881aedbd2b423553496b14482.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347226.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3b9/dfe54dbba6ff5f1625c32d18336dc3b9/dfe54dbba6ff5f1625c32d18336dc3b9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347226.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3b9/dfe54dbba6ff5f1625c32d18336dc3b9/dfe54dbba6ff5f1625c32d18336dc3b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347230.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/1c7/3b2d65c390556dda28f85f1ec1e251c7/3b2d65c390556dda28f85f1ec1e251c7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347230.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/1c7/3b2d65c390556dda28f85f1ec1e251c7/3b2d65c390556dda28f85f1ec1e251c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347234.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/c1f/f39933cb0b09682ee8bb0554640c8c1f/f39933cb0b09682ee8bb0554640c8c1f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347234.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/c1f/f39933cb0b09682ee8bb0554640c8c1f/f39933cb0b09682ee8bb0554640c8c1f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347238.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/945/4b63c47fdc719585ec74a6c6532e4945/4b63c47fdc719585ec74a6c6532e4945.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347238.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/945/4b63c47fdc719585ec74a6c6532e4945/4b63c47fdc719585ec74a6c6532e4945.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347242.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/7b2/459391a02a7ee49e4d9208825c6937b2/459391a02a7ee49e4d9208825c6937b2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347242.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/7b2/459391a02a7ee49e4d9208825c6937b2/459391a02a7ee49e4d9208825c6937b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347250.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/852/ccb5eca81c56d6174ab863236bab5852/ccb5eca81c56d6174ab863236bab5852.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347250.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/852/ccb5eca81c56d6174ab863236bab5852/ccb5eca81c56d6174ab863236bab5852.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347254.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/897/8f762330af74b828c6061ca84d5d5897/8f762330af74b828c6061ca84d5d5897.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347254.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/897/8f762330af74b828c6061ca84d5d5897/8f762330af74b828c6061ca84d5d5897.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347258.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/432/69acb38df4c85b8ea722733317ccb432/69acb38df4c85b8ea722733317ccb432.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347258.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/432/69acb38df4c85b8ea722733317ccb432/69acb38df4c85b8ea722733317ccb432.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347262.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/78b/57f3f920ca99ad5493b9f9103fd1b78b/57f3f920ca99ad5493b9f9103fd1b78b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347262.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/78b/57f3f920ca99ad5493b9f9103fd1b78b/57f3f920ca99ad5493b9f9103fd1b78b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347266.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/41e/92f67ef3444669c841ee16f4c1f1741e/92f67ef3444669c841ee16f4c1f1741e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347266.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/41e/92f67ef3444669c841ee16f4c1f1741e/92f67ef3444669c841ee16f4c1f1741e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347270.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/27c/57e1f02445fb7f62caa6b9f66d43a27c/57e1f02445fb7f62caa6b9f66d43a27c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347270.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/27c/57e1f02445fb7f62caa6b9f66d43a27c/57e1f02445fb7f62caa6b9f66d43a27c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347274.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/bc6/72fb366a25ab94867d6e9676b0ed4bc6/72fb366a25ab94867d6e9676b0ed4bc6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347274.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/bc6/72fb366a25ab94867d6e9676b0ed4bc6/72fb366a25ab94867d6e9676b0ed4bc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347278.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/d74/c237522584b79ae55117fdec5d757d74/c237522584b79ae55117fdec5d757d74.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347278.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/d74/c237522584b79ae55117fdec5d757d74/c237522584b79ae55117fdec5d757d74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347282.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0b8/cc1af7f7dbec7aae69232417e0b890b8/cc1af7f7dbec7aae69232417e0b890b8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347282.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0b8/cc1af7f7dbec7aae69232417e0b890b8/cc1af7f7dbec7aae69232417e0b890b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347286.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/93f/63029f619cf6f041b5be15301935993f/63029f619cf6f041b5be15301935993f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347286.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/93f/63029f619cf6f041b5be15301935993f/63029f619cf6f041b5be15301935993f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347290.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/330/c03db3830a94cac07f4c957d75692330/c03db3830a94cac07f4c957d75692330.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347290.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/330/c03db3830a94cac07f4c957d75692330/c03db3830a94cac07f4c957d75692330.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347294.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/fad/c314653a706a109dc59e420aa317efad/c314653a706a109dc59e420aa317efad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347294.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/fad/c314653a706a109dc59e420aa317efad/c314653a706a109dc59e420aa317efad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347298.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/770/93621c0641a9e0406a4925b6b59cd770/93621c0641a9e0406a4925b6b59cd770.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347298.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/770/93621c0641a9e0406a4925b6b59cd770/93621c0641a9e0406a4925b6b59cd770.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347302.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/cd5/813fa967e4f64ce17a5cef4cb41d5cd5/813fa967e4f64ce17a5cef4cb41d5cd5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347302.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/cd5/813fa967e4f64ce17a5cef4cb41d5cd5/813fa967e4f64ce17a5cef4cb41d5cd5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347310.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/66c/8948d8418d07200876a12a2bb423d66c/8948d8418d07200876a12a2bb423d66c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347310.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/66c/8948d8418d07200876a12a2bb423d66c/8948d8418d07200876a12a2bb423d66c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347314.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/4c7/45a5c321bb843cc7848fd6459de774c7/45a5c321bb843cc7848fd6459de774c7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347314.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/4c7/45a5c321bb843cc7848fd6459de774c7/45a5c321bb843cc7848fd6459de774c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347318.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/ae7/b62bf68e010614da9cda9b4ff25bcae7/b62bf68e010614da9cda9b4ff25bcae7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347318.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/ae7/b62bf68e010614da9cda9b4ff25bcae7/b62bf68e010614da9cda9b4ff25bcae7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347322.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/d1e/cb37ebc4efa9423123ee96f0302afd1e/cb37ebc4efa9423123ee96f0302afd1e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347322.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/d1e/cb37ebc4efa9423123ee96f0302afd1e/cb37ebc4efa9423123ee96f0302afd1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347326.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/8c5/7a643b855bccc4cf3c4eaeda83fcb8c5/7a643b855bccc4cf3c4eaeda83fcb8c5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347326.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/8c5/7a643b855bccc4cf3c4eaeda83fcb8c5/7a643b855bccc4cf3c4eaeda83fcb8c5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347330.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3c4/81a59d65052eb0ce081497b49f03e3c4/81a59d65052eb0ce081497b49f03e3c4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347330.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3c4/81a59d65052eb0ce081497b49f03e3c4/81a59d65052eb0ce081497b49f03e3c4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347334.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3a4/91378dc420b4ccedbfe24ff2d21a83a4/91378dc420b4ccedbfe24ff2d21a83a4.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('1e1c336e-4fcf-4690-9b46-e50fd924278e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.78630 seconds (1.27177/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpjolpf8av/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ea5f18aaa16e157a9562a13022af9aca', '90ed9f1d-2d91-429e-8e12-56e1e1049e3a-sda', '1717347159.76863', '1717347162', '1717347161', '', '1717347162', 1, 1) test DEBUG: /tmp/tmpjolpf8av/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'ea5f18aaa16e157a9562a13022af9aca', '90ed9f1d-2d91-429e-8e12-56e1e1049e3a-sda', '1717347159.76863', '1717347162', '1717347161', '', '1717347162', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('90ed9f1d-2d91-429e-8e12-56e1e1049e3a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpjolpf8av/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12691 seconds (7.87965/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpjolpf8av/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'ef782ae70f279bc295b252b50a7a3086', '9b341077-2553-4175-a775-b95818c8f624-sda', '1717347159.94519', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9b341077-2553-4175-a775-b95818c8f624-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'ef782ae70f279bc295b252b50a7a3086', '9b341077-2553-4175-a775-b95818c8f624-sda', '1717347159.94519', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9b341077-2553-4175-a775-b95818c8f624-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'c6ebf885353706991524b8b03a887266', 'ee133af0-02ca-41f1-a32e-24e3a89290f6-sda', '1717347161.34660', '1717347161.00000', '0', '{"x-container-meta-test": ["foo", "1717347161.00000"]}', '1717347161.00000', 1, 0) test DEBUG: replicator-rpc-sync time for info: 0.02s test DEBUG: replicator-rpc-sync time for update_metadata: 0.01s test DEBUG: replicator-rpc-sync time for merge_timestamps: 0.02s test DEBUG: replicator-rpc-sync time for get_sync: 0.00s test DEBUG: replicator-rpc-sync time for merge_syncs: 0.02s REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347162.61212', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347162.61212', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61212', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347162.61219', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347162.61223', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347162.61223', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61223', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347162.66262', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0152879s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347162.61212', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347162.61212', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61212', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347162.61219', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347162.61223', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347162.61223', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61223', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347162.61212', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347162.61212', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61212', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 13, 'bytes_used': 123, 'meta_timestamp': '1717347162.76528', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347162.61223', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347162.61223', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61223', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347162.61212', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347162.61212', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61212', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347163.04802', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347162.61223', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347162.61223', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61223', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347163.07658', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347163.07658', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347163.07658', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347163.04802', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347162.61223', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347162.61223', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61223', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347163.07658', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347163.07658', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347163.07658', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347163.04802', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347163.11900', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 30, 'meta_timestamp': '1717347163.11900', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.11900', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.5:1005/sdf (txn: txf4cb86bdd05940ddb466c-00665ca352) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.2:1002/sdc (txn: tx46767a0bd02346f6b7f4a-00665ca352) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.5:1005/sdf (txn: tx2147edc5fea04af997633-00665ca352) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config ... ok proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "PUT /path HTTP/1.0" 200 105 0.000087 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET ///some-leading-slashes HTTP/1.0" 200 118 0.000061 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET /ohÿboy%what$now%E2%80%bd HTTP/1.0" 200 120 0.000049 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET /some%fFpath%D8%AA HTTP/1.0" 200 113 0.000049 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET / HTTP/1.1" 200 120 0.000088 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 44\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET /%FF HTTP/1.1" 200 123 0.000064 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 47\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "/%FF", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET /ÿ HTTP/1.1" 200 126 0.000085 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 50\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "/\\u00ff", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "PUT /Here%20Is%20A%20SnowMan:☃ HTTP/1.0" 200 181 0.000061 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 86\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\nConnection: close\r\n\r\n{"RAW_PATH_INFO": "/Here%20Is%20A%20SnowMan:\\u00e2\\u0098\\u0083", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "POST /?and%20it=does+nothing+to+params&PALMTREE=ð%9fŒ%b4 HTTP/1.1" 200 178 0.000068 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 101\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": "and%20it=does+nothing+to+params&PALMTREE=\\u00f0%9f\\u008c%b4"}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET // HTTP/1.1" 200 121 0.000063 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 45\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "//", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET //bar HTTP/1.1" 200 124 0.000058 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 48\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "//bar", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET //////baz HTTP/1.1" 200 128 0.000063 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 52\r\nDate: Sun, 02 Jun 2024 16:51:57 GMT\r\n\r\n{"RAW_PATH_INFO": "//////baz", "QUERY_STRING": null}' proxy INFO: (3610488) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3610488) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:51:57] "GET /someurl HTTP/1.0" 200 102 0.000053 proxy INFO: wsgi exiting proxy INFO: (3610488) wsgi exited, is_accepting=True test ERROR: Problem finding shard upper in '/tmp/tmp16s9uvmp/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp16s9uvmp/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmp3qj6tops/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmp3qj6tops/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable ... ok test ERROR: Failed to unlink '/tmp/tmpr55zo4ud/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpr55zo4ud/containers/part/suffix/hash/container_1717347171.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpr55zo4ud/containers/part/suffix/hash/container_1717347171.00000.db': ['/tmp/tmpr55zo4ud/containers/part/suffix/hash/container.db', '/tmp/tmpr55zo4ud/containers/part/suffix/hash/container_1717347172.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpr55zo4ud/containers/part/suffix/hash/container_1717347172.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Skipping the update of recently changed /tmp/tmpfzg1w8ik/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmpfzg1w8ik/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmpfzg1w8ik/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmpfzg1w8ik/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping: /tmp/tmpt0e54wh4/garbage is not a directory test DEBUG: Skipping: /tmp/tmpt0e54wh4/garbage is not mounted test DEBUG: Skipping: /tmp/tmpdfuc7ce1/sdq is not mounted test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (10.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3610512] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3610512] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/2 pid=pid-1] Starting object reconstructor in daemon mode. test INFO: [worker 1/2 pid=pid-1] Starting object reconstruction pass. test INFO: [worker 1/2 pid=pid-1] Object reconstruction complete. (0.10 minutes) test DEBUG: [worker 1/2 pid=pid-1] reconstruction sleeping for 30.0 seconds. test INFO: [worker 2/4 pid=20641] Running object reconstructor in script mode. test DEBUG: [worker 2/4 pid=20641] debug message test INFO: [worker 2/4 pid=20641] info message test WARNING: [worker 2/4 pid=20641] warning message test ERROR: [worker 2/4 pid=20641] error message test INFO: [worker 2/4 pid=20641] Object reconstruction complete (once). (0.00 minutes) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx8c83b6a3ce4b4af996ae6-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbac4b6bb221342cb94ebd-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx3f3e4cbe6b6a4a3da1f03-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd0869cf3ea02472994f56-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx3b309809cbb94c00b5e81-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx46a84bc91e6b4f3084cb0-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx1baa53a586174194a4e17-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txee95b88c4afa4ae684587-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txdfd856e5a30444b9ab23c-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txcacb6974045646fda2cc5-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txb6e3c3de1113433e89776-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx4b253f61c05145cebe739-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx087accff8f744c5294a8b-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx1236c00137e440ea86602-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx811e02b9ad2a4c4da1053-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe4a1ceb9952149cf9fc41-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx9bc6ad436b0a4606836d7-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx46d60fa0a1314a048aeb1-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx70090c84f1d947f7a5071-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txea84712432b045ddb7237-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx34e2a9abd875458a8c99b-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx8169517810b24dcebdcd0-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx55d3bbd340d44059a58b5-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txb8d6d21cf0224d22a1bc2-00665ca364) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database ... ok proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmpkbtl3sjz/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpkbtl3sjz/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmpkbtl3sjz/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpkbtl3sjz/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0223 "-" 3610484 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0163 "-" 3610484 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0336 "-" 3610484 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0155 "-" 3610484 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txcd4c4fc0d4504282a0ab5-00665ca356" "proxy-server 3610484" 0.0026 "-" 3610484 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:43537/sda1/3/a/c" "txcd4c4fc0d4504282a0ab5-00665ca356" "container-server 3610484" 0.0019 "-" 3610484 0 test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: tx4720576ad2ad468f93f17-00665ca364) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: tx419bd8e1f1684ea584447-00665ca364) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txdb4fe4a37c3f4fd8b72fe-00665ca364) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txe7b45fc9ae2d4c93860ee-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx83916ebe761c4898920cc-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: tx14ef716edb74404e825c8-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx714b34b0a4a24b89ab486-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: txf90cc9f3a759451391fcb-00665ca364) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: txea2e67359371457ea7f75-00665ca364) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx41c30320b23d43ba82d68-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe444c04fd2744cc69f5a7-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx55a863180d6f4a31bbbec-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx78033d660dd541479271c-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx06fcd576ddca45f8af198-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx75a9e9fbc9b54e549b243-00665ca364) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: tx4128903950d640758c098-00665ca364) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx4128903950d640758c098-00665ca364) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag 'a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:38 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:46233/sdb1/3/a/c" "txcd4c4fc0d4504282a0ab5-00665ca356" "container-server 3610484" 0.0020 "-" 3610484 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txcd4c4fc0d4504282a0ab5-00665ca356" "proxy-server 3610484" 3.1671 "-" 3610484 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txcd4c4fc0d4504282a0ab5-00665ca356" "proxy-server 3610484" 3.1520 "-" 3610484 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/42 PUT /v1/a/c HTTP/1.0 201 - - t - - - txcd4c4fc0d4504282a0ab5-00665ca356 - 3.1838 - - 1717347158.867418528 1717347162.051206350 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txee9a8615ad044a4383b46-00665ca35a" "proxy-server 3610484" 0.0018 "-" 3610484 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:43537/sda1/0/a1/c1" "txee9a8615ad044a4383b46-00665ca35a" "container-server 3610484" 0.0014 "-" 3610484 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:46233/sdb1/0/a1/c1" "txee9a8615ad044a4383b46-00665ca35a" "container-server 3610484" 0.0013 "-" 3610484 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txee9a8615ad044a4383b46-00665ca35a" "proxy-server 3610484" 0.3919 "-" 3610484 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txee9a8615ad044a4383b46-00665ca35a" "proxy-server 3610484" 0.0314 "-" 3610484 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/42 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txee9a8615ad044a4383b46-00665ca35a - 0.4039 - - 1717347162.053383827 1717347162.457263708 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa9edef50c51344d990d7a-00665ca35a" "proxy-server 3610484" 0.0080 "-" 3610484 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:46233/sdb1/3/a/c1" "txa9edef50c51344d990d7a-00665ca35a" "container-server 3610484" 0.0014 "-" 3610484 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:43537/sda1/3/a/c1" "txa9edef50c51344d990d7a-00665ca35a" "container-server 3610484" 0.0106 "-" 3610484 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa9edef50c51344d990d7a-00665ca35a" "proxy-server 3610484" 0.0461 "-" 3610484 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa9edef50c51344d990d7a-00665ca35a" "proxy-server 3610484" 0.0329 "-" 3610484 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/42 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txa9edef50c51344d990d7a-00665ca35a - 0.0640 - - 1717347162.462424278 1717347162.526398897 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx975887e70a4c4c8c943e6-00665ca35a" "proxy-server 3610484" 0.0118 "-" 3610484 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:46233/sdb1/0/a/c2" "tx975887e70a4c4c8c943e6-00665ca35a" "container-server 3610484" 0.0083 "-" 3610484 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:43537/sda1/0/a/c2" "tx975887e70a4c4c8c943e6-00665ca35a" "container-server 3610484" 0.0015 "-" 3610484 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx975887e70a4c4c8c943e6-00665ca35a" "proxy-server 3610484" 0.0480 "-" 3610484 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:52:42 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx975887e70a4c4c8c943e6-00665ca35a" "proxy-server 3610484" 0.0349 "-" 3610484 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/52/42 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx975887e70a4c4c8c943e6-00665ca35a - 0.0700 - - 1717347162.528626204 1717347162.598625183 2 test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) test ERROR: Failed to unlink '/tmp/tmp5yoht7qn/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp5yoht7qn/containers/part/suffix/hash/container_1717347173.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp5yoht7qn/containers/part/suffix/hash/container_1717347173.00000.db': ['/tmp/tmp5yoht7qn/containers/part/suffix/hash/container.db', '/tmp/tmp5yoht7qn/containers/part/suffix/hash/container_1717347174.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp5yoht7qn/containers/part/suffix/hash/container_1717347174.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txb7c1e4ea7dae469a922d3-00665ca366) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txb7c1e4ea7dae469a922d3-00665ca366) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txb7c1e4ea7dae469a922d3-00665ca366) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: txb7c1e4ea7dae469a922d3-00665ca366) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txb7c1e4ea7dae469a922d3-00665ca366) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txb7c1e4ea7dae469a922d3-00665ca366) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txb7c1e4ea7dae469a922d3-00665ca366) test INFO: Client disconnected on read of '/a/c' (txn: txb7c1e4ea7dae469a922d3-00665ca366) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx087686df309d4875a3253-00665ca366) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx087686df309d4875a3253-00665ca366) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx087686df309d4875a3253-00665ca366) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: tx31b80ba2f0e145849e932-00665ca366) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx2e94948666fd4eddadfb4-00665ca366) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: txd9f10a633be3429ab34d3-00665ca366) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: tx315b55dcc86a4ca1863c0-00665ca366) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txaacb6d77f9414b6289d3b-00665ca366) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txc67e92ec142249128b520-00665ca366) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txc67e92ec142249128b520-00665ca366) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: txf06e42f91cf2405b9fe1b-00665ca366) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: txa8eba5c7eaa942f993899-00665ca366) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: txc808e37162254b5d92f4f-00665ca366) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: txfd2bbdedc0b648b284123-00665ca366) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx71776226c9f64af7a3af2-00665ca366) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txbb6f384ab70647dcada70-00665ca367) test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347334.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/3a4/91378dc420b4ccedbfe24ff2d21a83a4/91378dc420b4ccedbfe24ff2d21a83a4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347338.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0a8/4579f4ac8764f36d4f642693c18fa0a8/4579f4ac8764f36d4f642693c18fa0a8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347338.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0a8/4579f4ac8764f36d4f642693c18fa0a8/4579f4ac8764f36d4f642693c18fa0a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347342.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/be8/d1e6d63bc37b0bf4de0ccee169f6dbe8/d1e6d63bc37b0bf4de0ccee169f6dbe8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347342.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/be8/d1e6d63bc37b0bf4de0ccee169f6dbe8/d1e6d63bc37b0bf4de0ccee169f6dbe8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347346.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/042/b36421fbfb3a8e851264ca2526d55042/b36421fbfb3a8e851264ca2526d55042.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347346.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/042/b36421fbfb3a8e851264ca2526d55042/b36421fbfb3a8e851264ca2526d55042.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347350.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/c21/ab2aca59c6e2e7b749ff85496c9bec21/ab2aca59c6e2e7b749ff85496c9bec21.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347350.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/c21/ab2aca59c6e2e7b749ff85496c9bec21/ab2aca59c6e2e7b749ff85496c9bec21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347354.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/aad/cbe8b3a39a5315ecce77f5e039d40aad/cbe8b3a39a5315ecce77f5e039d40aad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347354.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/aad/cbe8b3a39a5315ecce77f5e039d40aad/cbe8b3a39a5315ecce77f5e039d40aad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347358.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/e84/e2fa085ec713f27e6d311a061d6a9e84/e2fa085ec713f27e6d311a061d6a9e84.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347358.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/e84/e2fa085ec713f27e6d311a061d6a9e84/e2fa085ec713f27e6d311a061d6a9e84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347362.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/6cb/770ff6d23af59839cd6b5c9add4026cb/770ff6d23af59839cd6b5c9add4026cb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347362.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/6cb/770ff6d23af59839cd6b5c9add4026cb/770ff6d23af59839cd6b5c9add4026cb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347366.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/e53/561c335c709174eba02b2bdfe989fe53/561c335c709174eba02b2bdfe989fe53.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347366.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/e53/561c335c709174eba02b2bdfe989fe53/561c335c709174eba02b2bdfe989fe53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347370.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/53c/df9967c822c0d3f28e5f106708ed853c/df9967c822c0d3f28e5f106708ed853c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347370.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/53c/df9967c822c0d3f28e5f106708ed853c/df9967c822c0d3f28e5f106708ed853c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347374.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/41a/13009b72bafaacdd9c8f3f3ca369041a/13009b72bafaacdd9c8f3f3ca369041a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347374.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/41a/13009b72bafaacdd9c8f3f3ca369041a/13009b72bafaacdd9c8f3f3ca369041a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347378.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0b2/357e1ad5168c6bd03a2cc7db5f1c20b2/357e1ad5168c6bd03a2cc7db5f1c20b2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347378.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0b2/357e1ad5168c6bd03a2cc7db5f1c20b2/357e1ad5168c6bd03a2cc7db5f1c20b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347382.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0e7/827f441d78c25cbc2413aa2028e3e0e7/827f441d78c25cbc2413aa2028e3e0e7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347382.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/0e7/827f441d78c25cbc2413aa2028e3e0e7/827f441d78c25cbc2413aa2028e3e0e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347386.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/5ca/e95de7b9d6e28529830faa640d79a5ca/e95de7b9d6e28529830faa640d79a5ca.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347386.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/5ca/e95de7b9d6e28529830faa640d79a5ca/e95de7b9d6e28529830faa640d79a5ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347390.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/b55/00450b2ff39880cefc212a2275a74b55/00450b2ff39880cefc212a2275a74b55.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347390.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/b55/00450b2ff39880cefc212a2275a74b55/00450b2ff39880cefc212a2275a74b55.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347394.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/5a1/7669ad17c6b066c176c34deacf7b85a1/7669ad17c6b066c176c34deacf7b85a1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347394.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/5a1/7669ad17c6b066c176c34deacf7b85a1/7669ad17c6b066c176c34deacf7b85a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347398.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/26e/160b797ffcc0d82d7d026c723764926e/160b797ffcc0d82d7d026c723764926e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347398.00000-0, db: /tmp/tmpfpiu_fol/sda/containers/0/26e/160b797ffcc0d82d7d026c723764926e/160b797ffcc0d82d7d026c723764926e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347157.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7b4/1852ad1bde74009fc5ec62fd035f07b4/1852ad1bde74009fc5ec62fd035f07b4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347157.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7b4/1852ad1bde74009fc5ec62fd035f07b4/1852ad1bde74009fc5ec62fd035f07b4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347160.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/c30/6afefcb559d21e780fffa5567d9f2c30/6afefcb559d21e780fffa5567d9f2c30.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347160.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/c30/6afefcb559d21e780fffa5567d9f2c30/6afefcb559d21e780fffa5567d9f2c30.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347163.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/db9/43059fb7473ac8e252c6f0ddb14a8db9/43059fb7473ac8e252c6f0ddb14a8db9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347163.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/db9/43059fb7473ac8e252c6f0ddb14a8db9/43059fb7473ac8e252c6f0ddb14a8db9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347166.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/89b/25a5febe83c1248af555fc682ec0089b/25a5febe83c1248af555fc682ec0089b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347166.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/89b/25a5febe83c1248af555fc682ec0089b/25a5febe83c1248af555fc682ec0089b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347169.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/2d8/3eaedbc9c7174dc0a3089f3cd8b402d8/3eaedbc9c7174dc0a3089f3cd8b402d8.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347169.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/2d8/3eaedbc9c7174dc0a3089f3cd8b402d8/3eaedbc9c7174dc0a3089f3cd8b402d8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347172.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/4f0/a897279f1a117184e57901c9444f54f0/a897279f1a117184e57901c9444f54f0.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347172.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/4f0/a897279f1a117184e57901c9444f54f0/a897279f1a117184e57901c9444f54f0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347175.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/3f7/01a809bcace06f5996189d29867c13f7/01a809bcace06f5996189d29867c13f7.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347175.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/3f7/01a809bcace06f5996189d29867c13f7/01a809bcace06f5996189d29867c13f7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347178.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/3c1/ee04a4813833ab2321cf4d781f72c3c1/ee04a4813833ab2321cf4d781f72c3c1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347178.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/3c1/ee04a4813833ab2321cf4d781f72c3c1/ee04a4813833ab2321cf4d781f72c3c1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347181.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/71e/f3f9546e6bd95c47724d04d854a8071e/f3f9546e6bd95c47724d04d854a8071e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347181.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/71e/f3f9546e6bd95c47724d04d854a8071e/f3f9546e6bd95c47724d04d854a8071e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347184.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/e03/4a04f7b013c2437a9d912b67daa69e03/4a04f7b013c2437a9d912b67daa69e03.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347184.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/e03/4a04f7b013c2437a9d912b67daa69e03/4a04f7b013c2437a9d912b67daa69e03.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347187.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/a54/1f5028289f0b07234737183d74307a54/1f5028289f0b07234737183d74307a54.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347187.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/a54/1f5028289f0b07234737183d74307a54/1f5028289f0b07234737183d74307a54.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347190.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/1b6/a8ee6c196fa08d7fce5891617d1111b6/a8ee6c196fa08d7fce5891617d1111b6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347190.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/1b6/a8ee6c196fa08d7fce5891617d1111b6/a8ee6c196fa08d7fce5891617d1111b6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347193.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/a14/252e9eaf90549575df4d01c22fad3a14/252e9eaf90549575df4d01c22fad3a14.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347193.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/a14/252e9eaf90549575df4d01c22fad3a14/252e9eaf90549575df4d01c22fad3a14.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347196.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/af5/2722d14adf6006e454f1bd1f56ae9af5/2722d14adf6006e454f1bd1f56ae9af5.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347196.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/af5/2722d14adf6006e454f1bd1f56ae9af5/2722d14adf6006e454f1bd1f56ae9af5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347205.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/819/4303aa522a892bb737933448b0edd819/4303aa522a892bb737933448b0edd819.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347205.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/819/4303aa522a892bb737933448b0edd819/4303aa522a892bb737933448b0edd819.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347208.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/030/987dbb3f91948969290126a3a4e19030/987dbb3f91948969290126a3a4e19030.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347208.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/030/987dbb3f91948969290126a3a4e19030/987dbb3f91948969290126a3a4e19030.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347214.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/10c/fe5158f0241342f968eedb7093cce10c/fe5158f0241342f968eedb7093cce10c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347214.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/10c/fe5158f0241342f968eedb7093cce10c/fe5158f0241342f968eedb7093cce10c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347217.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7ed/c1f4da3a9decd47df845c3bfa99f37ed/c1f4da3a9decd47df845c3bfa99f37ed.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347217.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7ed/c1f4da3a9decd47df845c3bfa99f37ed/c1f4da3a9decd47df845c3bfa99f37ed.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347220.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/71e/6de3f330f9e8d84055274ee652dbf71e/6de3f330f9e8d84055274ee652dbf71e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347220.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/71e/6de3f330f9e8d84055274ee652dbf71e/6de3f330f9e8d84055274ee652dbf71e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347223.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/2e0/0c90cfa44b400db3516e9c9f9e4932e0/0c90cfa44b400db3516e9c9f9e4932e0.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347223.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/2e0/0c90cfa44b400db3516e9c9f9e4932e0/0c90cfa44b400db3516e9c9f9e4932e0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347226.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/3b9/dfe54dbba6ff5f1625c32d18336dc3b9/dfe54dbba6ff5f1625c32d18336dc3b9.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347226.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/3b9/dfe54dbba6ff5f1625c32d18336dc3b9/dfe54dbba6ff5f1625c32d18336dc3b9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347229.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/5b8/917d320f7d467dcd8762d7740a04e5b8/917d320f7d467dcd8762d7740a04e5b8.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347229.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/5b8/917d320f7d467dcd8762d7740a04e5b8/917d320f7d467dcd8762d7740a04e5b8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347232.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/23b/d4be284624ac64b24c804dab5c62b23b/d4be284624ac64b24c804dab5c62b23b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347232.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/23b/d4be284624ac64b24c804dab5c62b23b/d4be284624ac64b24c804dab5c62b23b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347235.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7dd/8577c4222d3f528dffdf240ed803d7dd/8577c4222d3f528dffdf240ed803d7dd.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347235.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7dd/8577c4222d3f528dffdf240ed803d7dd/8577c4222d3f528dffdf240ed803d7dd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347238.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/945/4b63c47fdc719585ec74a6c6532e4945/4b63c47fdc719585ec74a6c6532e4945.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347238.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/945/4b63c47fdc719585ec74a6c6532e4945/4b63c47fdc719585ec74a6c6532e4945.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347241.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/da5/468ece8cf53547f1b8a568b80793eda5/468ece8cf53547f1b8a568b80793eda5.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347241.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/da5/468ece8cf53547f1b8a568b80793eda5/468ece8cf53547f1b8a568b80793eda5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347244.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/896/3f022372527e997d8a5822f80d1bf896/3f022372527e997d8a5822f80d1bf896.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347244.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/896/3f022372527e997d8a5822f80d1bf896/3f022372527e997d8a5822f80d1bf896.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347247.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/f36/a6fdfe3b1e889bb8982a2e1b707d4f36/a6fdfe3b1e889bb8982a2e1b707d4f36.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347247.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/f36/a6fdfe3b1e889bb8982a2e1b707d4f36/a6fdfe3b1e889bb8982a2e1b707d4f36.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347250.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/852/ccb5eca81c56d6174ab863236bab5852/ccb5eca81c56d6174ab863236bab5852.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347250.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/852/ccb5eca81c56d6174ab863236bab5852/ccb5eca81c56d6174ab863236bab5852.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347253.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/de0/6627224e4a0739110fa30cce1463fde0/6627224e4a0739110fa30cce1463fde0.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347253.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/de0/6627224e4a0739110fa30cce1463fde0/6627224e4a0739110fa30cce1463fde0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347256.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/8a1/99e66f00d4f64cf385e7c517ec9a28a1/99e66f00d4f64cf385e7c517ec9a28a1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347256.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/8a1/99e66f00d4f64cf385e7c517ec9a28a1/99e66f00d4f64cf385e7c517ec9a28a1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347259.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/f31/22b3f04cfacc35be1397a6982ed8bf31/22b3f04cfacc35be1397a6982ed8bf31.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347259.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/f31/22b3f04cfacc35be1397a6982ed8bf31/22b3f04cfacc35be1397a6982ed8bf31.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347262.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/78b/57f3f920ca99ad5493b9f9103fd1b78b/57f3f920ca99ad5493b9f9103fd1b78b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347262.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/78b/57f3f920ca99ad5493b9f9103fd1b78b/57f3f920ca99ad5493b9f9103fd1b78b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347265.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/cea/d274ba3d519b0f756b04649a47cabcea/d274ba3d519b0f756b04649a47cabcea.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347265.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/cea/d274ba3d519b0f756b04649a47cabcea/d274ba3d519b0f756b04649a47cabcea.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347268.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/a31/63588ba1a47a5d32808e745e2f5f1a31/63588ba1a47a5d32808e745e2f5f1a31.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347268.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/a31/63588ba1a47a5d32808e745e2f5f1a31/63588ba1a47a5d32808e745e2f5f1a31.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347271.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/d48/ca9cc2335300ff7749b4d62fa2484d48/ca9cc2335300ff7749b4d62fa2484d48.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347271.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/d48/ca9cc2335300ff7749b4d62fa2484d48/ca9cc2335300ff7749b4d62fa2484d48.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347274.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/bc6/72fb366a25ab94867d6e9676b0ed4bc6/72fb366a25ab94867d6e9676b0ed4bc6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347274.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/bc6/72fb366a25ab94867d6e9676b0ed4bc6/72fb366a25ab94867d6e9676b0ed4bc6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347277.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/bb4/a31f4140018d58cd6c543245e5b34bb4/a31f4140018d58cd6c543245e5b34bb4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347277.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/bb4/a31f4140018d58cd6c543245e5b34bb4/a31f4140018d58cd6c543245e5b34bb4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347280.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/e9f/ade24454704b1adac4bcd7f8f7c35e9f/ade24454704b1adac4bcd7f8f7c35e9f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347280.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/e9f/ade24454704b1adac4bcd7f8f7c35e9f/ade24454704b1adac4bcd7f8f7c35e9f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347283.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/0a2/1d8ade84b040826192a108cb3d9f20a2/1d8ade84b040826192a108cb3d9f20a2.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347283.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/0a2/1d8ade84b040826192a108cb3d9f20a2/1d8ade84b040826192a108cb3d9f20a2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347286.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/93f/63029f619cf6f041b5be15301935993f/63029f619cf6f041b5be15301935993f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347286.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/93f/63029f619cf6f041b5be15301935993f/63029f619cf6f041b5be15301935993f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347289.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/259/51f1805f017f47f1a26ca5304fd5f259/51f1805f017f47f1a26ca5304fd5f259.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347289.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/259/51f1805f017f47f1a26ca5304fd5f259/51f1805f017f47f1a26ca5304fd5f259.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347292.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/ac4/7468d55d914d6a24804619a20fa04ac4/7468d55d914d6a24804619a20fa04ac4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347292.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/ac4/7468d55d914d6a24804619a20fa04ac4/7468d55d914d6a24804619a20fa04ac4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347295.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/b04/a52ff746370192c0b7092fb39e323b04/a52ff746370192c0b7092fb39e323b04.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347295.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/b04/a52ff746370192c0b7092fb39e323b04/a52ff746370192c0b7092fb39e323b04.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347298.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/770/93621c0641a9e0406a4925b6b59cd770/93621c0641a9e0406a4925b6b59cd770.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347298.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/770/93621c0641a9e0406a4925b6b59cd770/93621c0641a9e0406a4925b6b59cd770.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347163.07658', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347163.07658', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347163.07658', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347163.04802', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347163.11900', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717347163.15696', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.11900', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347163.18553', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347163.07658', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.07658', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347163.04802', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347163.11900', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717347163.15696', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.11900', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpyomrx81q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd252d225ff80eaea845bbb2fd416c7ac', 'd9569836-f4cb-41e2-916e-725f1a575bac-sda', '1717347162.57621', '1717347162.57147', '0', '', '1717347162.57147', 1, 0) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347163.18553', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347163.07658', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.07658', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347162.61219', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347163.04802', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347162.61219', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347163.11900', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717347163.15696', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.11900', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347162.75919', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347162.75919', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347162.75907', 'epoch': '1717347162.75907', 'reported': 0, 'tombstones': -1}], 'd9569836-f4cb-41e2-916e-725f1a575bac-sda') test DEBUG: /tmp/tmpyomrx81q/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.0:1000/sda test WARNING: /tmp/tmpyomrx81q/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.0:1000/sda; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '049eb7ecad367c6e1621817fdf1d99e2', '5507b87c-96d1-4ff5-b0b1-1cf14782d719-sda', '1717347163.24267', '1717347163.23902', '0', '', '1717347163.23902', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': 'a/c', 'timestamp': '1717347163.26134', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.26134', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347163.26115', 'epoch': '1717347163.26115', 'reported': 0, 'tombstones': -1}], '5507b87c-96d1-4ff5-b0b1-1cf14782d719-sda') test ERROR: ERROR Bad response 500 from 10.0.0.1 test DEBUG: /tmp/tmp4o71yfsh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347163.26079', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '5507b87c-96d1-4ff5-b0b1-1cf14782d719-sda') test DEBUG: /tmp/tmp4o71yfsh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0163405s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e39f0d14-7b04-4470-b99a-87291af1f0ea-sda', '1717347163.35408', '1717347163.35015', '0', '', '1717347163.35015', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347163.37676', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347163.37676', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.37676', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347163.37681', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347163.37681', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.37681', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347163.37684', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347163.37684', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347163.37684', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'e39f0d14-7b04-4470-b99a-87291af1f0ea-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '3c6b8a57-86ab-437e-ab60-3bdf777f54cb-sda', '1717347164.15421', '1717347164.14991', '0', '', '1717347164.14991', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347164.17566', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347164.17566', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.17566', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347164.17573', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347164.17573', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.17573', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347164.17576', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347164.17576', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.17576', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34978', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34978', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34978', 'epoch': None, 'reported': 0, 'tombstones': -1}], '3c6b8a57-86ab-437e-ab60-3bdf777f54cb-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fb7da0b3-d8ed-4949-a092-a5d0ee6cf6db-sda', '1717347164.23756', '1717347164.23434', '0', '', '1717347164.23434', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347164.25828', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347164.25828', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.25828', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347164.25834', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347164.25834', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.25834', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347164.25837', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347164.25837', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.25837', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34978', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34978', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34978', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'fb7da0b3-d8ed-4949-a092-a5d0ee6cf6db-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '16ded641-8d1a-4540-a5d2-2387e0f58e2d-sda', '1717347164.33320', '1717347164.32687', '0', '', '1717347164.32687', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347164.35950', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347164.35950', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.35950', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347164.35957', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347164.35957', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.35957', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347164.35960', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347164.35960', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.35960', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34978', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34978', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34978', 'epoch': None, 'reported': 0, 'tombstones': -1}], '16ded641-8d1a-4540-a5d2-2387e0f58e2d-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '681f4149-1289-4ae2-b7be-45cbd26d9b08-sda', '1717347164.43255', '1717347164.42620', '0', '', '1717347164.42620', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347164.45925', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347164.45925', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.45925', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347164.45932', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347164.45932', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.45932', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347164.45935', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347164.45935', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.45935', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34978', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34978', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34978', 'epoch': None, 'reported': 0, 'tombstones': -1}], '681f4149-1289-4ae2-b7be-45cbd26d9b08-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0923d18c-b858-4d5f-82d2-6e4b98d7ff08-sda', '1717347164.52573', '1717347164.52039', '0', '', '1717347164.52039', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347164.55850', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347164.55850', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.55850', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347164.55857', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347164.55857', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.55857', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347164.55860', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347164.55860', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347164.55860', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34993', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34993', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34993', 'epoch': '1717347163.34996', 'reported': 0, 'tombstones': -1}], '0923d18c-b858-4d5f-82d2-6e4b98d7ff08-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0d45707d-71c8-4f8f-ba46-7a39f4d835e6-sda', '1717347165.46253', '1717347165.45642', '0', '', '1717347165.45642', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347165.48644', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347165.48644', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.48644', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347165.48650', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347165.48650', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.48650', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347165.48653', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347165.48653', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.48653', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34993', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34993', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34993', 'epoch': '1717347163.34996', 'reported': 0, 'tombstones': -1}], '0d45707d-71c8-4f8f-ba46-7a39f4d835e6-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c38fc7c8-9fc1-4cdd-b009-8959bf8ed760-sda', '1717347165.55735', '1717347165.55314', '0', '', '1717347165.55314', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347165.58433', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347165.58433', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.58433', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347165.58439', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347165.58439', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.58439', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347165.58442', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347165.58442', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.58442', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34993', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34993', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34993', 'epoch': '1717347163.34996', 'reported': 0, 'tombstones': -1}], 'c38fc7c8-9fc1-4cdd-b009-8959bf8ed760-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2bd9e4f7-8df9-4db3-9ad7-17b6fd781eba-sda', '1717347165.66349', '1717347165.65627', '0', '', '1717347165.65627', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347165.69092', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347165.69092', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.69092', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347165.69098', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347165.69098', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.69098', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347165.69101', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347165.69101', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.69101', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.35004', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.35004', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.35004', 'epoch': '1717347163.35006', 'reported': 0, 'tombstones': -1}], '2bd9e4f7-8df9-4db3-9ad7-17b6fd781eba-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a069cbee-d179-4df1-a6b0-34ffa34627cf-sda', '1717347165.77861', '1717347165.77291', '0', '', '1717347165.77291', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync ... ok test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txbb6f384ab70647dcada70-00665ca367) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: txa56be1d2acc94ef8b2a4e-00665ca367) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx94c039ba6f7743f5943d5-00665ca367) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: txfc998e2d2ad7452b8226e-00665ca367) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: tx3a72ab60bb7c46c6befc0-00665ca367) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx2d4b6e9f95c043f29729f-00665ca367) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txb45604187d2d402fb1f87-00665ca367) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txb45604187d2d402fb1f87-00665ca367) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx718faf1b97c043c686376-00665ca367) test INFO: Client disconnected on read of '/a/c' (txn: txca0eb059063a458a80a3b-00665ca367) test INFO: Client disconnected on read of '/a/c' (txn: txf4e8ad10614f4f07b6d0b-00665ca367) test INFO: Client disconnected on read of '/a/c' (txn: txf4e8ad10614f4f07b6d0b-00665ca367) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347165.79864', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347165.79864', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.79864', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347165.79869', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347165.79869', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.79869', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347165.79872', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347165.79872', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347165.79872', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34993', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34993', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34993', 'epoch': '1717347163.34996', 'reported': 0, 'tombstones': -1}], 'a069cbee-d179-4df1-a6b0-34ffa34627cf-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '042087ca-3257-4725-94b4-9b97dc7ed261-sda', '1717347167.71658', '1717347167.71315', '0', '', '1717347167.71315', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347168.41699', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347168.41699', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347168.41699', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347168.41706', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347168.41706', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347168.41706', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347168.41709', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347168.41709', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347168.41709', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.34993', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34993', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34993', 'epoch': '1717347163.34996', 'reported': 0, 'tombstones': -1}], '042087ca-3257-4725-94b4-9b97dc7ed261-sda') test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '65615966-f2ec-491b-9499-1ceee2d21dcd-sda', '1717347169.64245', '1717347169.63518', '0', '', '1717347169.63518', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347169.67640', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347169.67640', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347169.67640', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347169.67651', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347169.67651', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347169.67651', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347169.67654', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347169.67654', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347169.67654', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347163.35013', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34978', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34978', 'epoch': None, 'reported': 0, 'tombstones': -1}], '65615966-f2ec-491b-9499-1ceee2d21dcd-sda') test WARNING: Ignoring remote osr w/o epoch, own_sr: {'name': 'a/c', 'timestamp': '1717347163.34993', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34993', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34993', 'epoch': '1717347163.34996', 'reported': 0, 'tombstones': -1}, remote_sr: {'name': 'a/c', 'timestamp': '1717347163.35013', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347163.34978', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347163.34978', 'epoch': None, 'reported': 0, 'tombstones': -1}, source: repl_req test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmphe3qo_p7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'dbb9f024c4004bc7a28933f7dd7e82db', '7ead627b-e1db-4298-97ac-3293864439d3-sda', '1717347169.76118', '1717347169.75613', '0', '', '1717347169.75613', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347169.79625', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '7ead627b-e1db-4298-97ac-3293864439d3-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test DEBUG: Replicating db /tmp/tmpkzv7s6se/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '4f6c6b95-ca3f-44de-85d1-6e89ac49c03d-sda', '1717347169.87187', '1717347169.86265', '0', '', '1717347169.86265', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test ERROR: ERROR syncing /tmp/tmpkzv7s6se/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db with node {'ip': '10.0.0.1', 'replication_ip': '10.0.0.1', 'port': 1001, 'replication_port': 1001, 'device': 'sdb', 'zone': 1, 'region': 1, 'id': 1, 'weight': 1, 'index': 1}: Timeout (0.001s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '73ccce36932fe5c93f1be70e2459190f', '4577f84b-0cc1-4a3b-8b66-cfee248529a9-sda', '1717347169.94525', '1717347169.93867', '0', '', '1717347169.93867', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347169.99165', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '4577f84b-0cc1-4a3b-8b66-cfee248529a9-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp8kyvesdx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda', '1717347170.05517', '1717347170.05147', '0', '', '1717347170.05147', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda', '1717347170.05517', '1717347170.05147', '0', '', '1717347170.05147', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06417 seconds (15.58326/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp8kyvesdx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda', '1717347170.05517', '1717347170.05147', '0', '', '1717347170.05147', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347170.17091', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347170.17091', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347170.17091', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347170.08017', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347170.08017', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347170.08017', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347170.08020', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347170.08020', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347170.08020', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347170.08046', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347170.08046', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347170.08023', 'epoch': '1717347170.08023', 'reported': 0, 'tombstones': -1}], '71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda') test DEBUG: /tmp/tmp8kyvesdx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp8kyvesdx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda', '1717347170.05517', '1717347170.05147', '0', '', '1717347170.05147', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347170.17091', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347170.17091', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347170.17091', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347170.08017', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347170.08017', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347170.08017', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347170.08020', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347170.08020', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347170.08020', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347170.08046', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347170.08046', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347170.08023', 'epoch': '1717347170.08023', 'reported': 0, 'tombstones': -1}], '71f2ab21-28d7-4f2b-8a92-2852baf4e6dc-sda') test DEBUG: /tmp/tmp8kyvesdx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.2:1002/sdc test WARNING: /tmp/tmp8kyvesdx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.2:1002/sdc; have shard ranges and will wait for cleaving test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04797 seconds (20.84536/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fc4556ea-5382-4118-aa9f-9e1f3189a559-sda', '1717347170.25956', '1717347170.25491', '1717347170.28669', '', '1717347170.28669', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpmn0es_75/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '10476b5e-ccf7-4c79-96ff-fec09077e73e-sda', '1717347170.34123', '1717347170.00000', '0', '', '1717347170.00000', 0, 0) test DEBUG: /tmp/tmpmn0es_75/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '10476b5e-ccf7-4c79-96ff-fec09077e73e-sda', '1717347170.34123', '1717347170.00000', '0', '', '1717347170.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('10476b5e-ccf7-4c79-96ff-fec09077e73e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06668 seconds (14.99710/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphq_9j0h0/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '0da1aa7a-6b63-417f-b9a8-bc3c2b2d58d7-sda', '1717347170.48244', '1717347171.00000', '0', '{"X-Container-Sync-To": ["a", "1717347172.00000"], "X-Container-Sync-Key": ["secret", "1717347173.00000"]}', '1717347171.00000', 0, 0) REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('0da1aa7a-6b63-417f-b9a8-bc3c2b2d58d7-sda', 'c48d89b33840539f9c5442ab3b10027d.db') REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '0da1aa7a-6b63-417f-b9a8-bc3c2b2d58d7-sda', '1717347170.48244', '1717347171.00000', '0', '{"X-Container-Sync-To": ["a", "1717347172.00000"], "X-Container-Sync-Key": ["secret", "1717347173.00000"]}', '1717347171.00000', 0, 0) REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('0da1aa7a-6b63-417f-b9a8-bc3c2b2d58d7-sda', 'c48d89b33840539f9c5442ab3b10027d.db') test DEBUG: Replicating db /tmp/tmphq_9j0h0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f72fcb5d-502a-439b-8ec1-d1f793271def-sda', '1717347170.46952', '1717347170.00000', '0', '', '1717347170.00000', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object ... ok test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (201, 404, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (201, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f72fcb5d-502a-439b-8ec1-d1f793271def-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f72fcb5d-502a-439b-8ec1-d1f793271def-sda', '1717347170.46952', '1717347170.00000', '0', '', '1717347170.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f72fcb5d-502a-439b-8ec1-d1f793271def-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.12215 seconds (16.37337/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:4 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmphq_9j0h0/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '0da1aa7a-6b63-417f-b9a8-bc3c2b2d58d7-sda', '1717347170.48244', '1717347171.00000', '0', '{"X-Container-Sync-To": ["", "1717347174.00000"], "X-Container-Sync-Key": ["secret", "1717347173.00000"]}', '1717347171.00000', 0, 0) test DEBUG: /tmp/tmphq_9j0h0/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '0da1aa7a-6b63-417f-b9a8-bc3c2b2d58d7-sda', '1717347170.48244', '1717347171.00000', '0', '{"X-Container-Sync-To": ["", "1717347174.00000"], "X-Container-Sync-Key": ["secret", "1717347173.00000"]}', '1717347171.00000', 0, 0) test DEBUG: /tmp/tmphq_9j0h0/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.2:1002/sdc, nothing to do test DEBUG: Replicating db /tmp/tmphq_9j0h0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f72fcb5d-502a-439b-8ec1-d1f793271def-sda', '1717347170.46952', '1717347170.00000', '0', '', '1717347170.00000', 0, 0) test DEBUG: /tmp/tmphq_9j0h0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'f72fcb5d-502a-439b-8ec1-d1f793271def-sda', '1717347170.46952', '1717347170.00000', '0', '', '1717347170.00000', 0, 0) test DEBUG: /tmp/tmphq_9j0h0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.05180 seconds (38.60942/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:4 remote_merge:0 rsync:0 ts_repl:0 test ERROR: Failed to update sync_store /tmp/tmp2gndq49d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/replicator.py", line 301, in _post_replicate_hook self.sync_store.update_sync_store(broker) File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_replicator.py", line 1226, in update_sync_store raise OSError(1, '1') PermissionError: [Errno 1] 1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Unexpected response while deleting object account container obj: 404 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 412 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 500 test-expirer DEBUG: b'' test-expirer ERROR: Exception while deleting object account container obj : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer ERROR: Exception while deleting object .expiring_objects 1717260770 1717260770-a0/c0/o0 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260770 1717260770-a1/c1/o1 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260770 1717260770-a2/c2/o2 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260770 1717260770-a3/c3/o3 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260770 1717260770-a4/c4/o4 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init test.unit.obj.test_expirer.TestObjectExpirer.test_init ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_report test.unit.obj.test_expirer.TestObjectExpirer.test_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted ... ok test-expirer ERROR: Exception while deleting object .expiring_objects 1717347169 1717347169-a5/c5/o5 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347169 1717347169-a6/c6/o6 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347169 1717347169-a7/c7/o7 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347169 1717347169-a8/c8/o8♡ TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347169 1717347169-a9/c9/o9ø TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer ERROR: Unexcepted error handling task '1717260773-invalid0': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid0 test-expirer ERROR: Unexcepted error handling task '1717260773-invalid1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test-expirer INFO: Pass so far 0s; 0 objects expired test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0000', 'task_object': '0000-invalid0', 'delete_timestamp': 0000000000.00000_0000000000000000, 'target_path': 'invalid0', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0001', 'task_object': '0001-invalid1', 'delete_timestamp': 0000000001.00000_0000000000000000, 'target_path': 'invalid1', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0002', 'task_object': '0002-invalid2', 'delete_timestamp': 0000000002.00000_0000000000000000, 'target_path': 'invalid2', 'is_async_delete': False} test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 380, in run_forever self.run_once(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/test/unit/obj/test_expirer.py", line 907, in raise_exceptions raise Exception('exception %d' % raises[0]) Exception: exception 1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 330, in run_once self.swift.get_account_info(task_account) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'get_account_info' test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: txabf61f7ae8824ef183cb5-00665ca369) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txeafc63a04d3f45d9b957b-00665ca369) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: Container GET returning 503 for [] (txn: txc710cbbdfbd24ce8bb9d8-00665ca367) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx187ee775ef0c49579ca53-00665ca367) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx187ee775ef0c49579ca53-00665ca367) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx187ee775ef0c49579ca53-00665ca367) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: tx187ee775ef0c49579ca53-00665ca367) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx187ee775ef0c49579ca53-00665ca367) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx187ee775ef0c49579ca53-00665ca367) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx187ee775ef0c49579ca53-00665ca367) test INFO: Client disconnected on read of '/a/c' (txn: tx616f8dc2fe714459adacf-00665ca367) test INFO: Client disconnected on read of '/a/c' (txn: tx7d299e7c0402474ba432e-00665ca367) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx371ec0f128924a109c770-00665ca367) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx371ec0f128924a109c770-00665ca367) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx371ec0f128924a109c770-00665ca367) test ERROR: Container GET returning 503 for [] (txn: tx371ec0f128924a109c770-00665ca367) test INFO: Client disconnected on read of '/a/c' (txn: tx371ec0f128924a109c770-00665ca367) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: Container GET returning 503 for [] (txn: tx60f71b92429342b891f34-00665ca367) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd88e16e3ae554e789cbd1-00665ca367) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd88e16e3ae554e789cbd1-00665ca367) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd88e16e3ae554e789cbd1-00665ca367) test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd88e16e3ae554e789cbd1-00665ca367) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txd88e16e3ae554e789cbd1-00665ca367) test ERROR: Container GET returning 503 for [] (txn: txd88e16e3ae554e789cbd1-00665ca367) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc29a4a789149454392f7e-00665ca368) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc29a4a789149454392f7e-00665ca368) test ERROR: Container GET returning 503 for [] (txn: txc29a4a789149454392f7e-00665ca368) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx806cb0ffb36d48c78e93f-00665ca368) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx98388b55f7fc4fe594b7f-00665ca368) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx98388b55f7fc4fe594b7f-00665ca368) test ERROR: Container GET returning 503 for [500, 500] (txn: tx98388b55f7fc4fe594b7f-00665ca368) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx73b22aa4b6f841c58ef89-00665ca368) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txdf13cf72b4964657843ea-00665ca368) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txdf13cf72b4964657843ea-00665ca368) test INFO: Client disconnected on read of '/a/c' (txn: txdf13cf72b4964657843ea-00665ca368) test ERROR: Container GET returning 503 for [] (txn: txf13840288944486ab25c9-00665ca368) test INFO: Client disconnected on read of '/a/c' (txn: txf13840288944486ab25c9-00665ca368) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txea82987b1eee494489b8c-00665ca368) test INFO: Client disconnected on read of '/a/c' (txn: txea82987b1eee494489b8c-00665ca368) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txedb197913b284bc283216-00665ca368) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txedb197913b284bc283216-00665ca368) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txedb197913b284bc283216-00665ca368) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txedb197913b284bc283216-00665ca368) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txedb197913b284bc283216-00665ca368) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: txedb197913b284bc283216-00665ca368) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ddba527173848659e325-00665ca368) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ddba527173848659e325-00665ca368) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ddba527173848659e325-00665ca368) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ddba527173848659e325-00665ca368) test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347301.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/9b3/0ecce7ff9e1f31833bc6e5df5c9539b3/0ecce7ff9e1f31833bc6e5df5c9539b3.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347301.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/9b3/0ecce7ff9e1f31833bc6e5df5c9539b3/0ecce7ff9e1f31833bc6e5df5c9539b3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347304.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/b43/fa0a6b08aaa815cbe399709bc72c2b43/fa0a6b08aaa815cbe399709bc72c2b43.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347304.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/b43/fa0a6b08aaa815cbe399709bc72c2b43/fa0a6b08aaa815cbe399709bc72c2b43.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347307.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/87f/f4cc0bc2dd03e42d68f44b648134f87f/f4cc0bc2dd03e42d68f44b648134f87f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347307.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/87f/f4cc0bc2dd03e42d68f44b648134f87f/f4cc0bc2dd03e42d68f44b648134f87f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347310.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/66c/8948d8418d07200876a12a2bb423d66c/8948d8418d07200876a12a2bb423d66c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347310.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/66c/8948d8418d07200876a12a2bb423d66c/8948d8418d07200876a12a2bb423d66c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347313.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/1cb/b2ca56cb3445357bb0f10ad7b4d6f1cb/b2ca56cb3445357bb0f10ad7b4d6f1cb.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347313.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/1cb/b2ca56cb3445357bb0f10ad7b4d6f1cb/b2ca56cb3445357bb0f10ad7b4d6f1cb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347316.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/d98/ab6feeba2011355fccdf84c60aa17d98/ab6feeba2011355fccdf84c60aa17d98.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347316.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/d98/ab6feeba2011355fccdf84c60aa17d98/ab6feeba2011355fccdf84c60aa17d98.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347319.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7ad/ede077a8091e939457ef3dbc791087ad/ede077a8091e939457ef3dbc791087ad.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347319.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/7ad/ede077a8091e939457ef3dbc791087ad/ede077a8091e939457ef3dbc791087ad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347322.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/d1e/cb37ebc4efa9423123ee96f0302afd1e/cb37ebc4efa9423123ee96f0302afd1e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347322.00000-0, db: /tmp/tmp9vb8pr5k/sda/containers/0/d1e/cb37ebc4efa9423123ee96f0302afd1e/cb37ebc4efa9423123ee96f0302afd1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347171.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e58/64187d1dbb62210e8df0a37eed086e58/64187d1dbb62210e8df0a37eed086e58.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347176.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/343/5e2d0efa8baa83468002dedd31db9343/5e2d0efa8baa83468002dedd31db9343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347181.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/71e/f3f9546e6bd95c47724d04d854a8071e/f3f9546e6bd95c47724d04d854a8071e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347186.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b68/d1cf8a60f7bf9b0415f9d61e896b3b68/d1cf8a60f7bf9b0415f9d61e896b3b68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347191.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a7f/77f119b07a582438555b16744acf8a7f/77f119b07a582438555b16744acf8a7f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347196.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/af5/2722d14adf6006e454f1bd1f56ae9af5/2722d14adf6006e454f1bd1f56ae9af5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347201.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a73/ba77b742b64680e0c5d093ec0937fa73/ba77b742b64680e0c5d093ec0937fa73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347221.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/fe7/2565fb6b452c3357d772b13165edbfe7/2565fb6b452c3357d772b13165edbfe7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347226.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3b9/dfe54dbba6ff5f1625c32d18336dc3b9/dfe54dbba6ff5f1625c32d18336dc3b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347236.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a8f/ebfb13dc304bbd03a49c50d07f4e8a8f/ebfb13dc304bbd03a49c50d07f4e8a8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347241.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/da5/468ece8cf53547f1b8a568b80793eda5/468ece8cf53547f1b8a568b80793eda5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347251.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3fd/7751ae5ee3588174f058cdf892da93fd/7751ae5ee3588174f058cdf892da93fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347256.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8a1/99e66f00d4f64cf385e7c517ec9a28a1/99e66f00d4f64cf385e7c517ec9a28a1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347261.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8ac/81e1169bc4e5fb82f1f14eb5ad7e78ac/81e1169bc4e5fb82f1f14eb5ad7e78ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347266.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/41e/92f67ef3444669c841ee16f4c1f1741e/92f67ef3444669c841ee16f4c1f1741e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347271.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d48/ca9cc2335300ff7749b4d62fa2484d48/ca9cc2335300ff7749b4d62fa2484d48.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347281.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/9e7/394ac10a5273e9fd36787515f1e1c9e7/394ac10a5273e9fd36787515f1e1c9e7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347286.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/93f/63029f619cf6f041b5be15301935993f/63029f619cf6f041b5be15301935993f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347296.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ede/fb883c1b75b874242cbae9762c9daede/fb883c1b75b874242cbae9762c9daede.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347301.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/9b3/0ecce7ff9e1f31833bc6e5df5c9539b3/0ecce7ff9e1f31833bc6e5df5c9539b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347311.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a21/b95f4c519c69274b073697f12bc79a21/b95f4c519c69274b073697f12bc79a21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347316.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d98/ab6feeba2011355fccdf84c60aa17d98/ab6feeba2011355fccdf84c60aa17d98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347321.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/287/8358b8edaaf2a01d62c53bf67645e287/8358b8edaaf2a01d62c53bf67645e287.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347326.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8c5/7a643b855bccc4cf3c4eaeda83fcb8c5/7a643b855bccc4cf3c4eaeda83fcb8c5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347331.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/015/d3562d2e4c11fb2711fd44745c331015/d3562d2e4c11fb2711fd44745c331015.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347336.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/6a9/cb56574485967632113a25de82a576a9/cb56574485967632113a25de82a576a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347341.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e8f/524994fab6a7b9f6c50b1aadccf3fe8f/524994fab6a7b9f6c50b1aadccf3fe8f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347346.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/042/b36421fbfb3a8e851264ca2526d55042/b36421fbfb3a8e851264ca2526d55042.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347351.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/07c/98151d85b578b8be96f86b27962e607c/98151d85b578b8be96f86b27962e607c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347356.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/fd9/5e75a8aabafea6a2b2a2950786459fd9/5e75a8aabafea6a2b2a2950786459fd9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347361.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f67/cead5fd9d4fc3e92b86935946cfd6f67/cead5fd9d4fc3e92b86935946cfd6f67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347366.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e53/561c335c709174eba02b2bdfe989fe53/561c335c709174eba02b2bdfe989fe53.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347371.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/01c/042976de285ca25e79377d6018f6a01c/042976de285ca25e79377d6018f6a01c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347376.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/11a/1fd1ed0006e3b44abafa1ab44bc7011a/1fd1ed0006e3b44abafa1ab44bc7011a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347381.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/505/7efe0fac6ce1dead199810f434c1e505/7efe0fac6ce1dead199810f434c1e505.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347386.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5ca/e95de7b9d6e28529830faa640d79a5ca/e95de7b9d6e28529830faa640d79a5ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347391.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/4f5/776a39fcf13ae439e800e99a41f964f5/776a39fcf13ae439e800e99a41f964f5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347396.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/57c/65b8a04cfc9b201d45fbeba64040e57c/65b8a04cfc9b201d45fbeba64040e57c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347401.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/001/9bf4dabc3693acf7cf251d73854c6001/9bf4dabc3693acf7cf251d73854c6001.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347406.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ed4/41f4ba63647d5a496efd0396c7710ed4/41f4ba63647d5a496efd0396c7710ed4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347411.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f46/9ebb1d40fb8411967f98e5d9e0b91f46/9ebb1d40fb8411967f98e5d9e0b91f46.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347416.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5c8/49cdec80e148f9846e74797f86acf5c8/49cdec80e148f9846e74797f86acf5c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347421.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/7db/5520c78713d238fbff5e39a4cc57c7db/5520c78713d238fbff5e39a4cc57c7db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347426.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/635/f73e832299517c227d02e17562ced635/f73e832299517c227d02e17562ced635.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347431.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c36/b4f2063855b74c713b6b6f78accabc36/b4f2063855b74c713b6b6f78accabc36.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347436.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/508/6f7bc76dab03d7cfd8769c0a0a9cc508/6f7bc76dab03d7cfd8769c0a0a9cc508.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347441.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c7a/8f309f953aa2c3faad69cbadb5b4ac7a/8f309f953aa2c3faad69cbadb5b4ac7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347446.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/39c/6b840ade6c08b0f5d0fe2c152c2de39c/6b840ade6c08b0f5d0fe2c152c2de39c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347451.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f3a/ff242e03b700be3fc02edd358a06bf3a/ff242e03b700be3fc02edd358a06bf3a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347456.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/df2/93c5c28d0484465f57318ba1e6c2adf2/93c5c28d0484465f57318ba1e6c2adf2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347461.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b0b/323b8e2f1be3a0220329bca463ffcb0b/323b8e2f1be3a0220329bca463ffcb0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347466.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/afb/a423957a5707e51dcefac3410b2fbafb/a423957a5707e51dcefac3410b2fbafb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347471.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/bdf/be189244488ca5068644e982de7aebdf/be189244488ca5068644e982de7aebdf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347476.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/96b/02634527f89f260bcea4612470ee396b/02634527f89f260bcea4612470ee396b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347481.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/7ae/d2067d78e367ddb1dbe21d8196bc37ae/d2067d78e367ddb1dbe21d8196bc37ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347486.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/697/5863b44d485bd2c1571c757cefdea697/5863b44d485bd2c1571c757cefdea697.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347491.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a73/74e53c5ecd37dd833fd159ed2cfb6a73/74e53c5ecd37dd833fd159ed2cfb6a73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347496.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b29/9f5294e180318fe2c2801f2efcec7b29/9f5294e180318fe2c2801f2efcec7b29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347501.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3e8/3bdd79d191994b4971ce3c9d17d873e8/3bdd79d191994b4971ce3c9d17d873e8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347506.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8fc/631c26eae7b81c47dea411a0b50b38fc/631c26eae7b81c47dea411a0b50b38fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347511.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/09a/fbbd3e4526333b1a9174e1bd1419d09a/fbbd3e4526333b1a9174e1bd1419d09a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347516.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c83/40771d6f040a89cd670ab1e7cd402c83/40771d6f040a89cd670ab1e7cd402c83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347521.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/675/519b729bbb981564ecd13efad3f26675/519b729bbb981564ecd13efad3f26675.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347526.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a79/2f4439ce67d7a4305b21be2bf7743a79/2f4439ce67d7a4305b21be2bf7743a79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347531.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ebe/3685f67842e65c80bbf136b17d56cebe/3685f67842e65c80bbf136b17d56cebe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347536.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b2d/a9d22f3355b7608297c18e95f068eb2d/a9d22f3355b7608297c18e95f068eb2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347541.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/385/fb57c201897216b0e1c6b3c853126385/fb57c201897216b0e1c6b3c853126385.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347546.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/83d/0c2e50e8c39873593ae91e4d1c96083d/0c2e50e8c39873593ae91e4d1c96083d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347551.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ae5/32301a34a576c1c5bb1be6efe9a16ae5/32301a34a576c1c5bb1be6efe9a16ae5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347556.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/bd4/6131607b2464f36af991062975070bd4/6131607b2464f36af991062975070bd4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347561.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/1c9/323bad91ef1dfccbfceda6d1c45b31c9/323bad91ef1dfccbfceda6d1c45b31c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347566.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/56d/3f38089a621431766aad7d1d4da1156d/3f38089a621431766aad7d1d4da1156d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347571.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347576.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/4b8/2480e27a7bd428748e78db4a5f9694b8/2480e27a7bd428748e78db4a5f9694b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347581.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/87c/095c2a458ebb8e52c9cd8b9fbbdbb87c/095c2a458ebb8e52c9cd8b9fbbdbb87c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347586.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ab0/5c4022110f825609da641f495808dab0/5c4022110f825609da641f495808dab0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347591.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/9dc/c4990f853103b5378ab8adce6c5799dc/c4990f853103b5378ab8adce6c5799dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347596.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/952/0d04f62a905f37d3008784e8df3f8952/0d04f62a905f37d3008784e8df3f8952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347601.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/603/8c1d0fb6cd18823fe540266f482db603/8c1d0fb6cd18823fe540266f482db603.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347606.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d62/0bc6cfc11cda2d7d9ee4de6a7e320d62/0bc6cfc11cda2d7d9ee4de6a7e320d62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347611.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c15/7a1b1890283c6d39e2b2a8372f554c15/7a1b1890283c6d39e2b2a8372f554c15.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347621.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ec5/d1e5ed1cb9b3ae4c20faa5558fbefec5/d1e5ed1cb9b3ae4c20faa5558fbefec5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347626.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a98/1e3cba8648d01a23ad6749a55042aa98/1e3cba8648d01a23ad6749a55042aa98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347631.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/823/272092cf742dfbeef04f71672e31d823/272092cf742dfbeef04f71672e31d823.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347656.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b0e/0d131d949dc99f78dddc400126474b0e/0d131d949dc99f78dddc400126474b0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347661.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/fa9/2ee0011b7836ef50a6548f0f5439cfa9/2ee0011b7836ef50a6548f0f5439cfa9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347666.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b01/ded215a90f8af0bd03a459a3039a7b01/ded215a90f8af0bd03a459a3039a7b01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347671.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/474/5dd5386cac23f769594971f1284ae474/5dd5386cac23f769594971f1284ae474.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347686.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/933/5e8d9b5582f0bad722c8505f57ade933/5e8d9b5582f0bad722c8505f57ade933.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347691.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/223/9562307612d5aeab22c4e29141429223/9562307612d5aeab22c4e29141429223.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347701.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/340/338f10fab22b80566a33c54f7707b340/338f10fab22b80566a33c54f7707b340.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347706.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/250/390b9a2e91110c85bc4826e989ad2250/390b9a2e91110c85bc4826e989ad2250.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347711.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/6df/57dd647dd6d1de8ae7123608ff9736df/57dd647dd6d1de8ae7123608ff9736df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347716.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ef8/2bd31ca5e5fb2da1a764945d5aaeeef8/2bd31ca5e5fb2da1a764945d5aaeeef8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347721.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/30b/a840879dd01db1abe26a5be62c2b730b/a840879dd01db1abe26a5be62c2b730b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347726.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/919/2d3fccec5449ced10bab5e792b82d919/2d3fccec5449ced10bab5e792b82d919.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347731.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/aae/1e162350d0a369474f8a486f29a8aaae/1e162350d0a369474f8a486f29a8aaae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347746.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/294/0955b73a8c32ee5a94b397491515a294/0955b73a8c32ee5a94b397491515a294.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347751.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b8b/ef2502503030e69f7858112638ac2b8b/ef2502503030e69f7858112638ac2b8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347761.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/0c3/4b2285ee40dca96fe129bd70f95380c3/4b2285ee40dca96fe129bd70f95380c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347766.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c45/e6438e2ceb444065cbbbb631c9be9c45/e6438e2ceb444065cbbbb631c9be9c45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347771.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/41c/318dbae52b2facfd3d7308f395f2b41c/318dbae52b2facfd3d7308f395f2b41c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347776.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a49/76c0b29cc74d17206aace3a14e682a49/76c0b29cc74d17206aace3a14e682a49.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347781.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ca0/6759e6311b4637ebee8520056518fca0/6759e6311b4637ebee8520056518fca0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347786.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/48e/3cd1b208bf2bcff2ebe5a4ef91ae948e/3cd1b208bf2bcff2ebe5a4ef91ae948e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347791.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ed8/9151146d914dc706c02b972cace62ed8/9151146d914dc706c02b972cace62ed8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347796.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/2e9/b24ef43711af2c751d4669c3d515d2e9/b24ef43711af2c751d4669c3d515d2e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347801.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f97/83e0c884ad3cccd2eeb6c7628ad72f97/83e0c884ad3cccd2eeb6c7628ad72f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347806.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/51f/e0fd15fc9906f2f18fa47a8866bf551f/e0fd15fc9906f2f18fa47a8866bf551f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347811.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/733/6c0df9a203dc1579ae135234cc9fb733/6c0df9a203dc1579ae135234cc9fb733.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347816.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/48c/d67b765547bbc5013b2795518c7a348c/d67b765547bbc5013b2795518c7a348c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347821.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5e1/5c5cd5d35895fceb0a365e99a32285e1/5c5cd5d35895fceb0a365e99a32285e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347826.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/873/6416c224725f217ec6279be6fb802873/6416c224725f217ec6279be6fb802873.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347831.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/6d2/40a071409db54313470d032a2c5df6d2/40a071409db54313470d032a2c5df6d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347836.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e5c/44d385caacc6153ade9eca4f23cfde5c/44d385caacc6153ade9eca4f23cfde5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347841.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5c3/3b58f7e1fdada2f37f5bc1d682bc15c3/3b58f7e1fdada2f37f5bc1d682bc15c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347846.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/20e/c83e295a4ab6eaf41c64e3ecbffa820e/c83e295a4ab6eaf41c64e3ecbffa820e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347851.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d68/9c12eb783405094d82dc1dd8cd573d68/9c12eb783405094d82dc1dd8cd573d68.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347856.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/285/d3a10181415135d1939d0a3d5bb4d285/d3a10181415135d1939d0a3d5bb4d285.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347861.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/64b/9fb673b94d6407e7932278e8e1acc64b/9fb673b94d6407e7932278e8e1acc64b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347866.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3d3/db17459df0047fc27fc4c5ef9bcad3d3/db17459df0047fc27fc4c5ef9bcad3d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347871.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8a6/0ea6faa97cde278e2326ffc6f2cce8a6/0ea6faa97cde278e2326ffc6f2cce8a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347876.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/565/b413d5e29b4d50e7764db0333e0a8565/b413d5e29b4d50e7764db0333e0a8565.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347881.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/61b/2f3aec0313542588c84e9f2dfc09e61b/2f3aec0313542588c84e9f2dfc09e61b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347886.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d2d/dde980c4b7d06283764610a1f0cf1d2d/dde980c4b7d06283764610a1f0cf1d2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347891.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/54b/a0e45c14ee21a432cb1966b74a9bb54b/a0e45c14ee21a432cb1966b74a9bb54b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347896.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/05b/e2ed524e8060da23f95e47135292005b/e2ed524e8060da23f95e47135292005b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347901.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/131/dcf7cfdeb95325c6bd5dba5eaeb06131/dcf7cfdeb95325c6bd5dba5eaeb06131.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347906.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ddc/3d623cd442e116c6aeeae75cdcc33ddc/3d623cd442e116c6aeeae75cdcc33ddc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347911.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ca8/97896419c08cadbf2fe0e41511ddaca8/97896419c08cadbf2fe0e41511ddaca8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347916.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c09/dd1ed34f0917ea13f0b85442b07ecc09/dd1ed34f0917ea13f0b85442b07ecc09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347921.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/adc/8ab0e947c4b4e8ad7b9cd32da9874adc/8ab0e947c4b4e8ad7b9cd32da9874adc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347926.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/eb8/5bf53b617ac36e7eb9df6cc941b2feb8/5bf53b617ac36e7eb9df6cc941b2feb8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347931.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/44b/69a40c5ec70ba711f39ec83a4ae7744b/69a40c5ec70ba711f39ec83a4ae7744b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347936.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/aef/350415aca3e9717e132d2b4b9c284aef/350415aca3e9717e132d2b4b9c284aef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347941.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/4fb/edac15b85b7c7a507807b52cf0f844fb/edac15b85b7c7a507807b52cf0f844fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347946.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b65/c7ca2235947816d806284af631033b65/c7ca2235947816d806284af631033b65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347951.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3ca/99c2aa3fc06d541787e89474575c13ca/99c2aa3fc06d541787e89474575c13ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347956.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d2d/88c1ca3ab9b1764849c170469e2ced2d/88c1ca3ab9b1764849c170469e2ced2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347961.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/267/d6a5f2131e549f18bb5b242406490267/d6a5f2131e549f18bb5b242406490267.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347966.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/2f0/43d4820b18b0205bc3c7d2267d24a2f0/43d4820b18b0205bc3c7d2267d24a2f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347971.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ad1/4181ecfdeb223788d788ce21fd0b8ad1/4181ecfdeb223788d788ce21fd0b8ad1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347976.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/00f/3d2d8bbcb194eab9832eef73529dc00f/3d2d8bbcb194eab9832eef73529dc00f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347981.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/81b/4e945cb38c3ebbc1ee6fd98ed6b6281b/4e945cb38c3ebbc1ee6fd98ed6b6281b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347986.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/259/f62b66ff0f9e671fb67b8464e2bba259/f62b66ff0f9e671fb67b8464e2bba259.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347991.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/4f0/118ded90726d86e6183375ad86a944f0/118ded90726d86e6183375ad86a944f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347996.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/6f8/a8a95b8853f53efd045879a9fe8456f8/a8a95b8853f53efd045879a9fe8456f8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348001.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f33/5116e0ccec346a3dc6d0aaddd04b6f33/5116e0ccec346a3dc6d0aaddd04b6f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348006.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/1d1/afa7fda231999cb100a9413335d7c1d1/afa7fda231999cb100a9413335d7c1d1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348011.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a98/b9ebf82c77391efee54587f0a8acca98/b9ebf82c77391efee54587f0a8acca98.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348016.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/00b/d9ee39111d673d7cae8a0355141e000b/d9ee39111d673d7cae8a0355141e000b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348021.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5b9/dfa7c8da860bfa45974ae43fd7c8d5b9/dfa7c8da860bfa45974ae43fd7c8d5b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348026.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/79f/b0997c389df87f7ec4d9a2cc17ba779f/b0997c389df87f7ec4d9a2cc17ba779f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348031.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/7cf/8495cd601b2ed80346ed9079d9e0b7cf/8495cd601b2ed80346ed9079d9e0b7cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348036.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/1a5/514f3ee9b3f8b8fcd3adea99f02401a5/514f3ee9b3f8b8fcd3adea99f02401a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348041.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/add/ad8b005ff9a10fe82fc94bf386225add/ad8b005ff9a10fe82fc94bf386225add.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348046.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/468/7fb6074278f19cfd272e07d35dc02468/7fb6074278f19cfd272e07d35dc02468.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348051.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/604/24e10ccff6365c95971094f516db8604/24e10ccff6365c95971094f516db8604.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348056.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/9fa/cc4bcab6963551ceb295ca97841339fa/cc4bcab6963551ceb295ca97841339fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348061.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/089/4eb071be17b3731b7bcd20315fb19089/4eb071be17b3731b7bcd20315fb19089.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348066.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e5f/6207977c881c1e6f97460f0c40cf3e5f/6207977c881c1e6f97460f0c40cf3e5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348071.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/086/8018471d397b572673d41cc59e417086/8018471d397b572673d41cc59e417086.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348076.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/aa3/f42c5e768772f5ca5a9592890f391aa3/f42c5e768772f5ca5a9592890f391aa3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348081.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8cc/f0afae1cd204d19f6f07e2161f9d48cc/f0afae1cd204d19f6f07e2161f9d48cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348086.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ff3/5431f2d3ae24818660f7be43f62bdff3/5431f2d3ae24818660f7be43f62bdff3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348091.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/7e2/a059694237d153f74cc7b4091e53c7e2/a059694237d153f74cc7b4091e53c7e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348096.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f67/cc3de10bf4129ecbc2470be5ac438f67/cc3de10bf4129ecbc2470be5ac438f67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348101.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/8e1/6b64550eb0dbf684df85638591c288e1/6b64550eb0dbf684df85638591c288e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348106.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/031/c9badc067b71177cc4368aaaa8c3d031/c9badc067b71177cc4368aaaa8c3d031.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348111.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/040/bfa3568d3582f0eff86841ed6f6a6040/bfa3568d3582f0eff86841ed6f6a6040.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348116.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c2e/8aaf648894753f3b784680267123fc2e/8aaf648894753f3b784680267123fc2e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348121.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b0c/86ad1ef9e40d63cc885a26c15d284b0c/86ad1ef9e40d63cc885a26c15d284b0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348126.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/0a9/d1bd1cd25da042d18c1f1d84bc26b0a9/d1bd1cd25da042d18c1f1d84bc26b0a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348131.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/cdf/fe41fea5ffd650822d06e33a08916cdf/fe41fea5ffd650822d06e33a08916cdf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348136.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f37/1b74102165f18aa0af90a4651688cf37/1b74102165f18aa0af90a4651688cf37.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348141.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/742/f456edcfb31fc44e9dd88ea6debff742/f456edcfb31fc44e9dd88ea6debff742.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348146.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/4f1/645dcf5cdb9f24d7996ec28c13f334f1/645dcf5cdb9f24d7996ec28c13f334f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348151.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3a9/3f236f53b35ff91cbb7cc873030733a9/3f236f53b35ff91cbb7cc873030733a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348156.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/ccb/ba204a9106dec0c0e64d559cbddacccb/ba204a9106dec0c0e64d559cbddacccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348161.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a01/d6e4713cee68cbbe26f209a599a97a01/d6e4713cee68cbbe26f209a599a97a01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348166.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/96c/9f1785ac6efaa24c572255c68955d96c/9f1785ac6efaa24c572255c68955d96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348171.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3ae/9c2d94c74d2e4e34a5f91c3f8ae253ae/9c2d94c74d2e4e34a5f91c3f8ae253ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348176.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/dce/da100bcec9c20bf1503cebc26a83bdce/da100bcec9c20bf1503cebc26a83bdce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348181.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/2fe/ea31ae915600f3cc8606ea05fcf4e2fe/ea31ae915600f3cc8606ea05fcf4e2fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348186.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/04f/4e736ed44421d4282d52ba261f96604f/4e736ed44421d4282d52ba261f96604f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348191.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/bed/5d8a45319f33a197c55b6019b5d44bed/5d8a45319f33a197c55b6019b5d44bed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348196.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/4f3/6f87571588752a39d28272e61ada34f3/6f87571588752a39d28272e61ada34f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348201.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/969/73cbf72857c4b0290fe47053b6dc8969/73cbf72857c4b0290fe47053b6dc8969.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348206.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/36d/651707b5dae18a941b43f0f61660a36d/651707b5dae18a941b43f0f61660a36d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348211.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e1a/1a647ac6f90f287730d70a464dd4ce1a/1a647ac6f90f287730d70a464dd4ce1a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348216.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/30e/e124a1ce5dea3c143f62aa560e54a30e/e124a1ce5dea3c143f62aa560e54a30e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348221.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/55d/a993e64b92658631515bc6141d80e55d/a993e64b92658631515bc6141d80e55d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348226.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/411/bf173784507133f6d5260e5a47e0e411/bf173784507133f6d5260e5a47e0e411.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348231.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/03c/c96a19e00f4fa75ead9f6dde82b3803c/c96a19e00f4fa75ead9f6dde82b3803c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348236.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3ff/34045f77562af3d050d88046786ce3ff/34045f77562af3d050d88046786ce3ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348241.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d4d/82d04553ab33d48344d434747c45ad4d/82d04553ab33d48344d434747c45ad4d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348246.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/5e0/746bccc44d82a9a993848daa65c9a5e0/746bccc44d82a9a993848daa65c9a5e0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348251.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e87/fd503f23dd3b3b91263139c325ad5e87/fd503f23dd3b3b91263139c325ad5e87.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348256.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/9c8/3e429b05e0a104e971d064968bb569c8/3e429b05e0a104e971d064968bb569c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348261.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d00/3e73565531f2383a60d820da70a7dd00/3e73565531f2383a60d820da70a7dd00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348266.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/1af/4b8aa28063785dbc1f6d480cf8cd91af/4b8aa28063785dbc1f6d480cf8cd91af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348271.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/813/eb79afbf1dce6055e92ded3029ac1813/eb79afbf1dce6055e92ded3029ac1813.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348276.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/658/dfa0836eea7b5446d043a9113e0b8658/dfa0836eea7b5446d043a9113e0b8658.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348281.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e79/a04efb8036ac593dc0d02782b1108e79/a04efb8036ac593dc0d02782b1108e79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348286.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/196/3367f4cbe7b69a9d31ec5d908a5e5196/3367f4cbe7b69a9d31ec5d908a5e5196.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348291.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/884/b6dd21a0310d96d500d8d1128d082884/b6dd21a0310d96d500d8d1128d082884.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348296.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3b5/8c6ff896fbc6394e192b4eb8540bf3b5/8c6ff896fbc6394e192b4eb8540bf3b5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348301.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/bb2/4886bd0ce7d5489d975d4a758065abb2/4886bd0ce7d5489d975d4a758065abb2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348306.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/acd/5a1623b3ae9df78ac1f40ef5856e3acd/5a1623b3ae9df78ac1f40ef5856e3acd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348311.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/752/75b045cb7c9628ca2e919e06bd0ef752/75b045cb7c9628ca2e919e06bd0ef752.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348316.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/284/59070ee50cb052fe8509429bbfa11284/59070ee50cb052fe8509429bbfa11284.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348321.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/683/815cb3caf752a8b03ed9839fcb81a683/815cb3caf752a8b03ed9839fcb81a683.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348326.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/2b4/dc13414f77056ca85cb36277591e12b4/dc13414f77056ca85cb36277591e12b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348331.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e3a/325b330e26984f5e727975ad3a914e3a/325b330e26984f5e727975ad3a914e3a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348336.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/2c6/b02ac3acc389946fb384838eb6f562c6/b02ac3acc389946fb384838eb6f562c6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348341.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a5e/3df7d1cd0ebbb50386925021de1b8a5e/3df7d1cd0ebbb50386925021de1b8a5e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348346.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a40/9376bd617f18b8694d27ded12bd55a40/9376bd617f18b8694d27ded12bd55a40.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348351.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/d88/31aca23d35bf561c6a77100a47861d88/31aca23d35bf561c6a77100a47861d88.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348356.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3cf/d74a6176dc150028440ad9a7961313cf/d74a6176dc150028440ad9a7961313cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348361.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/309/9ca2bd63e35953b8dc75cec9007d9309/9ca2bd63e35953b8dc75cec9007d9309.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348366.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/08d/8469093dded3c7e8cfcab616e81d408d/8469093dded3c7e8cfcab616e81d408d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348371.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e3b/1aa71ddb00f5560a61958fd95bbfee3b/1aa71ddb00f5560a61958fd95bbfee3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348376.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f38/106f79678a167f3ec916de2713848f38/106f79678a167f3ec916de2713848f38.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348381.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/1ad/634bdc4185ade3bb0316fb50bc7071ad/634bdc4185ade3bb0316fb50bc7071ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348386.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/b6f/e1f061ad63b9d4149b84f8a10f033b6f/e1f061ad63b9d4149b84f8a10f033b6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348391.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c59/1b78fa21d071e85e78741e68606b2c59/1b78fa21d071e85e78741e68606b2c59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348396.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/1f6/48ff379d9fdd848d474796029076f1f6/48ff379d9fdd848d474796029076f1f6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348401.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/3ec/725986599b9914f8be45ec0cadf8a3ec/725986599b9914f8be45ec0cadf8a3ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348406.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/00e/de751ca9e7c5a647d7bfe253e856b00e/de751ca9e7c5a647d7bfe253e856b00e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348411.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/a21/9656f5b46253baa1f61874cd055b6a21/9656f5b46253baa1f61874cd055b6a21.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348416.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/c23/1472cdcb57bc8b6594fafdad5c1b3c23/1472cdcb57bc8b6594fafdad5c1b3c23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348421.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/450/8817532ca4232ae1bf8940e06d283450/8817532ca4232ae1bf8940e06d283450.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348426.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/122/37bb3537c88b11f22c2d82ac82376122/37bb3537c88b11f22c2d82ac82376122.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348431.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/dcf/dd9875c0af37c057b68e73a61ebd7dcf/dd9875c0af37c057b68e73a61ebd7dcf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348436.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/e67/a7723ed9bdd7de7d4ce56ddbd20e0e67/a7723ed9bdd7de7d4ce56ddbd20e0e67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348441.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/f23/c6acdf83e7243a01f7df9ad8d8bbff23/c6acdf83e7243a01f7df9ad8d8bbff23.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2ddba527173848659e325-00665ca368) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: tx2ddba527173848659e325-00665ca368) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: Setting object count to 1032 (txn: tx2472d9e12add4dd78ad00-00665ca36d) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: Setting object count to 48 (txn: tx44c9d10ab0744e169fa17-00665ca36d) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: Setting object count to 1064 (txn: tx96aeabce0cac412086466-00665ca36d) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348446.00000-0, db: /tmp/tmppu27q3x6/sda/containers/0/0da/905b381b971f6d77bce4b3d7dac3d0da/905b381b971f6d77bce4b3d7dac3d0da.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7ed/f8c01d775095ff8ec816acfa94a0b7ed/f8c01d775095ff8ec816acfa94a0b7ed.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347183.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7ed/f8c01d775095ff8ec816acfa94a0b7ed/f8c01d775095ff8ec816acfa94a0b7ed_1717347183.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/a54/1f5028289f0b07234737183d74307a54/1f5028289f0b07234737183d74307a54.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347187.00000-0, db: /tmp/tmp020hq008/sda/containers/0/a54/1f5028289f0b07234737183d74307a54/1f5028289f0b07234737183d74307a54_1717347187.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/a7f/77f119b07a582438555b16744acf8a7f/77f119b07a582438555b16744acf8a7f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347191.00000-0, db: /tmp/tmp020hq008/sda/containers/0/a7f/77f119b07a582438555b16744acf8a7f/77f119b07a582438555b16744acf8a7f_1717347191.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/2dc/11d3fc44d684c2ca9264da909b89e2dc/11d3fc44d684c2ca9264da909b89e2dc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347195.00000-0, db: /tmp/tmp020hq008/sda/containers/0/2dc/11d3fc44d684c2ca9264da909b89e2dc/11d3fc44d684c2ca9264da909b89e2dc_1717347195.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmp020hq008/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca_1717347199.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347203.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67_1717347203.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmp020hq008/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa_1717347207.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmp020hq008/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6_1717347211.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/21e/7923680be1a8a91cbc1c464dfa5aa21e/7923680be1a8a91cbc1c464dfa5aa21e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347215.00000-0, db: /tmp/tmp020hq008/sda/containers/0/21e/7923680be1a8a91cbc1c464dfa5aa21e/7923680be1a8a91cbc1c464dfa5aa21e_1717347215.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/33e/f50715e7e4da888a7a403d93ef5d933e/f50715e7e4da888a7a403d93ef5d933e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347219.00000-0, db: /tmp/tmp020hq008/sda/containers/0/33e/f50715e7e4da888a7a403d93ef5d933e/f50715e7e4da888a7a403d93ef5d933e_1717347219.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/2e0/0c90cfa44b400db3516e9c9f9e4932e0/0c90cfa44b400db3516e9c9f9e4932e0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347223.00000-0, db: /tmp/tmp020hq008/sda/containers/0/2e0/0c90cfa44b400db3516e9c9f9e4932e0/0c90cfa44b400db3516e9c9f9e4932e0_1717347223.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b7d/7b6b622ace786e36072306e238d15b7d/7b6b622ace786e36072306e238d15b7d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347227.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b7d/7b6b622ace786e36072306e238d15b7d/7b6b622ace786e36072306e238d15b7d_1717347227.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4_1717347231.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7dd/8577c4222d3f528dffdf240ed803d7dd/8577c4222d3f528dffdf240ed803d7dd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347235.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7dd/8577c4222d3f528dffdf240ed803d7dd/8577c4222d3f528dffdf240ed803d7dd_1717347235.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/aec/0579a4d6692b95f48f056c034531aaec/0579a4d6692b95f48f056c034531aaec.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347239.00000-0, db: /tmp/tmp020hq008/sda/containers/0/aec/0579a4d6692b95f48f056c034531aaec/0579a4d6692b95f48f056c034531aaec_1717347239.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/df7/c00a00be6de65f597e910ebb4e958df7/c00a00be6de65f597e910ebb4e958df7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347243.00000-0, db: /tmp/tmp020hq008/sda/containers/0/df7/c00a00be6de65f597e910ebb4e958df7/c00a00be6de65f597e910ebb4e958df7_1717347243.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f36/a6fdfe3b1e889bb8982a2e1b707d4f36/a6fdfe3b1e889bb8982a2e1b707d4f36.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347247.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f36/a6fdfe3b1e889bb8982a2e1b707d4f36/a6fdfe3b1e889bb8982a2e1b707d4f36_1717347247.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/3fd/7751ae5ee3588174f058cdf892da93fd/7751ae5ee3588174f058cdf892da93fd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347251.00000-0, db: /tmp/tmp020hq008/sda/containers/0/3fd/7751ae5ee3588174f058cdf892da93fd/7751ae5ee3588174f058cdf892da93fd_1717347251.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/04b/521252a77ebf5a7f06960774f821804b/521252a77ebf5a7f06960774f821804b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347255.00000-0, db: /tmp/tmp020hq008/sda/containers/0/04b/521252a77ebf5a7f06960774f821804b/521252a77ebf5a7f06960774f821804b_1717347255.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f31/22b3f04cfacc35be1397a6982ed8bf31/22b3f04cfacc35be1397a6982ed8bf31.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347259.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f31/22b3f04cfacc35be1397a6982ed8bf31/22b3f04cfacc35be1397a6982ed8bf31_1717347259.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/02a/2378f0e35f4619c53ff2d0ba3be7802a/2378f0e35f4619c53ff2d0ba3be7802a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347263.00000-0, db: /tmp/tmp020hq008/sda/containers/0/02a/2378f0e35f4619c53ff2d0ba3be7802a/2378f0e35f4619c53ff2d0ba3be7802a_1717347263.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b69/82e22ad90ffdc866aa2f1d347b2fbb69/82e22ad90ffdc866aa2f1d347b2fbb69.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347267.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b69/82e22ad90ffdc866aa2f1d347b2fbb69/82e22ad90ffdc866aa2f1d347b2fbb69_1717347267.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d48/ca9cc2335300ff7749b4d62fa2484d48/ca9cc2335300ff7749b4d62fa2484d48.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347271.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d48/ca9cc2335300ff7749b4d62fa2484d48/ca9cc2335300ff7749b4d62fa2484d48_1717347271.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d5b/e988b7fba418eb7d06f2d1de14cf8d5b/e988b7fba418eb7d06f2d1de14cf8d5b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347275.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d5b/e988b7fba418eb7d06f2d1de14cf8d5b/e988b7fba418eb7d06f2d1de14cf8d5b_1717347275.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/778/d87a1e9d0933d5e425cc6be10d38e778/d87a1e9d0933d5e425cc6be10d38e778.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347279.00000-0, db: /tmp/tmp020hq008/sda/containers/0/778/d87a1e9d0933d5e425cc6be10d38e778/d87a1e9d0933d5e425cc6be10d38e778_1717347279.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/0a2/1d8ade84b040826192a108cb3d9f20a2/1d8ade84b040826192a108cb3d9f20a2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347283.00000-0, db: /tmp/tmp020hq008/sda/containers/0/0a2/1d8ade84b040826192a108cb3d9f20a2/1d8ade84b040826192a108cb3d9f20a2_1717347283.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d00/06f5263ae44816bc1ebd58b179d87d00/06f5263ae44816bc1ebd58b179d87d00.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347287.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d00/06f5263ae44816bc1ebd58b179d87d00/06f5263ae44816bc1ebd58b179d87d00_1717347287.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmp020hq008/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104_1717347291.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b04/a52ff746370192c0b7092fb39e323b04/a52ff746370192c0b7092fb39e323b04.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347295.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b04/a52ff746370192c0b7092fb39e323b04/a52ff746370192c0b7092fb39e323b04_1717347295.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f25/4bd3560552e9e648eae2a1f24a71ef25/4bd3560552e9e648eae2a1f24a71ef25.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347299.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f25/4bd3560552e9e648eae2a1f24a71ef25/4bd3560552e9e648eae2a1f24a71ef25_1717347299.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ab4/7d2af95f451fea4d55b5f950c2efcab4/7d2af95f451fea4d55b5f950c2efcab4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347303.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ab4/7d2af95f451fea4d55b5f950c2efcab4/7d2af95f451fea4d55b5f950c2efcab4_1717347303.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/87f/f4cc0bc2dd03e42d68f44b648134f87f/f4cc0bc2dd03e42d68f44b648134f87f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347307.00000-0, db: /tmp/tmp020hq008/sda/containers/0/87f/f4cc0bc2dd03e42d68f44b648134f87f/f4cc0bc2dd03e42d68f44b648134f87f_1717347307.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/a21/b95f4c519c69274b073697f12bc79a21/b95f4c519c69274b073697f12bc79a21.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347311.00000-0, db: /tmp/tmp020hq008/sda/containers/0/a21/b95f4c519c69274b073697f12bc79a21/b95f4c519c69274b073697f12bc79a21_1717347311.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/db2/b4bb66fd1dc808f7353ed339a3ea3db2/b4bb66fd1dc808f7353ed339a3ea3db2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347315.00000-0, db: /tmp/tmp020hq008/sda/containers/0/db2/b4bb66fd1dc808f7353ed339a3ea3db2/b4bb66fd1dc808f7353ed339a3ea3db2_1717347315.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7ad/ede077a8091e939457ef3dbc791087ad/ede077a8091e939457ef3dbc791087ad.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347319.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7ad/ede077a8091e939457ef3dbc791087ad/ede077a8091e939457ef3dbc791087ad_1717347319.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/5b8/e290f8ce5fb11abd378b211bc49075b8/e290f8ce5fb11abd378b211bc49075b8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347323.00000-0, db: /tmp/tmp020hq008/sda/containers/0/5b8/e290f8ce5fb11abd378b211bc49075b8/e290f8ce5fb11abd378b211bc49075b8_1717347323.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347327.00000-0, db: /tmp/tmp020hq008/sda/containers/0/c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c_1717347327.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/015/d3562d2e4c11fb2711fd44745c331015/d3562d2e4c11fb2711fd44745c331015.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347331.00000-0, db: /tmp/tmp020hq008/sda/containers/0/015/d3562d2e4c11fb2711fd44745c331015/d3562d2e4c11fb2711fd44745c331015_1717347331.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7b4/b2dadd87d7e78a6ebaebf985188d97b4/b2dadd87d7e78a6ebaebf985188d97b4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347335.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7b4/b2dadd87d7e78a6ebaebf985188d97b4/b2dadd87d7e78a6ebaebf985188d97b4_1717347335.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/cec/8229efdb4612b4fc8338fc4fc8db1cec/8229efdb4612b4fc8338fc4fc8db1cec.db' test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347339.00000-0, db: /tmp/tmp020hq008/sda/containers/0/cec/8229efdb4612b4fc8338fc4fc8db1cec/8229efdb4612b4fc8338fc4fc8db1cec_1717347339.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/595/a3a3aba0b0fd14e227e3f0838245b595/a3a3aba0b0fd14e227e3f0838245b595.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347343.00000-0, db: /tmp/tmp020hq008/sda/containers/0/595/a3a3aba0b0fd14e227e3f0838245b595/a3a3aba0b0fd14e227e3f0838245b595_1717347343.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/962/982f03db02fd044a060995bf51f45962/982f03db02fd044a060995bf51f45962.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347347.00000-0, db: /tmp/tmp020hq008/sda/containers/0/962/982f03db02fd044a060995bf51f45962/982f03db02fd044a060995bf51f45962_1717347347.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/07c/98151d85b578b8be96f86b27962e607c/98151d85b578b8be96f86b27962e607c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347351.00000-0, db: /tmp/tmp020hq008/sda/containers/0/07c/98151d85b578b8be96f86b27962e607c/98151d85b578b8be96f86b27962e607c_1717347351.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/00e/11c04ecf265d901f0017a92f5d88500e/11c04ecf265d901f0017a92f5d88500e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347355.00000-0, db: /tmp/tmp020hq008/sda/containers/0/00e/11c04ecf265d901f0017a92f5d88500e/11c04ecf265d901f0017a92f5d88500e_1717347355.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/fec/0b1f632ba313e67ee6d4e7b387971fec/0b1f632ba313e67ee6d4e7b387971fec.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347359.00000-0, db: /tmp/tmp020hq008/sda/containers/0/fec/0b1f632ba313e67ee6d4e7b387971fec/0b1f632ba313e67ee6d4e7b387971fec_1717347359.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/96a/131b82fe320055ef940216a46b44996a/131b82fe320055ef940216a46b44996a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347363.00000-0, db: /tmp/tmp020hq008/sda/containers/0/96a/131b82fe320055ef940216a46b44996a/131b82fe320055ef940216a46b44996a_1717347363.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/04d/bb2186c7cda0b67e85cb7530b7b4304d/bb2186c7cda0b67e85cb7530b7b4304d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347367.00000-0, db: /tmp/tmp020hq008/sda/containers/0/04d/bb2186c7cda0b67e85cb7530b7b4304d/bb2186c7cda0b67e85cb7530b7b4304d_1717347367.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/01c/042976de285ca25e79377d6018f6a01c/042976de285ca25e79377d6018f6a01c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347371.00000-0, db: /tmp/tmp020hq008/sda/containers/0/01c/042976de285ca25e79377d6018f6a01c/042976de285ca25e79377d6018f6a01c_1717347371.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/46f/2adf6cdb550cf95afc0ced92e244246f/2adf6cdb550cf95afc0ced92e244246f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347375.00000-0, db: /tmp/tmp020hq008/sda/containers/0/46f/2adf6cdb550cf95afc0ced92e244246f/2adf6cdb550cf95afc0ced92e244246f_1717347375.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ba4/00c6cee04336037ac418e04665850ba4/00c6cee04336037ac418e04665850ba4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347379.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ba4/00c6cee04336037ac418e04665850ba4/00c6cee04336037ac418e04665850ba4_1717347379.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d20/8425aa094b48f6998289f0a154df3d20/8425aa094b48f6998289f0a154df3d20.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347383.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d20/8425aa094b48f6998289f0a154df3d20/8425aa094b48f6998289f0a154df3d20_1717347383.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/36e/9f3f4f5210f894f16271615b5610b36e/9f3f4f5210f894f16271615b5610b36e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347387.00000-0, db: /tmp/tmp020hq008/sda/containers/0/36e/9f3f4f5210f894f16271615b5610b36e/9f3f4f5210f894f16271615b5610b36e_1717347387.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/4f5/776a39fcf13ae439e800e99a41f964f5/776a39fcf13ae439e800e99a41f964f5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347391.00000-0, db: /tmp/tmp020hq008/sda/containers/0/4f5/776a39fcf13ae439e800e99a41f964f5/776a39fcf13ae439e800e99a41f964f5_1717347391.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/a21/7ffd4dc2310fc40b8278ceef489f7a21/7ffd4dc2310fc40b8278ceef489f7a21.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347395.00000-0, db: /tmp/tmp020hq008/sda/containers/0/a21/7ffd4dc2310fc40b8278ceef489f7a21/7ffd4dc2310fc40b8278ceef489f7a21_1717347395.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/0fb/db275794d93b73f782f2b55ea7d760fb/db275794d93b73f782f2b55ea7d760fb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347399.00000-0, db: /tmp/tmp020hq008/sda/containers/0/0fb/db275794d93b73f782f2b55ea7d760fb/db275794d93b73f782f2b55ea7d760fb_1717347399.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d92/ce20794daa65b895d7a251f27d1f0d92/ce20794daa65b895d7a251f27d1f0d92.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347403.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d92/ce20794daa65b895d7a251f27d1f0d92/ce20794daa65b895d7a251f27d1f0d92_1717347403.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/fa7/223cdc8a6b7d84ac24cbb77d0944bfa7/223cdc8a6b7d84ac24cbb77d0944bfa7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347407.00000-0, db: /tmp/tmp020hq008/sda/containers/0/fa7/223cdc8a6b7d84ac24cbb77d0944bfa7/223cdc8a6b7d84ac24cbb77d0944bfa7_1717347407.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f46/9ebb1d40fb8411967f98e5d9e0b91f46/9ebb1d40fb8411967f98e5d9e0b91f46.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347411.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f46/9ebb1d40fb8411967f98e5d9e0b91f46/9ebb1d40fb8411967f98e5d9e0b91f46_1717347411.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/96d/c2206d3cefaf73a2e6d8fce9cfb4796d/c2206d3cefaf73a2e6d8fce9cfb4796d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347415.00000-0, db: /tmp/tmp020hq008/sda/containers/0/96d/c2206d3cefaf73a2e6d8fce9cfb4796d/c2206d3cefaf73a2e6d8fce9cfb4796d_1717347415.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7d4/1ef874b91da954bb0307486f013b37d4/1ef874b91da954bb0307486f013b37d4.db' test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347419.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7d4/1ef874b91da954bb0307486f013b37d4/1ef874b91da954bb0307486f013b37d4_1717347419.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/c96/15b3839964a3ccf81dd44afa704f6c96/15b3839964a3ccf81dd44afa704f6c96.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347423.00000-0, db: /tmp/tmp020hq008/sda/containers/0/c96/15b3839964a3ccf81dd44afa704f6c96/15b3839964a3ccf81dd44afa704f6c96_1717347423.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/db2/eb8aba642f81fcc271ed4019a5572db2/eb8aba642f81fcc271ed4019a5572db2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347427.00000-0, db: /tmp/tmp020hq008/sda/containers/0/db2/eb8aba642f81fcc271ed4019a5572db2/eb8aba642f81fcc271ed4019a5572db2_1717347427.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/c36/b4f2063855b74c713b6b6f78accabc36/b4f2063855b74c713b6b6f78accabc36.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347431.00000-0, db: /tmp/tmp020hq008/sda/containers/0/c36/b4f2063855b74c713b6b6f78accabc36/b4f2063855b74c713b6b6f78accabc36_1717347431.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/831/ab3589427446e2368ada995d76443831/ab3589427446e2368ada995d76443831.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347435.00000-0, db: /tmp/tmp020hq008/sda/containers/0/831/ab3589427446e2368ada995d76443831/ab3589427446e2368ada995d76443831_1717347435.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/00c/77f321d247988933fe7348605f9e900c/77f321d247988933fe7348605f9e900c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347439.00000-0, db: /tmp/tmp020hq008/sda/containers/0/00c/77f321d247988933fe7348605f9e900c/77f321d247988933fe7348605f9e900c_1717347439.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f1f/8145c71ded8f887d0116bbdebcba6f1f/8145c71ded8f887d0116bbdebcba6f1f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347443.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f1f/8145c71ded8f887d0116bbdebcba6f1f/8145c71ded8f887d0116bbdebcba6f1f_1717347443.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/41b/3bd8c097bb71ea45c5ed86e85b37541b/3bd8c097bb71ea45c5ed86e85b37541b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347447.00000-0, db: /tmp/tmp020hq008/sda/containers/0/41b/3bd8c097bb71ea45c5ed86e85b37541b/3bd8c097bb71ea45c5ed86e85b37541b_1717347447.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f3a/ff242e03b700be3fc02edd358a06bf3a/ff242e03b700be3fc02edd358a06bf3a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347451.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f3a/ff242e03b700be3fc02edd358a06bf3a/ff242e03b700be3fc02edd358a06bf3a_1717347451.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/3df/bbe07289a8ed0ac9e8c9fa7c0ac3a3df/bbe07289a8ed0ac9e8c9fa7c0ac3a3df.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347455.00000-0, db: /tmp/tmp020hq008/sda/containers/0/3df/bbe07289a8ed0ac9e8c9fa7c0ac3a3df/bbe07289a8ed0ac9e8c9fa7c0ac3a3df_1717347455.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/478/a3c41931b982f9d65de0c0c344d9c478/a3c41931b982f9d65de0c0c344d9c478.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347459.00000-0, db: /tmp/tmp020hq008/sda/containers/0/478/a3c41931b982f9d65de0c0c344d9c478/a3c41931b982f9d65de0c0c344d9c478_1717347459.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/0e8/c8444b8a46c7aab934fdbabcbe6f70e8/c8444b8a46c7aab934fdbabcbe6f70e8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347463.00000-0, db: /tmp/tmp020hq008/sda/containers/0/0e8/c8444b8a46c7aab934fdbabcbe6f70e8/c8444b8a46c7aab934fdbabcbe6f70e8_1717347463.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/0a7/03485bcd78f4f1b06f4e7665427a00a7/03485bcd78f4f1b06f4e7665427a00a7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347467.00000-0, db: /tmp/tmp020hq008/sda/containers/0/0a7/03485bcd78f4f1b06f4e7665427a00a7/03485bcd78f4f1b06f4e7665427a00a7_1717347467.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/bdf/be189244488ca5068644e982de7aebdf/be189244488ca5068644e982de7aebdf.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347471.00000-0, db: /tmp/tmp020hq008/sda/containers/0/bdf/be189244488ca5068644e982de7aebdf/be189244488ca5068644e982de7aebdf_1717347471.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/140/ccf87331728d789684a244966ca67140/ccf87331728d789684a244966ca67140.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347475.00000-0, db: /tmp/tmp020hq008/sda/containers/0/140/ccf87331728d789684a244966ca67140/ccf87331728d789684a244966ca67140_1717347475.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/cc9/0d5d4e030b0015293df6780f3f36fcc9/0d5d4e030b0015293df6780f3f36fcc9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347479.00000-0, db: /tmp/tmp020hq008/sda/containers/0/cc9/0d5d4e030b0015293df6780f3f36fcc9/0d5d4e030b0015293df6780f3f36fcc9_1717347479.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f1a/ec7ab3a5d89ecdb455fb3791686c6f1a/ec7ab3a5d89ecdb455fb3791686c6f1a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347483.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f1a/ec7ab3a5d89ecdb455fb3791686c6f1a/ec7ab3a5d89ecdb455fb3791686c6f1a_1717347483.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/0a3/0f5a22e8a91b0ebe304e46d2cec7f0a3/0f5a22e8a91b0ebe304e46d2cec7f0a3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347487.00000-0, db: /tmp/tmp020hq008/sda/containers/0/0a3/0f5a22e8a91b0ebe304e46d2cec7f0a3/0f5a22e8a91b0ebe304e46d2cec7f0a3_1717347487.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/a73/74e53c5ecd37dd833fd159ed2cfb6a73/74e53c5ecd37dd833fd159ed2cfb6a73.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347491.00000-0, db: /tmp/tmp020hq008/sda/containers/0/a73/74e53c5ecd37dd833fd159ed2cfb6a73/74e53c5ecd37dd833fd159ed2cfb6a73_1717347491.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f04/aa59e7f54b84e91f8471710529e40f04/aa59e7f54b84e91f8471710529e40f04.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347495.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f04/aa59e7f54b84e91f8471710529e40f04/aa59e7f54b84e91f8471710529e40f04_1717347495.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/289/99a0c0683ae232e5d57b4a92278a2289/99a0c0683ae232e5d57b4a92278a2289.db' test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347499.00000-0, db: /tmp/tmp020hq008/sda/containers/0/289/99a0c0683ae232e5d57b4a92278a2289/99a0c0683ae232e5d57b4a92278a2289_1717347499.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b38/41d46c99be4c666dfdafde0454f4fb38/41d46c99be4c666dfdafde0454f4fb38.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347503.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b38/41d46c99be4c666dfdafde0454f4fb38/41d46c99be4c666dfdafde0454f4fb38_1717347503.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/e39/ffe0340df4c128e78a34f78f4363ce39/ffe0340df4c128e78a34f78f4363ce39.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347507.00000-0, db: /tmp/tmp020hq008/sda/containers/0/e39/ffe0340df4c128e78a34f78f4363ce39/ffe0340df4c128e78a34f78f4363ce39_1717347507.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/09a/fbbd3e4526333b1a9174e1bd1419d09a/fbbd3e4526333b1a9174e1bd1419d09a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347511.00000-0, db: /tmp/tmp020hq008/sda/containers/0/09a/fbbd3e4526333b1a9174e1bd1419d09a/fbbd3e4526333b1a9174e1bd1419d09a_1717347511.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b6b/b4bf65b97c1a71191182ae2b29284b6b/b4bf65b97c1a71191182ae2b29284b6b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347515.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b6b/b4bf65b97c1a71191182ae2b29284b6b/b4bf65b97c1a71191182ae2b29284b6b_1717347515.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/da7/bae9f77600ce19b3a42d9227fc167da7/bae9f77600ce19b3a42d9227fc167da7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347519.00000-0, db: /tmp/tmp020hq008/sda/containers/0/da7/bae9f77600ce19b3a42d9227fc167da7/bae9f77600ce19b3a42d9227fc167da7_1717347519.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/788/cdef4c82c26c023cc4d1e5a4e5649788/cdef4c82c26c023cc4d1e5a4e5649788.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347523.00000-0, db: /tmp/tmp020hq008/sda/containers/0/788/cdef4c82c26c023cc4d1e5a4e5649788/cdef4c82c26c023cc4d1e5a4e5649788_1717347523.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/929/d25c6356e6b6a144c7493dba1928c929/d25c6356e6b6a144c7493dba1928c929.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347527.00000-0, db: /tmp/tmp020hq008/sda/containers/0/929/d25c6356e6b6a144c7493dba1928c929/d25c6356e6b6a144c7493dba1928c929_1717347527.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ebe/3685f67842e65c80bbf136b17d56cebe/3685f67842e65c80bbf136b17d56cebe.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347531.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ebe/3685f67842e65c80bbf136b17d56cebe/3685f67842e65c80bbf136b17d56cebe_1717347531.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/e19/8f608ea7ba9bac808afae7d90ab95e19/8f608ea7ba9bac808afae7d90ab95e19.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347535.00000-0, db: /tmp/tmp020hq008/sda/containers/0/e19/8f608ea7ba9bac808afae7d90ab95e19/8f608ea7ba9bac808afae7d90ab95e19_1717347535.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/140/68fcf0dce00adff6f7a2ec651c869140/68fcf0dce00adff6f7a2ec651c869140.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347539.00000-0, db: /tmp/tmp020hq008/sda/containers/0/140/68fcf0dce00adff6f7a2ec651c869140/68fcf0dce00adff6f7a2ec651c869140_1717347539.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/d33/47a7e5c5e517f66c8708650aaf9b7d33/47a7e5c5e517f66c8708650aaf9b7d33.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347543.00000-0, db: /tmp/tmp020hq008/sda/containers/0/d33/47a7e5c5e517f66c8708650aaf9b7d33/47a7e5c5e517f66c8708650aaf9b7d33_1717347543.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/a53/bc496b85734183931805046611a25a53/bc496b85734183931805046611a25a53.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347547.00000-0, db: /tmp/tmp020hq008/sda/containers/0/a53/bc496b85734183931805046611a25a53/bc496b85734183931805046611a25a53_1717347547.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ae5/32301a34a576c1c5bb1be6efe9a16ae5/32301a34a576c1c5bb1be6efe9a16ae5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347551.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ae5/32301a34a576c1c5bb1be6efe9a16ae5/32301a34a576c1c5bb1be6efe9a16ae5_1717347551.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/337/5f9cffd8ede9042707fc31c244241337/5f9cffd8ede9042707fc31c244241337.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347555.00000-0, db: /tmp/tmp020hq008/sda/containers/0/337/5f9cffd8ede9042707fc31c244241337/5f9cffd8ede9042707fc31c244241337_1717347555.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/8f7/58f1edd740f3cbfbe5fcfd0328ffc8f7/58f1edd740f3cbfbe5fcfd0328ffc8f7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347559.00000-0, db: /tmp/tmp020hq008/sda/containers/0/8f7/58f1edd740f3cbfbe5fcfd0328ffc8f7/58f1edd740f3cbfbe5fcfd0328ffc8f7_1717347559.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/af2/d9b039976713995d03c4fa35f10fdaf2/d9b039976713995d03c4fa35f10fdaf2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347563.00000-0, db: /tmp/tmp020hq008/sda/containers/0/af2/d9b039976713995d03c4fa35f10fdaf2/d9b039976713995d03c4fa35f10fdaf2_1717347563.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/310/f1dfa5c9105f569366e7466f0d7ca310/f1dfa5c9105f569366e7466f0d7ca310.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347567.00000-0, db: /tmp/tmp020hq008/sda/containers/0/310/f1dfa5c9105f569366e7466f0d7ca310/f1dfa5c9105f569366e7466f0d7ca310_1717347567.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347571.00000-0, db: /tmp/tmp020hq008/sda/containers/0/bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd/23a30b8a59a66c2ac83ae07a9f0e6bdd_1717347571.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/9c0/3c9103365b9f07c97ada0f0bcdd9b9c0/3c9103365b9f07c97ada0f0bcdd9b9c0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347575.00000-0, db: /tmp/tmp020hq008/sda/containers/0/9c0/3c9103365b9f07c97ada0f0bcdd9b9c0/3c9103365b9f07c97ada0f0bcdd9b9c0_1717347575.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/96c/acc4be3c2a5435a70ee5e100715ac96c/acc4be3c2a5435a70ee5e100715ac96c.db' test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx988fcf1db3264b79bbe2c-00665ca36d) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe95f274e4ad84e2e89c53-00665ca36d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Setting object count to 1065 (txn: tx0604659a9dce46b490cb7-00665ca36f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Setting object count to 1065 (txn: txe87e199e5e7f4002b9e3f-00665ca372) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx39ca58faa1364bea866f0-00665ca374) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: tx39ca58faa1364bea866f0-00665ca374) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: txa1297241b8af4f8993dcc-00665ca377) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error ... ok test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: txa1297241b8af4f8993dcc-00665ca377) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txa1297241b8af4f8993dcc-00665ca377) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: txa1297241b8af4f8993dcc-00665ca377) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx1d73f52d4ad643c0b1e38-00665ca377) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1d73f52d4ad643c0b1e38-00665ca377) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1d73f52d4ad643c0b1e38-00665ca377) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: tx1d73f52d4ad643c0b1e38-00665ca377) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: txbe7eb2b5c6c7406781bcf-00665ca377) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbe7eb2b5c6c7406781bcf-00665ca377) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txbe7eb2b5c6c7406781bcf-00665ca377) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: txbe7eb2b5c6c7406781bcf-00665ca377) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: Setting object count to 3120 (txn: tx5dc34aa656984ecb9e663-00665ca379) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: Setting object count to 3120 (txn: txf9bccc778fbb43be939a0-00665ca379) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: Setting object count to 3120 (txn: tx6f06763bbf3346dfbbce1-00665ca379) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: Setting object count to 3120 (txn: tx2a1620d956e14b0d848ce-00665ca379) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: Setting object count to 3120 (txn: txb3a4595bb7bf4fe3a58f3-00665ca379) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347579.00000-0, db: /tmp/tmp020hq008/sda/containers/0/96c/acc4be3c2a5435a70ee5e100715ac96c/acc4be3c2a5435a70ee5e100715ac96c_1717347579.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f22/ee6ab8bc025e8ba0946a428867430f22/ee6ab8bc025e8ba0946a428867430f22.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347583.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f22/ee6ab8bc025e8ba0946a428867430f22/ee6ab8bc025e8ba0946a428867430f22_1717347583.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/0d2/730115b1f8bafac52ebf0b3d880ed0d2/730115b1f8bafac52ebf0b3d880ed0d2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347587.00000-0, db: /tmp/tmp020hq008/sda/containers/0/0d2/730115b1f8bafac52ebf0b3d880ed0d2/730115b1f8bafac52ebf0b3d880ed0d2_1717347587.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/9dc/c4990f853103b5378ab8adce6c5799dc/c4990f853103b5378ab8adce6c5799dc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347591.00000-0, db: /tmp/tmp020hq008/sda/containers/0/9dc/c4990f853103b5378ab8adce6c5799dc/c4990f853103b5378ab8adce6c5799dc_1717347591.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/927/cfb45dd245e2aafe51f405678f957927/cfb45dd245e2aafe51f405678f957927.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347595.00000-0, db: /tmp/tmp020hq008/sda/containers/0/927/cfb45dd245e2aafe51f405678f957927/cfb45dd245e2aafe51f405678f957927_1717347595.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/78e/6d41d2ba87c5329e184ef9cfb14ce78e/6d41d2ba87c5329e184ef9cfb14ce78e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347599.00000-0, db: /tmp/tmp020hq008/sda/containers/0/78e/6d41d2ba87c5329e184ef9cfb14ce78e/6d41d2ba87c5329e184ef9cfb14ce78e_1717347599.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/400/6fb2dffc3e38f009cd93bfa020eed400/6fb2dffc3e38f009cd93bfa020eed400.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347603.00000-0, db: /tmp/tmp020hq008/sda/containers/0/400/6fb2dffc3e38f009cd93bfa020eed400/6fb2dffc3e38f009cd93bfa020eed400_1717347603.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ccc/b8d05aee727bb6f6a1aeacb6fb3ccccc/b8d05aee727bb6f6a1aeacb6fb3ccccc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347607.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ccc/b8d05aee727bb6f6a1aeacb6fb3ccccc/b8d05aee727bb6f6a1aeacb6fb3ccccc_1717347607.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/c15/7a1b1890283c6d39e2b2a8372f554c15/7a1b1890283c6d39e2b2a8372f554c15.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347611.00000-0, db: /tmp/tmp020hq008/sda/containers/0/c15/7a1b1890283c6d39e2b2a8372f554c15/7a1b1890283c6d39e2b2a8372f554c15_1717347611.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/eee/615f85f1d7a071cefbd4654669b7beee/615f85f1d7a071cefbd4654669b7beee.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347615.00000-0, db: /tmp/tmp020hq008/sda/containers/0/eee/615f85f1d7a071cefbd4654669b7beee/615f85f1d7a071cefbd4654669b7beee_1717347615.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347619.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8_1717347619.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7e9/bd897a1ab75e252129a04cb6f134d7e9/bd897a1ab75e252129a04cb6f134d7e9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347623.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7e9/bd897a1ab75e252129a04cb6f134d7e9/bd897a1ab75e252129a04cb6f134d7e9_1717347623.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7e6/8f813e3ffa71e500721ac7a7309197e6/8f813e3ffa71e500721ac7a7309197e6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347627.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7e6/8f813e3ffa71e500721ac7a7309197e6/8f813e3ffa71e500721ac7a7309197e6_1717347627.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/823/272092cf742dfbeef04f71672e31d823/272092cf742dfbeef04f71672e31d823.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347631.00000-0, db: /tmp/tmp020hq008/sda/containers/0/823/272092cf742dfbeef04f71672e31d823/272092cf742dfbeef04f71672e31d823_1717347631.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/bc2/846d5e69d8b9bf77558085c3f5cc0bc2/846d5e69d8b9bf77558085c3f5cc0bc2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347635.00000-0, db: /tmp/tmp020hq008/sda/containers/0/bc2/846d5e69d8b9bf77558085c3f5cc0bc2/846d5e69d8b9bf77558085c3f5cc0bc2_1717347635.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/ea1/1f99b787458c5c33f5803b31cdb8bea1/1f99b787458c5c33f5803b31cdb8bea1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347639.00000-0, db: /tmp/tmp020hq008/sda/containers/0/ea1/1f99b787458c5c33f5803b31cdb8bea1/1f99b787458c5c33f5803b31cdb8bea1_1717347639.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/edc/4d7ad7abd294928b3ea16b25c810cedc/4d7ad7abd294928b3ea16b25c810cedc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347643.00000-0, db: /tmp/tmp020hq008/sda/containers/0/edc/4d7ad7abd294928b3ea16b25c810cedc/4d7ad7abd294928b3ea16b25c810cedc_1717347643.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347647.00000-0, db: /tmp/tmp020hq008/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8_1717347647.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmp020hq008/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3_1717347651.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b50/6af70ffeca5372ca4cd9d87b84614b50/6af70ffeca5372ca4cd9d87b84614b50.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347655.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b50/6af70ffeca5372ca4cd9d87b84614b50/6af70ffeca5372ca4cd9d87b84614b50_1717347655.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/8e4/c80963a7262b16b973d3564b7855d8e4/c80963a7262b16b973d3564b7855d8e4.db' test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347659.00000-0, db: /tmp/tmp020hq008/sda/containers/0/8e4/c80963a7262b16b973d3564b7855d8e4/c80963a7262b16b973d3564b7855d8e4_1717347659.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/b16/374952945e317b566e0211f2168adb16/374952945e317b566e0211f2168adb16.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347663.00000-0, db: /tmp/tmp020hq008/sda/containers/0/b16/374952945e317b566e0211f2168adb16/374952945e317b566e0211f2168adb16_1717347663.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/160/d5d4b98b5cade45bbaa678858ce60160/d5d4b98b5cade45bbaa678858ce60160.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347667.00000-0, db: /tmp/tmp020hq008/sda/containers/0/160/d5d4b98b5cade45bbaa678858ce60160/d5d4b98b5cade45bbaa678858ce60160_1717347667.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/474/5dd5386cac23f769594971f1284ae474/5dd5386cac23f769594971f1284ae474.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347671.00000-0, db: /tmp/tmp020hq008/sda/containers/0/474/5dd5386cac23f769594971f1284ae474/5dd5386cac23f769594971f1284ae474_1717347671.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmp020hq008/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0_1717347675.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/77f/410c026e68360671f267262b98eb577f/410c026e68360671f267262b98eb577f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347679.00000-0, db: /tmp/tmp020hq008/sda/containers/0/77f/410c026e68360671f267262b98eb577f/410c026e68360671f267262b98eb577f_1717347679.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/f3f/8e13d20812695cd519e01cf4029e2f3f/8e13d20812695cd519e01cf4029e2f3f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347683.00000-0, db: /tmp/tmp020hq008/sda/containers/0/f3f/8e13d20812695cd519e01cf4029e2f3f/8e13d20812695cd519e01cf4029e2f3f_1717347683.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/7a5/54dc622594c35edea8f581205cbbf7a5/54dc622594c35edea8f581205cbbf7a5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347687.00000-0, db: /tmp/tmp020hq008/sda/containers/0/7a5/54dc622594c35edea8f581205cbbf7a5/54dc622594c35edea8f581205cbbf7a5_1717347687.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp020hq008/sda/containers/0/223/9562307612d5aeab22c4e29141429223/9562307612d5aeab22c4e29141429223.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347691.00000-0, db: /tmp/tmp020hq008/sda/containers/0/223/9562307612d5aeab22c4e29141429223/9562307612d5aeab22c4e29141429223_1717347691.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347198.00000-0, db: /tmp/tmp_yy3514z/sda/containers/0/f79/be145ba856acc07f2de1e7e6d684ff79/be145ba856acc07f2de1e7e6d684ff79.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmp_yy3514z/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmp_yy3514z/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmp_yy3514z/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmpb_qtrlcb/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmpb_qtrlcb/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347205.00000-0, db: /tmp/tmpb_qtrlcb/sda/containers/0/819/4303aa522a892bb737933448b0edd819/4303aa522a892bb737933448b0edd819.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347208.00000-0, db: /tmp/tmpb_qtrlcb/sda/containers/0/030/987dbb3f91948969290126a3a4e19030/987dbb3f91948969290126a3a4e19030.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmporgbad10/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmporgbad10/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmporgbad10/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347203.00000-0, db: /tmp/tmporgbad10/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347203.00000-0, db: /tmp/tmporgbad10/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347203.00000-0, db: /tmp/tmporgbad10/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmporgbad10/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmporgbad10/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmporgbad10/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmporgbad10/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmporgbad10/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving ... ok sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347211.00000-0, db: /tmp/tmporgbad10/sda/containers/0/dc6/6f6ac0de35241dcde43cf7f81b873dc6/6f6ac0de35241dcde43cf7f81b873dc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347199.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/4ca/ef6f5df386b02a0156e4f87ccfc6e4ca/ef6f5df386b02a0156e4f87ccfc6e4ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347205.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/819/4303aa522a892bb737933448b0edd819/4303aa522a892bb737933448b0edd819.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347205.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/819/4303aa522a892bb737933448b0edd819/4303aa522a892bb737933448b0edd819.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347208.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/030/987dbb3f91948969290126a3a4e19030/987dbb3f91948969290126a3a4e19030.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347208.00000-0, db: /tmp/tmpu5i33qz1/sda/containers/0/030/987dbb3f91948969290126a3a4e19030/987dbb3f91948969290126a3a4e19030.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347200.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/839/ca32f0bad11a353bc320bc462f679839/ca32f0bad11a353bc320bc462f679839.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347200.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/839/ca32f0bad11a353bc320bc462f679839/ca32f0bad11a353bc320bc462f679839.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347203.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347203.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/d67/270da084194f8c713ff5f5e30dc45d67/270da084194f8c713ff5f5e30dc45d67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347209.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/cb2/420e34390634feb9deedc1658f158cb2/420e34390634feb9deedc1658f158cb2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347209.00000-0, db: /tmp/tmp2lr_16v_/sda/containers/0/cb2/420e34390634feb9deedc1658f158cb2/420e34390634feb9deedc1658f158cb2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347200.00000-0, db: /tmp/tmprgp2c39j/sda/containers/0/839/ca32f0bad11a353bc320bc462f679839/ca32f0bad11a353bc320bc462f679839.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347202.00000-0, db: /tmp/tmprgp2c39j/sda/containers/0/343/4a8b1bd0037d5db408d951632631e343/4a8b1bd0037d5db408d951632631e343.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347204.00000-0, db: /tmp/tmprgp2c39j/sda/containers/0/4d3/fc8f5bf8f660cb78da09caf6faac54d3/fc8f5bf8f660cb78da09caf6faac54d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347206.00000-0, db: /tmp/tmprgp2c39j/sda/containers/0/399/979b2ad8f3df771bd9ede110baf36399/979b2ad8f3df771bd9ede110baf36399.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347201.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/a73/ba77b742b64680e0c5d093ec0937fa73/ba77b742b64680e0c5d093ec0937fa73.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347201.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/a73/ba77b742b64680e0c5d093ec0937fa73/ba77b742b64680e0c5d093ec0937fa73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347204.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/4d3/fc8f5bf8f660cb78da09caf6faac54d3/fc8f5bf8f660cb78da09caf6faac54d3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347204.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/4d3/fc8f5bf8f660cb78da09caf6faac54d3/fc8f5bf8f660cb78da09caf6faac54d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347207.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/8aa/e96b2afe8a27dfc14b31c7a269a018aa/e96b2afe8a27dfc14b31c7a269a018aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347210.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/763/45115765ed2b68f94c0c53a98625a763/45115765ed2b68f94c0c53a98625a763.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347213.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/3bc/16db9adf175371ed8712e53cbc2ce3bc/16db9adf175371ed8712e53cbc2ce3bc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347213.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/3bc/16db9adf175371ed8712e53cbc2ce3bc/16db9adf175371ed8712e53cbc2ce3bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347216.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/a61/8991c328803ebd01f051e7ef22aa1a61/8991c328803ebd01f051e7ef22aa1a61.db test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx8bcc6f77245c4d5fa8e94-00665ca379) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx7eb9e7b4a52842ae9d269-00665ca379) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: txfdc26397b437421882e40-00665ca37a) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx2efa59063bdc41909ea50-00665ca37a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2efa59063bdc41909ea50-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2efa59063bdc41909ea50-00665ca37a) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx2efa59063bdc41909ea50-00665ca37a) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx1ef91efff3454859ab484-00665ca37a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx1ef91efff3454859ab484-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx1ef91efff3454859ab484-00665ca37a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx1ef91efff3454859ab484-00665ca37a) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx902d0a886d8e4e9eadff1-00665ca37a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx902d0a886d8e4e9eadff1-00665ca37a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx902d0a886d8e4e9eadff1-00665ca37a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx902d0a886d8e4e9eadff1-00665ca37a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: txafb77a3b863b4503a195a-00665ca37c) test DEBUG: Setting object count to 1068 (txn: txafb77a3b863b4503a195a-00665ca37c) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347219.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/33e/f50715e7e4da888a7a403d93ef5d933e/f50715e7e4da888a7a403d93ef5d933e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347219.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/33e/f50715e7e4da888a7a403d93ef5d933e/f50715e7e4da888a7a403d93ef5d933e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347222.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/482/3bc165f881aedbd2b423553496b14482/3bc165f881aedbd2b423553496b14482.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347222.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/482/3bc165f881aedbd2b423553496b14482/3bc165f881aedbd2b423553496b14482.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347225.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/570/a189990dd1593c1861c4d54a6925b570/a189990dd1593c1861c4d54a6925b570.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347225.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/570/a189990dd1593c1861c4d54a6925b570/a189990dd1593c1861c4d54a6925b570.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347228.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/906/f58109f54c63f9a53b744430e1dba906/f58109f54c63f9a53b744430e1dba906.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347228.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/906/f58109f54c63f9a53b744430e1dba906/f58109f54c63f9a53b744430e1dba906.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347231.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ae4/628406ac9bd8d2d09a0c1f418137eae4/628406ac9bd8d2d09a0c1f418137eae4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347234.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/c1f/f39933cb0b09682ee8bb0554640c8c1f/f39933cb0b09682ee8bb0554640c8c1f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347234.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/c1f/f39933cb0b09682ee8bb0554640c8c1f/f39933cb0b09682ee8bb0554640c8c1f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347237.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/bb1/09d45f0f99db59e44804209cbe007bb1/09d45f0f99db59e44804209cbe007bb1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347237.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/bb1/09d45f0f99db59e44804209cbe007bb1/09d45f0f99db59e44804209cbe007bb1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347240.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/d07/8462b1d830f66070f109037b39029d07/8462b1d830f66070f109037b39029d07.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347240.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/d07/8462b1d830f66070f109037b39029d07/8462b1d830f66070f109037b39029d07.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347243.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/df7/c00a00be6de65f597e910ebb4e958df7/c00a00be6de65f597e910ebb4e958df7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347243.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/df7/c00a00be6de65f597e910ebb4e958df7/c00a00be6de65f597e910ebb4e958df7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347246.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd/f9ee7ffe660dbf33de5f4eb856eb3dfd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347249.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/8d7/96f18f6b8df6044fed25f1de0a07f8d7/96f18f6b8df6044fed25f1de0a07f8d7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347249.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/8d7/96f18f6b8df6044fed25f1de0a07f8d7/96f18f6b8df6044fed25f1de0a07f8d7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347252.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/bd3/b6280feb989c820da91430dd7f1d0bd3/b6280feb989c820da91430dd7f1d0bd3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347252.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/bd3/b6280feb989c820da91430dd7f1d0bd3/b6280feb989c820da91430dd7f1d0bd3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347255.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/04b/521252a77ebf5a7f06960774f821804b/521252a77ebf5a7f06960774f821804b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347255.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/04b/521252a77ebf5a7f06960774f821804b/521252a77ebf5a7f06960774f821804b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347258.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/432/69acb38df4c85b8ea722733317ccb432/69acb38df4c85b8ea722733317ccb432.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347258.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/432/69acb38df4c85b8ea722733317ccb432/69acb38df4c85b8ea722733317ccb432.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347261.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/8ac/81e1169bc4e5fb82f1f14eb5ad7e78ac/81e1169bc4e5fb82f1f14eb5ad7e78ac.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347261.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/8ac/81e1169bc4e5fb82f1f14eb5ad7e78ac/81e1169bc4e5fb82f1f14eb5ad7e78ac.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347264.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ec7/bac4bc06c3b9745b2e448f664b31cec7/bac4bc06c3b9745b2e448f664b31cec7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347264.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ec7/bac4bc06c3b9745b2e448f664b31cec7/bac4bc06c3b9745b2e448f664b31cec7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347267.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/b69/82e22ad90ffdc866aa2f1d347b2fbb69/82e22ad90ffdc866aa2f1d347b2fbb69.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347267.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/b69/82e22ad90ffdc866aa2f1d347b2fbb69/82e22ad90ffdc866aa2f1d347b2fbb69.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347270.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/27c/57e1f02445fb7f62caa6b9f66d43a27c/57e1f02445fb7f62caa6b9f66d43a27c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347270.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/27c/57e1f02445fb7f62caa6b9f66d43a27c/57e1f02445fb7f62caa6b9f66d43a27c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347273.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/33d/8c83737c8f7cecbe96e7e3b5bbf2d33d/8c83737c8f7cecbe96e7e3b5bbf2d33d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347273.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/33d/8c83737c8f7cecbe96e7e3b5bbf2d33d/8c83737c8f7cecbe96e7e3b5bbf2d33d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347276.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/9a6/3e7f2050fe40b41d2f3c8e9dd05c39a6/3e7f2050fe40b41d2f3c8e9dd05c39a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347279.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/778/d87a1e9d0933d5e425cc6be10d38e778/d87a1e9d0933d5e425cc6be10d38e778.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347279.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/778/d87a1e9d0933d5e425cc6be10d38e778/d87a1e9d0933d5e425cc6be10d38e778.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347282.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/0b8/cc1af7f7dbec7aae69232417e0b890b8/cc1af7f7dbec7aae69232417e0b890b8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347282.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/0b8/cc1af7f7dbec7aae69232417e0b890b8/cc1af7f7dbec7aae69232417e0b890b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347285.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/20d/21453becc52d0fe6863eec62ef6ea20d/21453becc52d0fe6863eec62ef6ea20d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347285.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/20d/21453becc52d0fe6863eec62ef6ea20d/21453becc52d0fe6863eec62ef6ea20d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347288.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/370/22fb67f5943b6195582d689c6d5b9370/22fb67f5943b6195582d689c6d5b9370.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347288.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/370/22fb67f5943b6195582d689c6d5b9370/22fb67f5943b6195582d689c6d5b9370.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347291.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/104/e4400b5cb49704cb8c7bf8c92a25b104/e4400b5cb49704cb8c7bf8c92a25b104.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347294.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/fad/c314653a706a109dc59e420aa317efad/c314653a706a109dc59e420aa317efad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347294.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/fad/c314653a706a109dc59e420aa317efad/c314653a706a109dc59e420aa317efad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347297.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/500/22c14b20dc8ecb567b90e51dfc0a4500/22c14b20dc8ecb567b90e51dfc0a4500.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347297.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/500/22c14b20dc8ecb567b90e51dfc0a4500/22c14b20dc8ecb567b90e51dfc0a4500.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347300.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/621/eb0ffcb557d2dc1350343916fadc4621/eb0ffcb557d2dc1350343916fadc4621.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347300.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/621/eb0ffcb557d2dc1350343916fadc4621/eb0ffcb557d2dc1350343916fadc4621.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347303.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ab4/7d2af95f451fea4d55b5f950c2efcab4/7d2af95f451fea4d55b5f950c2efcab4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347303.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ab4/7d2af95f451fea4d55b5f950c2efcab4/7d2af95f451fea4d55b5f950c2efcab4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347306.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/d60/84589804d56f7606af3bd757f8741d60/84589804d56f7606af3bd757f8741d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347309.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ce9/2907c6ce038b9d8b8c50950486ceece9/2907c6ce038b9d8b8c50950486ceece9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347309.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ce9/2907c6ce038b9d8b8c50950486ceece9/2907c6ce038b9d8b8c50950486ceece9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347312.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/f74/89b43ec01c98425c5d156d89116cdf74/89b43ec01c98425c5d156d89116cdf74.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347312.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/f74/89b43ec01c98425c5d156d89116cdf74/89b43ec01c98425c5d156d89116cdf74.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347315.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/db2/b4bb66fd1dc808f7353ed339a3ea3db2/b4bb66fd1dc808f7353ed339a3ea3db2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347315.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/db2/b4bb66fd1dc808f7353ed339a3ea3db2/b4bb66fd1dc808f7353ed339a3ea3db2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347318.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ae7/b62bf68e010614da9cda9b4ff25bcae7/b62bf68e010614da9cda9b4ff25bcae7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347318.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ae7/b62bf68e010614da9cda9b4ff25bcae7/b62bf68e010614da9cda9b4ff25bcae7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347321.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/287/8358b8edaaf2a01d62c53bf67645e287/8358b8edaaf2a01d62c53bf67645e287.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347321.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/287/8358b8edaaf2a01d62c53bf67645e287/8358b8edaaf2a01d62c53bf67645e287.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347324.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/01c/979f99f4861d3c8354b062b9cc55b01c/979f99f4861d3c8354b062b9cc55b01c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347324.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/01c/979f99f4861d3c8354b062b9cc55b01c/979f99f4861d3c8354b062b9cc55b01c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347327.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347327.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c/9c406d08ca58ce93a0dd73b2e6aa5c3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347330.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/3c4/81a59d65052eb0ce081497b49f03e3c4/81a59d65052eb0ce081497b49f03e3c4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347330.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/3c4/81a59d65052eb0ce081497b49f03e3c4/81a59d65052eb0ce081497b49f03e3c4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347333.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/f9a/0ad556ae960143619bf39f438ad0ef9a/0ad556ae960143619bf39f438ad0ef9a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347333.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/f9a/0ad556ae960143619bf39f438ad0ef9a/0ad556ae960143619bf39f438ad0ef9a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347336.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/6a9/cb56574485967632113a25de82a576a9/cb56574485967632113a25de82a576a9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347336.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/6a9/cb56574485967632113a25de82a576a9/cb56574485967632113a25de82a576a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347339.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/cec/8229efdb4612b4fc8338fc4fc8db1cec/8229efdb4612b4fc8338fc4fc8db1cec.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347339.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/cec/8229efdb4612b4fc8338fc4fc8db1cec/8229efdb4612b4fc8338fc4fc8db1cec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347342.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/be8/d1e6d63bc37b0bf4de0ccee169f6dbe8/d1e6d63bc37b0bf4de0ccee169f6dbe8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347342.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/be8/d1e6d63bc37b0bf4de0ccee169f6dbe8/d1e6d63bc37b0bf4de0ccee169f6dbe8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347345.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ed7/09bbdf3a5661a9faac32ef21b42d6ed7/09bbdf3a5661a9faac32ef21b42d6ed7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347345.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/ed7/09bbdf3a5661a9faac32ef21b42d6ed7/09bbdf3a5661a9faac32ef21b42d6ed7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347348.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/148/739195e783e4f0c9a637cd182665d148/739195e783e4f0c9a637cd182665d148.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347348.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/148/739195e783e4f0c9a637cd182665d148/739195e783e4f0c9a637cd182665d148.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347351.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/07c/98151d85b578b8be96f86b27962e607c/98151d85b578b8be96f86b27962e607c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347351.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/07c/98151d85b578b8be96f86b27962e607c/98151d85b578b8be96f86b27962e607c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347354.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/aad/cbe8b3a39a5315ecce77f5e039d40aad/cbe8b3a39a5315ecce77f5e039d40aad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347354.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/aad/cbe8b3a39a5315ecce77f5e039d40aad/cbe8b3a39a5315ecce77f5e039d40aad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347357.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/a23/54939395d7e2ba98ba9096fa6f6aea23/54939395d7e2ba98ba9096fa6f6aea23.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347357.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/a23/54939395d7e2ba98ba9096fa6f6aea23/54939395d7e2ba98ba9096fa6f6aea23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347360.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/7b4/d1b6fc527793d618e9f51e76fa8797b4/d1b6fc527793d618e9f51e76fa8797b4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347360.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/7b4/d1b6fc527793d618e9f51e76fa8797b4/d1b6fc527793d618e9f51e76fa8797b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347363.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/96a/131b82fe320055ef940216a46b44996a/131b82fe320055ef940216a46b44996a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347363.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/96a/131b82fe320055ef940216a46b44996a/131b82fe320055ef940216a46b44996a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347366.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/e53/561c335c709174eba02b2bdfe989fe53/561c335c709174eba02b2bdfe989fe53.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_check_node test.unit.container.test_sharder.TestSharder.test_check_node ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347366.00000-0, db: /tmp/tmpa8_z1bns/sda/containers/0/e53/561c335c709174eba02b2bdfe989fe53/561c335c709174eba02b2bdfe989fe53.db sharder-test WARNING: Skipping d100 as it is not mounted sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002980232238769531s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016832351684570312s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_0 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpm2cew4_t/sda/containers/0/b85/9d7a328814f4e99b3c0e6a0597883b85/9d7a328814f4e99b3c0e6a0597883b85.db, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Cleaved ShardRange in 0.011s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002903938293457031s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016450881958007812s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpm2cew4_t/sda/containers/0/069/5a8214a697fa9e61b996df73488a7069/5a8214a697fa9e61b996df73488a7069.db, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: misplaced object source bounds [('', 'obj004')], path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019025802612304688s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00011873245239257812s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018715858459472656s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002887248992919922s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017261505126953125s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717346923.76713, (5, 5) as of 1717347199.32943, cleaved as of 1717346923.76713> in 0.038s, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpm2cew4_t/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347198.88701.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003352165222167969s, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.000217437744140625s, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test INFO: Cleaved ShardRange in 0.041s, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028705596923828125s, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00023126602172851562s, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies ... ok sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717346923.76713, (1, 6144) as of 1717347199.64471, cleaved as of 1717346923.76713> in 0.033s, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpn_mnbj44/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.40322.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0003571510314941406s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018072128295898438s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaved ShardRange in 0.052s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: got 6 rows (deleted=False) in 0.0005044937133789062s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015211105346679688s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717346923.76713, (6, 6) as of 1717347199.99528, cleaved as of 1717346923.76713> in 0.035s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '7a6ff0bc-a7e2-4805-856b-d5a4366375e4-sda', 'cursor': '', 'max_row': 12, 'cleave_to_row': 10, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Completed cleaving, DB remaining in sharding state, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db test.unit.container.test_sharder.TestSharder.test_cleave_repeated test.unit.container.test_sharder.TestSharder.test_cleave_repeated ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaving from row 10 into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002467632293701172s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003609657287597656s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaved ShardRange in 0.034s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaving from row 10 into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717346923.76713, (6, 6) as of 1717347199.99528, cleaved as of 1717346923.76713>, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00027489662170410156s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023746490478515625s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717346923.76713, (6, 6) as of 1717347199.99528, cleaved as of 1717346923.76713> in 0.024s, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpev3c5rwd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347199.71397.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025272369384765625s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014019012451171875s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaved ShardRange in 0.033s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, found as of 1717346923.76713>, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0004115104675292969s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001494884490966797s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmp2ypqnibh/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - shard db already in sync, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717346923.76713, (3, 6) as of 1717347200.66391, cleaved as of 1717346923.76713> in 0.012s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002493858337402344s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015401840209960938s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717346923.76713, (1, 100) as of 1717347200.75850, cleaved as of 1717346923.76713> in 0.038s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00027060508728027344s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00015473365783691406s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717346923.76713, (1, 1000) as of 1717347200.88948, cleaved as of 1717346923.76713> in 0.041s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717346923.76713, (2, 15) as of 1717347200.94841, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024080276489257812s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001361370086669922s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717346923.76713, (2, 15) as of 1717347200.94841, active as of 1717346923.76713> in 0.025s, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db test.unit.container.test_sharder.TestSharder.test_cleave_root test.unit.container.test_sharder.TestSharder.test_cleave_root ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges ... ok sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmp2ypqnibh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347200.30768.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00040721893310546875s, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013303756713867188s, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test INFO: Cleaved ShardRange in 0.034s, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpsujz43i5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.10793.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00026226043701171875s, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020885467529296875s, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022864341735839844s, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002281665802001953s, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'d' to 'x' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - zero objects found, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'x' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023245811462402344s, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013303756713867188s, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'x' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - zero objects found, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp022s0h5z/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.31583.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002422332763671875s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00015282630920410156s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 8.177757263183594e-05s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013899803161621094s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaved ShardRange in 0.043s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, found as of 1717346923.76713>, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028443336486816406s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00011348724365234375s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018525123596191406s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.012222290039062e-05s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001494884490966797s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmp2_kmwtej/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - shard db already in sync, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717346923.76713, (3, 6) as of 1717347202.05929, cleaved as of 1717346923.76713> in 0.013s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003371238708496094s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010752677917480469s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017070770263671875s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717346923.76713, (1, 100) as of 1717347202.16003, cleaved as of 1717346923.76713> in 0.04s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023889541625976562s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.179115295410156e-05s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001537799835205078s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.535385131835938e-05s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717346923.76713, (1, 1000) as of 1717347202.28037, cleaved as of 1717346923.76713> in 0.038s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717346923.76713, (2, 15) as of 1717347202.33177, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002357959747314453s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.5367431640625e-05s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001399517059326172s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717346923.76713, (2, 15) as of 1717347202.33177, active as of 1717346923.76713> in 0.025s, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmp2_kmwtej/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347201.66449.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002608299255371094s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013709068298339844s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change ... ok sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Cleaved ShardRange in 0.029s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717346923.76713, (500000, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002474784851074219s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00014925003051757812s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_x_1 for ShardRange, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Cleaved ShardRange<'d' to 'x' as of 1717346923.76713, (2, 103) as of 1717347202.68163, cleaved as of 1717346923.76713> in 0.044s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: Continuing to cleave (2 done, 1 todo), path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'d' to MaxBound as of 1717347202.72457, (500000, 0) as of 1717347202.72457, created as of 1717347202.72457>, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00025010108947753906s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.000152587890625s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__2 for ShardRange, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test INFO: Cleaved ShardRange<'d' to MaxBound as of 1717347202.72457, (3, 1103) as of 1717347202.79936, cleaved as of 1717347202.72457> in 0.037s, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpyx6xc3cx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347202.45926.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002512931823730469s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 8.559226989746094e-05s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001513957977294922s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/c_here_0 in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmp550s8eu9/sda/containers/0/e19/9e88c3bdfd03a830d1fc09d35d671e19/9e88c3bdfd03a830d1fc09d35d671e19.db, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012803077697753906s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00013899803161621094s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 7.724761962890625e-05s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 8.797645568847656e-05s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_0 for ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db test.unit.container.test_sharder.TestSharder.test_cleave_shard test.unit.container.test_sharder.TestSharder.test_cleave_shard ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range ... ok sharder-test DEBUG: got 3 rows (deleted=False) in 0.000255584716796875s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001342296600341797s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_there_0 for ShardRange<'here' to 'where' as of 1717347202.84490, (0, 0) as of 1717347202.84490, sharding as of 1717347202.84490>, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717346923.76713, (3, 6) as of 1717347203.11691, cleaved as of 1717346923.76713> in 0.032s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000232696533203125s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001239776611328125s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014829635620117188s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001316070556640625s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012445449829101562s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_1 for ShardRange<'there' to 'where' as of 1717346923.76713, (2, 15) as of 1717347203.17651, cleaved as of 1717346923.76713>, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00022649765014648438s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001354217529296875s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_where_1 for ShardRange<'here' to 'where' as of 1717347202.84490, (0, 0) as of 1717347202.84490, sharding as of 1717347202.84490>, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717346923.76713, (2, 15) as of 1717347203.17651, cleaved as of 1717346923.76713> in 0.021s, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp550s8eu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347202.84491.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpn0l6bfpl/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000202178955078125s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001163482666015625s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016045570373535156s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012040138244628906s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_1_ for ShardRange<'h' to MaxBound as of 1717347203.46878, (1000, 11111) as of 1717347203.46878, created as of 1717347203.46878>, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002543926239013672s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016236305236816406s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_1_ for ShardRange<'h' to 'w' as of 1717347226.00000, (0, 0) as of 1717347215.00000, shrunk as of 1717347215.00000>, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717347203.46878, (1000, 11111) as of 1717347203.46878, created as of 1717347203.46878> in 0.048s, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_1, db: /tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347216.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020432472229003906s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011992454528808594s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012922286987304688s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001575946807861328s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_2_ for ShardRange<'h' to MaxBound as of 1717347203.69398, (1000, 11111) as of 1717347203.69398, cleaved as of 1717347203.69398>, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002465248107910156s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001819133758544922s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_2_ for ShardRange<'h' to 'w' as of 1717347253.00000, (0, 0) as of 1717347242.00000, shrunk as of 1717347242.00000>, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717347203.69398, (1000, 11111) as of 1717347203.69398, cleaved as of 1717347203.69398> in 0.039s, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_2, db: /tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347243.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020694732666015625s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012135505676269531s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012445449829101562s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000125885009765625s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_3_ for ShardRange<'h' to MaxBound as of 1717347203.91675, (1000, 11111) as of 1717347203.91675, active as of 1717347203.91675>, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00024247169494628906s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013327598571777344s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_3_ for ShardRange<'h' to 'w' as of 1717347280.00000, (0, 0) as of 1717347269.00000, shrunk as of 1717347269.00000>, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717347203.91675, (1000, 11111) as of 1717347203.91675, active as of 1717347203.91675> in 0.028s, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_3, db: /tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347270.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020051002502441406s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011754035949707031s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001232624053955078s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012373924255371094s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_k for ShardRange<'d' to 'k' as of 1717347204.09886, (1000, 11111) as of 1717347204.09886, created as of 1717347204.09886>, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002846717834472656s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014281272888183594s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_k for ShardRange<'h' to 'w' as of 1717347296.00000, (0, 0) as of 1717347296.00000, shrinking as of 1717347296.00000>, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347204.09886, (1000, 11111) as of 1717347204.09886, created as of 1717347204.09886> in 0.026s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_ for ShardRange<'k' to MaxBound as of 1717347204.09890, (1000, 11111) as of 1717347204.09890, created as of 1717347204.09890>, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.000247955322265625s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001633167266845703s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_ for ShardRange<'h' to 'w' as of 1717347296.00000, (0, 0) as of 1717347296.00000, shrinking as of 1717347296.00000>, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717347204.09890, (1000, 11111) as of 1717347204.09890, created as of 1717347204.09890> in 0.021s, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_4, db: /tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347297.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000202178955078125s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012040138244628906s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012612342834472656s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Setting object count to 1068 (txn: tx8518a18c06c4476eb84fe-00665ca37d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: txa6fdaadf1182495d8a2ae-00665ca37f) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txed54b5473aba474583b96-00665ca382) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txed54b5473aba474583b96-00665ca382) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txed54b5473aba474583b96-00665ca382) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txed54b5473aba474583b96-00665ca382) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txed54b5473aba474583b96-00665ca382) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txed54b5473aba474583b96-00665ca382) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: txed54b5473aba474583b96-00665ca382) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: txed54b5473aba474583b96-00665ca382) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: txed54b5473aba474583b96-00665ca382) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: txed54b5473aba474583b96-00665ca382) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: txed54b5473aba474583b96-00665ca382) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: txed54b5473aba474583b96-00665ca382) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txed54b5473aba474583b96-00665ca382) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txed54b5473aba474583b96-00665ca382) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: txed54b5473aba474583b96-00665ca382) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: txc0806ce79ba94d83ae41c-00665ca384) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards ... ok sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012493133544921875s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_k for ShardRange<'d' to 'k' as of 1717347204.34088, (1000, 11111) as of 1717347204.34088, cleaved as of 1717347204.34088>, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002913475036621094s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014066696166992188s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_k for ShardRange<'h' to 'w' as of 1717347329.00000, (0, 0) as of 1717347329.00000, shrinking as of 1717347329.00000>, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347204.34088, (1000, 11111) as of 1717347204.34088, cleaved as of 1717347204.34088> in 0.022s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_ for ShardRange<'k' to MaxBound as of 1717347204.34092, (1000, 11111) as of 1717347204.34092, cleaved as of 1717347204.34092>, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024366378784179688s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001308917999267578s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_ for ShardRange<'h' to 'w' as of 1717347329.00000, (0, 0) as of 1717347329.00000, shrinking as of 1717347329.00000>, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717347204.34092, (1000, 11111) as of 1717347204.34092, cleaved as of 1717347204.34092> in 0.018s, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_5, db: /tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347330.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020933151245117188s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011992454528808594s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012421607971191406s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012421607971191406s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_k for ShardRange<'d' to 'k' as of 1717347204.67006, (1000, 11111) as of 1717347204.67006, active as of 1717347204.67006>, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023674964904785156s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013971328735351562s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_k for ShardRange<'h' to 'w' as of 1717347362.00000, (0, 0) as of 1717347362.00000, shrinking as of 1717347362.00000>, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347204.67006, (1000, 11111) as of 1717347204.67006, active as of 1717347204.67006> in 0.02s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_ for ShardRange<'k' to MaxBound as of 1717347204.67010, (1000, 11111) as of 1717347204.67010, active as of 1717347204.67010>, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00023794174194335938s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013828277587890625s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_ for ShardRange<'h' to 'w' as of 1717347362.00000, (0, 0) as of 1717347362.00000, shrinking as of 1717347362.00000>, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717347204.67010, (1000, 11111) as of 1717347204.67010, active as of 1717347204.67010> in 0.02s, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_6, db: /tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347363.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002071857452392578s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014019012451171875s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012612342834472656s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001251697540283203s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_k for ShardRange<'d' to 'k' as of 1717347204.89867, (1000, 11111) as of 1717347204.89867, cleaved as of 1717347204.89867>, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002627372741699219s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015473365783691406s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_k for ShardRange<'h' to 'w' as of 1717347395.00000, (0, 0) as of 1717347395.00000, shrinking as of 1717347395.00000>, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347204.89867, (1000, 11111) as of 1717347204.89867, cleaved as of 1717347204.89867> in 0.017s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_t for ShardRange<'k' to 't' as of 1717347204.89871, (1000, 11111) as of 1717347204.89871, cleaved as of 1717347204.89871>, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023698806762695312s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014519691467285156s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_t for ShardRange<'h' to 'w' as of 1717347395.00000, (0, 0) as of 1717347395.00000, shrinking as of 1717347395.00000>, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347204.89871, (1000, 11111) as of 1717347204.89871, cleaved as of 1717347204.89871> in 0.018s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_ for ShardRange<'t' to MaxBound as of 1717347204.89876, (1000, 11111) as of 1717347204.89876, cleaved as of 1717347204.89876>, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00022935867309570312s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001327991485595703s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_ for ShardRange<'h' to 'w' as of 1717347395.00000, (0, 0) as of 1717347395.00000, shrinking as of 1717347395.00000>, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717347204.89876, (1000, 11111) as of 1717347204.89876, cleaved as of 1717347204.89876> in 0.02s, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_7, db: /tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347396.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021338462829589844s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012040138244628906s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013065338134765625s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012636184692382812s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_k for ShardRange<'d' to 'k' as of 1717347205.18404, (1000, 11111) as of 1717347205.18404, created as of 1717347205.18404>, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024127960205078125s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013709068298339844s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_k for ShardRange<'h' to 'w' as of 1717347434.00000, (0, 0) as of 1717347434.00000, shrinking as of 1717347434.00000>, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347205.18404, (1000, 11111) as of 1717347205.18404, created as of 1717347205.18404> in 0.018s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_t for ShardRange<'k' to 't' as of 1717347205.18409, (1000, 11111) as of 1717347205.18409, created as of 1717347205.18409>, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023365020751953125s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001704692840576172s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_t for ShardRange<'h' to 'w' as of 1717347434.00000, (0, 0) as of 1717347434.00000, shrinking as of 1717347434.00000>, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347205.18409, (1000, 11111) as of 1717347205.18409, created as of 1717347205.18409> in 0.019s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_ for ShardRange<'t' to MaxBound as of 1717347205.18411, (1000, 11111) as of 1717347205.18411, created as of 1717347205.18411>, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002734661102294922s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013017654418945312s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_ for ShardRange<'h' to 'w' as of 1717347434.00000, (0, 0) as of 1717347434.00000, shrinking as of 1717347434.00000>, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717347205.18411, (1000, 11111) as of 1717347205.18411, created as of 1717347205.18411> in 0.018s, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_8, db: /tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347435.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019860267639160156s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00011754035949707031s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012254714965820312s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012135505676269531s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_k for ShardRange<'d' to 'k' as of 1717347205.45952, (1000, 11111) as of 1717347205.45952, active as of 1717347205.45952>, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023818016052246094s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001456737518310547s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_k for ShardRange<'h' to 'w' as of 1717347473.00000, (0, 0) as of 1717347473.00000, shrinking as of 1717347473.00000>, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347205.45952, (1000, 11111) as of 1717347205.45952, active as of 1717347205.45952> in 0.017s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_t for ShardRange<'k' to 't' as of 1717347205.45956, (1000, 11111) as of 1717347205.45956, active as of 1717347205.45956>, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023365020751953125s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013637542724609375s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_t for ShardRange<'h' to 'w' as of 1717347473.00000, (0, 0) as of 1717347473.00000, shrinking as of 1717347473.00000>, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347205.45956, (1000, 11111) as of 1717347205.45956, active as of 1717347205.45956> in 0.018s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_ for ShardRange<'t' to MaxBound as of 1717347205.45958, (1000, 11111) as of 1717347205.45958, active as of 1717347205.45958>, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023126602172851562s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013637542724609375s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_ for ShardRange<'h' to 'w' as of 1717347473.00000, (0, 0) as of 1717347473.00000, shrinking as of 1717347473.00000>, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717347205.45958, (1000, 11111) as of 1717347205.45958, active as of 1717347205.45958> in 0.018s, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_9, db: /tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347474.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplunfwsit/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000209808349609375s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012540817260742188s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001239776611328125s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012350082397460938s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_10_k for ShardRange<'d' to 'k' as of 1717347205.74004, (1000, 11111) as of 1717347205.74004, created as of 1717347205.74004>, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024247169494628906s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013589859008789062s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_10_k for ShardRange<'h' to 'w' as of 1717347512.00000, (0, 0) as of 1717347512.00000, shrinking as of 1717347512.00000>, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347205.74004, (1000, 11111) as of 1717347205.74004, created as of 1717347205.74004> in 0.019s, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3d9e4368-2dc6-4cbc-9c04-6e9e43a1cb9b-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 0}, path: .shards_a/donor_10, db: /tmp/tmplunfwsit/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347513.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00027871131896972656s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012254714965820312s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012540817260742188s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001246929168701172s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_k for ShardRange<'d' to 'k' as of 1717347205.91667, (1000, 11111) as of 1717347205.91667, cleaved as of 1717347205.91667>, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024437904357910156s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00045013427734375s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_k for ShardRange<'h' to 'w' as of 1717347538.00000, (0, 0) as of 1717347538.00000, shrinking as of 1717347538.00000>, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347205.91667, (1000, 11111) as of 1717347205.91667, cleaved as of 1717347205.91667> in 0.024s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_t for ShardRange<'k' to 't' as of 1717347205.91671, (1000, 11111) as of 1717347205.91671, cleaved as of 1717347205.91671>, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025010108947753906s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002067089080810547s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_t for ShardRange<'h' to 'w' as of 1717347538.00000, (0, 0) as of 1717347538.00000, shrinking as of 1717347538.00000>, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347205.91671, (1000, 11111) as of 1717347205.91671, cleaved as of 1717347205.91671> in 0.024s, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '6d067266-8338-4cdb-a075-9f1f3224d48e-sda', 'cursor': 't', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: .shards_a/donor_11, db: /tmp/tmplunfwsit/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347539.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002789497375488281s, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001404285430908203s, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013017654418945312s, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012564659118652344s, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok sharder-test WARNING: Repeat cleaving required, context: {'ref': 'f32f49b3-5713-46f6-a4a0-46e2d3f7cf98-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 1}, path: .shards_a/donor_12, db: /tmp/tmplunfwsit/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717347571.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021767616271972656s, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012111663818359375s, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012993812561035156s, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012445449829101562s, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '621099d8-2bf5-4a1b-9542-ff3b7be4e7dc-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 2}, path: .shards_a/donor_13, db: /tmp/tmplunfwsit/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717347591.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021719932556152344s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012111663818359375s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001246929168701172s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012230873107910156s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_14_k for ShardRange<'d' to 'k' as of 1717347206.81493, (1000, 11111) as of 1717347206.81493, cleaved as of 1717347206.81493>, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024390220642089844s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014519691467285156s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_14_k for ShardRange<'h' to 'w' as of 1717347610.00000, (0, 0) as of 1717347610.00000, shrinking as of 1717347610.00000>, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347206.81493, (1000, 11111) as of 1717347206.81493, cleaved as of 1717347206.81493> in 0.022s, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test INFO: Stopped cleave at gap: 'k' - 't', path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0f196035-abbe-46e4-9b87-5a07afcc2222-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 1}, path: .shards_a/donor_14, db: /tmp/tmplunfwsit/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717347611.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020742416381835938s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012445449829101562s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012731552124023438s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001246929168701172s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test INFO: Cleaving from row None into .shards/other for ShardRange<'here' to 'where' as of 1717347209.00000, (0, 0) as of 1717347209.00000, active as of 1717347209.00000>, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002281665802001953s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013518333435058594s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards/other for ShardRange<'here' to 'there' as of 1717347207.11886, (0, 0) as of 1717347207.00000, shrunk as of 1717347207.00000>, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test INFO: Cleaved ShardRange<'here' to 'where' as of 1717347209.00000, (0, 0) as of 1717347209.00000, active as of 1717347209.00000> in 0.046s, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpqt9l613y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020194053649902344s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012540817260742188s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012230873107910156s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013136863708496094s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test INFO: Cleaving from row None into .shards_a/shard_d for ShardRange<'here' to MaxBound as of 1717347210.00000, (0, 0) as of 1717347210.00000, active as of 1717347210.00000>, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00031256675720214844s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001366138458251953s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/shard_d for ShardRange<'here' to 'there' as of 1717347208.00000, (0, 0) as of 1717347208.00000, sharding as of 1717347208.00000>, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test INFO: Cleaved ShardRange<'here' to MaxBound as of 1717347210.00000, (0, 0) as of 1717347210.00000, active as of 1717347210.00000> in 0.027s, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmplrd7qmno/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020384788513183594s, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012135505676269531s, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012063980102539062s, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor ... ok sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002090930938720703s, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001251697540283203s, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpdqi18a1a/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347208.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024700164794921875s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001823902130126953s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012826919555664062s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013065338134765625s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00022935867309570312s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container a/c for ShardRange<'here' to 'there' as of 1717347207.90744, (0, 0) as of 1717347208.00000, shrunk as of 1717347208.00000>, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test INFO: Cleaved ShardRange in 0.032s, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpj_ts0bcs/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347209.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaving from row None into .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002715587615966797s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016117095947265625s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaved ShardRange in 0.029s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_2 for ShardRange<'b' to 'd' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002384185791015625s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013518333435058594s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found ... ok sharder-test INFO: Replicating new shard container .shards_a/c_d_2 for ShardRange, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaved ShardRange<'b' to 'd' as of 1717346923.76713, (2, 3) as of 1717347208.27637, cleaved as of 1717346923.76713> in 0.028s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Continuing to cleave (2 done, 2 todo), path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaving from row None into .shards_a/c_f_3 for ShardRange<'d' to 'f' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025653839111328125s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014281272888183594s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_f_3 for ShardRange, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaved ShardRange<'d' to 'f' as of 1717346923.76713, (2, 103) as of 1717347208.36314, cleaved as of 1717346923.76713> in 0.025s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Stopped cleave at unready ShardRange<'f' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, found as of 1717346923.76713>, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'f' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717347208.39139>, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023674964904785156s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016379356384277344s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test INFO: Cleaved ShardRange<'f' to MaxBound as of 1717346923.76713, (1, 1000) as of 1717347208.46209, cleaved as of 1717347208.39139> in 0.033s, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp0wv4ihf9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347207.94860.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002498626708984375s, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00016689300537109375s, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test INFO: Cleaved ShardRange in 0.042s, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023436546325683594s, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics ... ok sharder-test DEBUG: got 2 rows (deleted=True) in 0.00016379356384277344s, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717346923.76713, (1, 18432) as of 1717347208.78363, cleaved as of 1717346923.76713> in 0.061s, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmps6zjtpks/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.53658.db REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, sync, (5, '80247acec3582a94ec18e5fa997259bf', '2e3812b2-04dc-432e-8964-52dd492bcb9e-sdb', '1717347208.86868', '0000000000.00000', '0', '') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_items, ([{'ROWID': 1, 'name': 'obj002', 'created_at': '1717347210.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'obj003', 'created_at': '1717347211.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'obj004', 'created_at': '1717347212.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'obj005', 'created_at': '1717347213.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'obj006', 'created_at': '1717347214.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '2e3812b2-04dc-432e-8964-52dd492bcb9e-sdb') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_syncs, ([],) sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002589225769042969s, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013780593872070312s, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaved ShardRange in 0.024s, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002524852752685547s, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013494491577148438s, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717346923.76713, (3, 3) as of 1717347209.15924, cleaved as of 1717346923.76713> in 0.02s, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpmw47rh1f/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache ... ok REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, sync, (7, '00000000000000000000000000000000', '3b1821b6-b368-4997-85ed-b963f30e213d-', '1717347208.83989', '0000000000.00000', '0', '{"X-Container-Sysmeta-Shard-Context-0e3b9b86-b7be-4dc5-a5da-99f3bcdf18df-sda": ["{\\"ref\\": \\"0e3b9b86-b7be-4dc5-a5da-99f3bcdf18df-sda\\", \\"cursor\\": \\"\\", \\"max_row\\": 7, \\"cleave_to_row\\": 7, \\"last_cleave_to_row\\": null, \\"cleaving_done\\": true, \\"misplaced_done\\": true, \\"ranges_done\\": 2, \\"ranges_todo\\": 0}", "1717347209.18282"]}') REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, merge_shard_ranges, ([{'name': '.shards_a/c_obj004_0', 'timestamp': '1717346923.76713', 'lower': '', 'upper': 'obj004', 'object_count': 4, 'bytes_used': 4, 'meta_timestamp': '1717347209.10772', 'deleted': 0, 'state': 40, 'state_timestamp': '1717346923.76713', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/c__1', 'timestamp': '1717346923.76713', 'lower': 'obj004', 'upper': '', 'object_count': 3, 'bytes_used': 3, 'meta_timestamp': '1717347209.15924', 'deleted': 0, 'state': 40, 'state_timestamp': '1717346923.76713', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347208.93849', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347209.19139', 'deleted': 0, 'state': 70, 'state_timestamp': '1717347208.93854', 'epoch': '1717347208.93854', 'reported': 0, 'tombstones': -1}], '3b1821b6-b368-4997-85ed-b963f30e213d-') sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002446174621582031s, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018286705017089844s, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaved ShardRange in 0.035s, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717346923.76713, (3, 3) as of 1717347209.15924, active as of 1717346923.76713>, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024819374084472656s, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015473365783691406s, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717346923.76713, (3, 3) as of 1717347209.15924, active as of 1717346923.76713> in 0.02s, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpmw47rh1f/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347208.93854.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmpwn9w1mk8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347209.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '6c04be58-b385-43b4-bfc9-fc6050cafa37-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp5a6je9dz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347209.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '6c04be58-b385-43b4-bfc9-fc6050cafa37-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp5a6je9dz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347209.00000.db test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Setting object count to 1058 (txn: txc0806ce79ba94d83ae41c-00665ca384) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Setting object count to 1024 (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Setting object count to 1024 (txn: tx9410e06fcc8943788d591-00665ca386) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Setting object count to 1072 (txn: tx2f55f8cc56f6444598d81-00665ca38a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_root test.unit.container.test_sharder.TestSharder.test_complete_sharding_root ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap ... ok sharder-test WARNING: Repeat cleaving required, context: {'ref': '6c04be58-b385-43b4-bfc9-fc6050cafa37-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp5a6je9dz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347209.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '6c04be58-b385-43b4-bfc9-fc6050cafa37-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp5a6je9dz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347209.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'bad5274d-70a4-4c04-8ccc-8c0692a50771-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp5a6je9dz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347209.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b9435c8-7e91-49aa-ba0d-38c441d0223e-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnle928ev/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347214.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b9435c8-7e91-49aa-ba0d-38c441d0223e-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnle928ev/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347214.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b9435c8-7e91-49aa-ba0d-38c441d0223e-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnle928ev/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347214.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b9435c8-7e91-49aa-ba0d-38c441d0223e-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnle928ev/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347214.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'eb0952e6-35aa-4ca3-ab8a-a64604b5666e-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnle928ev/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347214.00000.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: TypeError('the JSON object must be str, bytes or bytearray, not dict'), path: a/c, db: /tmp/tmphfvubkkk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: JSONDecodeError('Expecting value: line 1 column 1 (char 0)'), path: a/c, db: /tmp/tmphfvubkkk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: ValueError('not a list'), path: a/c, db: /tmp/tmphfvubkkk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('name'), path: a/c, db: /tmp/tmphfvubkkk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('timestamp'), path: a/c, db: /tmp/tmphfvubkkk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('object_count'), path: a/c, db: /tmp/tmphfvubkkk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type None, path: a/c, db: /tmp/tmph2_4jej5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'object', path: a/c, db: /tmp/tmph2_4jej5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'disco', path: a/c, db: /tmp/tmph2_4jej5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get shard ranges from a/c: Unexpected response: 404, path: a/c, db: /tmp/tmpv4soa4sd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp01brt09i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmp01brt09i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Found 2 compactible sequences of length(s) [2, 2], path: a/c, db: /tmp/tmp01brt09i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp01brt09i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange<'c' to 'e' as of 1717347214.63186, (100000, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp01brt09i/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmposcqq60r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmposcqq60r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmposcqq60r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpvemmsvs3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size ... ok test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part ... ok sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp4chcth_b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347214.00000.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp28411kfg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp28411kfg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp28411kfg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: a/c, db: /tmp/tmp28411kfg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp28411kfg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmp28411kfg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp8qgnkvlc/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp8qgnkvlc/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp8qgnkvlc/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: .shards_a/c_, db: /tmp/tmp8qgnkvlc/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp8qgnkvlc/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmp8qgnkvlc/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp33buj9g3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp33buj9g3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp33buj9g3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmp33buj9g3/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmpm7glfeze/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmpm7glfeze/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmpm7glfeze/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmpm7glfeze/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp9wn2ctmb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp9wn2ctmb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp9wn2ctmb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp9wn2ctmb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmp51hhj_mp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmp51hhj_mp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmp51hhj_mp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmp51hhj_mp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp1h9lsarr/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp1h9lsarr/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp1h9lsarr/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate ... ok test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats test.unit.container.test_sharder.TestSharder.test_increment_stats ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd ... ok sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:35 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:36 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test WARNING: Repeat cleaving required, context: {'ref': '69df208f-c707-4110-a781-198213c6d596-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpy86ish2q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T16:53:36.000000. DB state: sharding, own_shard_range state: sharding, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 2, 'active': 0}, path: a/c, db: /tmp/tmpy86ish2q/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T16:53:37.000000. DB state: sharding, own_shard_range state: shrinking, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 0, 'active': 1}, path: .shards_a/shard_c, db: /tmp/tmpsheaxq7y/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347217.00000.db test.unit.container.test_sharder.TestSharder.test_init test.unit.container.test_sharder.TestSharder.test_init ... ok test.unit.container.test_sharder.TestSharder.test_init_deprecated_options test.unit.container.test_sharder.TestSharder.test_init_deprecated_options ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name ... ok test.unit.container.test_sharder.TestSharder.test_log_broker test.unit.container.test_sharder.TestSharder.test_log_broker ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception test.unit.container.test_sharder.TestSharder.test_log_broker_exception ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_levels test.unit.container.test_sharder.TestSharder.test_log_broker_levels ... ok sharder-test WARNING: shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test DEBUG: bonjour mes amis, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: bonjour mes amis, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: bonjour mes amis, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: bonjour mes amis, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpppbnq1jc/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: exception: test, path: a/c, db: /tmp/tmp_21ebklb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 448, in test_log_broker_exception raise ValueError('test') ValueError: test sharder-test DEBUG: bonjour mes amis, path: , db: /tmp/tmpkjf35fw8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: bonjour mes amis, path: a/c, db: sharder-test INFO: bonjour mes amis, path: , db: /tmp/tmpkjf35fw8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: bonjour mes amis, path: a/c, db: sharder-test WARNING: bonjour mes amis, path: , db: /tmp/tmpkjf35fw8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: bonjour mes amis, path: a/c, db: sharder-test ERROR: bonjour mes amis, path: , db: /tmp/tmpkjf35fw8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: bonjour mes amis, path: a/c, db: sharder-test DEBUG: Unlinked retiring db '/tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002307891845703125s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021910667419433594s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002124309539794922s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00023555755615234375s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002734661102294922s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001709461212158203s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001723766326904297s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015735626220703125s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001671314239501953s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002484321594238281s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022363662719726562s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347216.31730, (0, 0) as of 1717347216.31730, active as of 1717347216.31730>, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347216.31732, (0, 0) as of 1717347216.31732, active as of 1717347216.31732>, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/there-where in state active, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347216.31735, (0, 0) as of 1717347216.31735, active as of 1717347216.31735>, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/where-yonder in state active, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018262863159179688s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001595020294189453s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015687942504882812s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001552104949951172s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016045570373535156s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017547607421875s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017261505126953125s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015664100646972656s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016880035400390625s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016951560974121094s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001552104949951172s, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347216.31732, (0, 0) as of 1717347216.31732, active as of 1717347216.31732>, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347216.31735, (0, 0) as of 1717347216.31735, active as of 1717347216.31735>, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptovie8nj/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.32101.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020813941955566406s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024700164794921875s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002162456512451172s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00022363662719726562s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002551078796386719s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002334117889404297s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001671314239501953s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016880035400390625s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001590251922607422s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017118453979492188s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001614093780517578s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015854835510253906s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001671314239501953s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015592575073242188s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017452239990234375s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001621246337890625s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001857280731201172s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017499923706054688s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016450881958007812s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000156402587890625s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015354156494140625s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015497207641601562s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001842975616455078s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001659393310546875s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001659393310546875s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015807151794433594s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001544952392578125s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018548965454101562s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001652240753173828s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016355514526367188s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001590251922607422s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015354156494140625s, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpg_c7sjxc/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347216.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002384185791015625s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000217437744140625s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021839141845703125s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002384185791015625s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002372264862060547s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016307830810546875s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015854835510253906s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015091896057128906s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016260147094726562s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002276897430419922s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021529197692871094s, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347217.15722, (0, 0) as of 1717347217.15722, active as of 1717347217.15722>, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347217.15725, (0, 0) as of 1717347217.15725, active as of 1717347217.15725>, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347217.15727, (0, 0) as of 1717347217.15727, active as of 1717347217.15727>, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 1 required (not removing objects), shard db: /tmp/tmpyxpcjhbq/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpyxpcjhbq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.16111.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024056434631347656s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021314620971679688s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002143383026123047s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas ... ok test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Setting object count to 1072 (txn: txf5c047cb55354dc8b3889-00665ca38c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Setting object count to 1072 (txn: txeb263ff63a51465982ec0-00665ca38f) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache ... ok sharder-test DEBUG: got 2 rows (deleted=False) in 0.000244140625s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000240325927734375s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016427040100097656s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001590251922607422s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001533031463623047s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016570091247558594s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023174285888671875s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002257823944091797s, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347217.32245, (0, 0) as of 1717347217.32245, active as of 1717347217.32245>, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347217.32247, (0, 0) as of 1717347217.32247, active as of 1717347217.32247>, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/there-where in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmp0wzt_1pw/sda/containers/0/170/533f29a8ddb31e55151765e09ccbb170/533f29a8ddb31e55151765e09ccbb170.db, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347217.32249, (0, 0) as of 1717347217.32249, active as of 1717347217.32249>, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0wzt_1pw/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.32975.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002262592315673828s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021076202392578125s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020742416381835938s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.000232696533203125s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002562999725341797s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016617774963378906s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016760826110839844s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015664100646972656s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001804828643798828s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002512931823730469s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021600723266601562s, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347217.50223, (0, 0) as of 1717347217.50223, active as of 1717347217.50223>, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/here-there in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpcra5rokf/sda/containers/0/ed5/9b40bf3cdc484061806b8d0a545d2ed5/9b40bf3cdc484061806b8d0a545d2ed5.db, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347217.50225, (0, 0) as of 1717347217.50225, active as of 1717347217.50225>, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347217.50228, (0, 0) as of 1717347217.50228, active as of 1717347217.50228>, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 2 required (not removing objects), shard db: /tmp/tmpcra5rokf/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects ... ok sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpcra5rokf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.50591.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025963783264160156s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022339820861816406s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002884864807128906s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024509429931640625s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003402233123779297s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017309188842773438s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016808509826660156s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015425682067871094s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001671314239501953s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002694129943847656s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00026607513427734375s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347217.71149, (0, 0) as of 1717347217.71149, active as of 1717347217.71149>, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347217.71151, (0, 0) as of 1717347217.71151, active as of 1717347217.71151>, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347217.71154, (0, 0) as of 1717347217.71154, active as of 1717347217.71154>, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017380714416503906s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001571178436279297s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001595020294189453s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016808509826660156s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001876354217529297s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001709461212158203s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001556873321533203s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002181529998779297s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001976490020751953s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020194053649902344s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020384788513183594s, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347217.71149, (0, 0) as of 1717347217.71149, active as of 1717347217.71149>, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpmtpud4fb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347217.71730.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002696514129638672s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013589859008789062s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001857280731201172s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001575946807861328s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017905235290527344s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024127960205078125s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022602081298828125s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'yonder')], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018286705017089844s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00017690658569335938s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017189979553222656s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016117095947265625s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016188621520996094s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016117095947265625s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017380714416503906s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017642974853515625s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001621246337890625s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017189979553222656s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022292137145996094s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002124309539794922s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001819133758544922s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025200843811035156s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024437904357910156s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00036644935607910156s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002880096435546875s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016832351684570312s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001914501190185547s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001957416534423828s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015425682067871094s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003407001495361328s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002951622009277344s, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'yonder' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Placed 3 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp77hrxb9y/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347218.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002415180206298828s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.000255584716796875s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003237724304199219s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024366378784179688s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002903938293457031s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003197193145751953s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026035308837890625s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022745132446289062s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Placed 1 misplaced objects (2 unplaced), path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018143653869628906s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001773834228515625s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017642974853515625s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002181529998779297s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016427040100097656s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018262863159179688s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00020575523376464844s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017023086547851562s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0001881122589111328s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017547607421875s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018906593322753906s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017404556274414062s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017499923706054688s, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Placed 5 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpr9bpzok5/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347219.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000247955322265625s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014090538024902344s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013828277587890625s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018143653869628906s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001761913299560547s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00016951560974121094s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000171661376953125s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017499923706054688s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016260147094726562s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00019693374633789062s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015735626220703125s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014972686767578125s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016546249389648438s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019073486328125s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001895427703857422s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016045570373535156s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015807151794433594s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015664100646972656s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001499652862548828s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017786026000976562s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001957416534423828s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016689300537109375s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017499923706054688s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00015926361083984375s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded ... ok sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001621246337890625s, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmppb3ruh6q/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022339820861816406s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001418590545654297s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014328956604003906s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014066696166992188s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018095970153808594s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016355514526367188s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025582313537597656s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015544891357421875s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015044212341308594s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002014636993408203s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00015997886657714844s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001838207244873047s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0008356571197509766s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014972686767578125s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015115737915039062s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014734268188476562s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016498565673828125s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016307830810546875s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015735626220703125s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016045570373535156s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015306472778320312s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one ... ok sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001537799835205078s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014925003051757812s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001475811004638672s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014662742614746094s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017142295837402344s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016069412231445312s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016188621520996094s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014925003051757812s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001690387725830078s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001595020294189453s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015807151794433594s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014829635620117188s, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp989_bkde/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021958351135253906s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013589859008789062s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001361370086669922s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013899803161621094s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002148151397705078s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001709461212158203s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024271011352539062s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001499652862548828s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017213821411132812s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016045570373535156s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00021839141845703125s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000152587890625s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001480579376220703s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016164779663085938s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016021728515625s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017595291137695312s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000152587890625s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015282630920410156s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014853477478027344s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014829635620117188s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017523765563964844s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016045570373535156s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001647472381591797s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017881393432617188s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001583099365234375s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019669532775878906s, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, active as of 1717346923.76713>, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp2u27x39o/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002391338348388672s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025343894958496094s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002148151397705078s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00023746490478515625s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00031566619873046875s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001704692840576172s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020003318786621094s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015282630920410156s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016951560974121094s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023746490478515625s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002117156982421875s, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347219.44239, (0, 0) as of 1717347219.44239, active as of 1717347219.44239>, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347219.44242, (0, 0) as of 1717347219.44242, active as of 1717347219.44242>, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347219.44244, (0, 0) as of 1717347219.44244, active as of 1717347219.44244>, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpe0qs3b60/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347219.44915.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp8vxqsjmx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp8vxqsjmx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp8vxqsjmx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmp8vxqsjmx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp_e7u0t8y/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp_e7u0t8y/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp_e7u0t8y/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmp_e7u0t8y/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp5iwnit_r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp5iwnit_r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp5iwnit_r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmp5iwnit_r/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpky5lv9cg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpky5lv9cg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpky5lv9cg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpky5lv9cg/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:53:39 2024 visited - attempted:2 success:2 failure:0 skipped:1 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted test.unit.container.test_sharder.TestSharder.test_one_shard_cycle test.unit.container.test_sharder.TestSharder.test_one_shard_cycle ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard ... ok sharder-test ERROR: Unhandled exception while processing: kapow!, path: a/c1, db: /tmp/tmp66t2jxr8/sdb/containers/0/ash/c1hash/c1hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2516, in _one_shard_cycle self._process_broker(broker, node, part) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1245, in _execute_mock_call result = effect(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 886, in mock_processing raise Exception('kapow!') Exception: kapow! sharder-test INFO: Since Sun Jun 2 04:53:39 2024 visited - attempted:3 success:2 failure:1 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:53:39 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:39 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: Unlinked retiring db '/tmp/tmp66t2jxr8/sda/containers/0/ash/c0hash/c0hash.db' sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Found no containers directories sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers ... ok sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.05s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 0) sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.01s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdc) sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.00s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdb, sdc) sharder-test INFO: Since Sun Jun 2 04:53:40 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:40 2024 audit_shard - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters ... ok test.unit.container.test_sharder.TestSharder.test_periodic_warning test.unit.container.test_sharder.TestSharder.test_periodic_warning ... ok test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard ... ok sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmpq1859jtr/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmpq1859jtr/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmpq1859jtr/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmpq1859jtr/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpx3egqeg9/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347220.33174.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmppka7ugq5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmppka7ugq5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmppka7ugq5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmppka7ugq5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded (deleted), path: a/c, db: /tmp/tmppka7ugq5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrinking' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharding' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharded' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others ... ok sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrunk' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmpi6khvtxv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmplt0080gd/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021076202392578125s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012421607971191406s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002105236053466797s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012540817260742188s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash60False/hash60False_1717347220.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002636909484863281s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018095970153808594s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022912025451660156s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001380443572998047s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash60True/hash60True_1717347239.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000209808349609375s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012636184692382812s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020623207092285156s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012254714965820312s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/lse/hash50False/hash50False_1717347258.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges ... ok sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000209808349609375s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001266002655029297s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022912025451660156s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012493133544921875s, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - zero objects found, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmpjrvporak/sda/containers/0/rue/hash50True/hash50True_1717347274.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h60/hash60/hash60_1717347221.35507.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h50/hash50/hash50_1717347221.41932.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin ... ok test.unit.container.test_sharder.TestSharder.test_run_forever test.unit.container.test_sharder.TestSharder.test_run_forever ... ok sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpq9wlqyag/sda/containers/0/h70/hash70/hash70_1717347221.48956.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:01 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 5.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 3604.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 13:00:10 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 30.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test ERROR: Unhandled exception while dumping progress: Test over, path: a/c2, db: /tmp/tmpfpytrva9/sda/containers/0/ash/c2hash/c2hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over sharder-test ERROR: Exception in sharder: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2553, in run_forever self._one_shard_cycle(devices_to_shard=Everything(), File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2530, in _one_shard_cycle self._periodic_report_stats() File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1143, in _periodic_report_stats if (time.time() - self.reported) >= self.stats_interval: ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_send_shard_ranges test.unit.container.test_sharder.TestSharder.test_send_shard_ranges ... ok sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpf3v532pp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002472400665283203s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014472007751464844s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj002_0 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmpzawa6eka/sda/containers/0/8fc/857fe81dc763e7cff7e7cd987d6c28fc/857fe81dc763e7cff7e7cd987d6c28fc.db, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaved ShardRange in 0.012s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00024175643920898438s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015020370483398438s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmpzawa6eka/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001690387725830078s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001437664031982422s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713> - shard db already in sync, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmpzawa6eka/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.584426879882812e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=True) in 9.036064147949219e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713> - shard db already in sync, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 0 required, shard db: /tmp/tmpzawa6eka/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.417533874511719e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.916854858398438e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713> - shard db already in sync, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaved ShardRange<'obj002' to 'obj004' as of 1717346923.76713, (0, 0) as of 1717346923.76713, cleaved as of 1717346923.76713> in 0.003s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj006_2 for ShardRange<'obj004' to 'obj006' as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.000240325927734375s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001704692840576172s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj006_2 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaved ShardRange<'obj004' to 'obj006' as of 1717346923.76713, (2, 2) as of 1717347222.13966, cleaved as of 1717346923.76713> in 0.02s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.918212890625e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.96453857421875e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00023889541625976562s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012993812561035156s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__3 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmpzawa6eka/sda/containers/0/683/f7a6bb6d9367b7ae7b6b30a833289683/f7a6bb6d9367b7ae7b6b30a833289683.db, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.34600830078125e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: got 0 rows (deleted=True) in 8.7738037109375e-05s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713>, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaving ShardRange<'obj006' to MaxBound as of 1717346923.76713, (0, 0) as of 1717346923.76713, created as of 1717346923.76713> - shard db already in sync, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache ... ok test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts ... ok test.unit.container.test_sharder.TestSharder.test_sharding_enabled test.unit.container.test_sharder.TestSharder.test_sharding_enabled ... ok test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump ... ok sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test INFO: Cleaved ShardRange<'obj006' to MaxBound as of 1717346923.76713, (3, 3) as of 1717347222.27445, cleaved as of 1717346923.76713> in 0.008s, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpzawa6eka/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347221.75643.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '215987b2-caf9-4b7a-9c47-7ba3a38d9634-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpb50eqsl2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347222.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '215987b2-caf9-4b7a-9c47-7ba3a38d9634-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpb50eqsl2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347222.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '215987b2-caf9-4b7a-9c47-7ba3a38d9634-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpb50eqsl2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347222.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '215987b2-caf9-4b7a-9c47-7ba3a38d9634-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpb50eqsl2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347222.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '831a59a7-0c7b-4551-ad8e-f05cd89ac606-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpb50eqsl2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347222.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp0xtfn_46/sda/containers/0/ash/c2hash/c2hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmp0xtfn_46/sda/containers/0/ash/c1hash/c1hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmp0xtfn_46/sda/containers/0/ash/c3hash/c3hash.db' sharder-test INFO: Since Sun Jun 2 04:53:42 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 04:53:42 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range ... ok test.unit.container.test_sharder.TestSharder.test_update_stat test.unit.container.test_sharder.TestSharder.test_update_stat ... ok sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmph9m9jytn/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmp6uobypd2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmp6uobypd2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmp6uobypd2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmp6uobypd2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmp6uobypd2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmp6uobypd2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 3, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=3, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 4, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=4, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 5, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=5, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 6, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=2, tombstones=6, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 7, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=1, tombstones=7, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 8, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=8, path: a/c, db: /tmp/tmphr16o9t6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=False) in 0.00041604042053222656s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=True) in 0.00034689903259277344s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=False) in 0.00022602081298828125s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects test.unit.container.test_sharder.TestSharder.test_yield_objects ... ok sharder-test DEBUG: got 5 rows (deleted=False) in 0.00026607513427734375s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.00025177001953125s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00025534629821777344s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00027298927307128906s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00026702880859375s, path: a/c, db: /tmp/tmparqryhp2/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002396106719970703s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002593994140625s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002243518829345703s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00028824806213378906s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00022602081298828125s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0001876354217529297s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016188621520996094s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.00018596649169921875s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016021728515625s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002493858337402344s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017881393432617188s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.0002739429473876953s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016117095947265625s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016760826110839844s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002009868621826172s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0001690387725830078s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00018739700317382812s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00017380714416503906s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00019478797912597656s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001709461212158203s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001609325408935547s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018334388732910156s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002262592315673828s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0001697540283203125s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0001881122589111328s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.0001895427703857422s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00021123886108398438s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.0001723766326904297s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016546249389648438s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00016951560974121094s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00017213821411132812s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00018715858459472656s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00018596649169921875s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00020241737365722656s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.0001735687255859375s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001621246337890625s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range ... ok test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects ... ok test.unit.obj.test_diskfile.TestDiskFile.test_close_error test.unit.obj.test_diskfile.TestDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit test.unit.obj.test_diskfile.TestDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_delete test.unit.obj.test_diskfile.TestDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bool test.unit.obj.test_updater.TestRateLimiterBucket.test_bool ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_len test.unit.obj.test_updater.TestRateLimiterBucket.test_len ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id ... ok sharder-test DEBUG: got 10 rows (deleted=False) in 0.0002155303955078125s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.000186920166015625s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.00029754638671875s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00019216537475585938s, path: a/c, db: /tmp/tmp15171go0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020551681518554688s, path: a/c, db: /tmp/tmpl5w61ciy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001246929168701172s, path: a/c, db: /tmp/tmpl5w61ciy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpfj0kefvj/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpfj0kefvj/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpfj0kefvj/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpfj0kefvj/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0100 "-" 3610542 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0088 "-" 3610542 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0103 "-" 3610542 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0083 "-" 3610542 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx13c295be329946fd8adf7-00665ca397" "proxy-server 3610542" 0.0016 "-" 3610542 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:35377/sda1/1/a/c" "tx13c295be329946fd8adf7-00665ca397" "container-server 3610542" 0.0014 "-" 3610542 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:44721/sdb1/1/a/c" "tx13c295be329946fd8adf7-00665ca397" "container-server 3610542" 0.0013 "-" 3610542 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx13c295be329946fd8adf7-00665ca397" "proxy-server 3610542" 0.0286 "-" 3610542 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx13c295be329946fd8adf7-00665ca397" "proxy-server 3610542" 0.0416 "-" 3610542 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/43 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx13c295be329946fd8adf7-00665ca397 - 0.0522 - - 1717347223.735923767 1717347223.788148880 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx73b0bcaf04e04296b4a8e-00665ca397" "proxy-server 3610542" 0.0016 "-" 3610542 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:44721/sdb1/1/a1/c1" "tx73b0bcaf04e04296b4a8e-00665ca397" "container-server 3610542" 0.0015 "-" 3610542 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:35377/sda1/1/a1/c1" "tx73b0bcaf04e04296b4a8e-00665ca397" "container-server 3610542" 0.0012 "-" 3610542 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx73b0bcaf04e04296b4a8e-00665ca397" "proxy-server 3610542" 0.0167 "-" 3610542 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx73b0bcaf04e04296b4a8e-00665ca397" "proxy-server 3610542" 0.0290 "-" 3610542 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/43 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx73b0bcaf04e04296b4a8e-00665ca397 - 0.0387 - - 1717347223.789921999 1717347223.828645229 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx4b2566dc2d6249b19ef8e-00665ca397" "proxy-server 3610542" 0.0078 "-" 3610542 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:35377/sda1/2/a/c1" "tx4b2566dc2d6249b19ef8e-00665ca397" "container-server 3610542" 0.0014 "-" 3610542 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:44721/sdb1/2/a/c1" "tx4b2566dc2d6249b19ef8e-00665ca397" "container-server 3610542" 0.0069 "-" 3610542 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx4b2566dc2d6249b19ef8e-00665ca397" "proxy-server 3610542" 0.0230 "-" 3610542 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx4b2566dc2d6249b19ef8e-00665ca397" "proxy-server 3610542" 0.0343 "-" 3610542 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/43 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx4b2566dc2d6249b19ef8e-00665ca397 - 0.0502 - - 1717347223.830392361 1717347223.880593300 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx7d67ad675c0147f393ae7-00665ca397" "proxy-server 3610542" 0.0078 "-" 3610542 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:35377/sda1/1/a/c2" "tx7d67ad675c0147f393ae7-00665ca397" "container-server 3610542" 0.0014 "-" 3610542 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:44721/sdb1/1/a/c2" "tx7d67ad675c0147f393ae7-00665ca397" "container-server 3610542" 0.0062 "-" 3610542 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx7d67ad675c0147f393ae7-00665ca397" "proxy-server 3610542" 0.0199 "-" 3610542 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:43 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx7d67ad675c0147f393ae7-00665ca397" "proxy-server 3610542" 0.0313 "-" 3610542 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/43 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx7d67ad675c0147f393ae7-00665ca397 - 0.0473 - - 1717347223.882339954 1717347223.929668427 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_server.TestProxyServer.test_check_response_503 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_server.TestProxyServer.test_denied_host_header ... ok test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_server.TestProxyServer.test_error_occurred test.unit.proxy.test_server.TestProxyServer.test_error_occurred ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test.unit.proxy.test_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_server.TestProxyServer.test_exception_occurred ... ok test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_info_defaults test.unit.proxy.test_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_server.TestProxyServer.test_node_timing test.unit.proxy.test_server.TestProxyServer.test_node_timing ... ok test.unit.proxy.test_server.TestProxyServer.test_private_method_request test.unit.proxy.test_server.TestProxyServer.test_private_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Setting object count to 1072 (txn: txb5169df382ca4679b7d67-00665ca391) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc3ee3a8f2e624261b4a40-00665ca394) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txc3ee3a8f2e624261b4a40-00665ca394) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc3ee3a8f2e624261b4a40-00665ca394) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txc3ee3a8f2e624261b4a40-00665ca394) test ERROR: Aborting listing from shards due to bad response: [404] (txn: txc3ee3a8f2e624261b4a40-00665ca394) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx48834ffdf0804f78964a0-00665ca396) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: tx4e70e8eb669545308300b-00665ca399) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txe4b9a55cc44149869b227-00665ca398) test INFO: Client disconnected on read of '/a/c' (txn: txe4b9a55cc44149869b227-00665ca398) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: tx7c9c553b464447c287669-00665ca398) test INFO: Client disconnected on read of '/a/c' (txn: tx7c9c553b464447c287669-00665ca398) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: tx14a34e8e40754cbfb0649-00665ca399) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: tx14a34e8e40754cbfb0649-00665ca399) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: tx14a34e8e40754cbfb0649-00665ca399) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: tx14a34e8e40754cbfb0649-00665ca399) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix ... ok test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_server.TestProxyServer.test_valid_api_version ... ok test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txdfc0b283a51045a597b34-00665ca39b) test DEBUG: Setting object count to 3120 (txn: txdfc0b283a51045a597b34-00665ca39b) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txf640ad571bff4922947e1-00665ca39b) test DEBUG: Setting object count to 3120 (txn: txf640ad571bff4922947e1-00665ca39b) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx400fca0fcebd41bfaa066-00665ca39b) test DEBUG: Setting object count to 3120 (txn: tx400fca0fcebd41bfaa066-00665ca39b) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txb10b8863c39d45228b1f5-00665ca39c) test DEBUG: Setting object count to 3120 (txn: txb10b8863c39d45228b1f5-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test DEBUG: Setting object count to 3120 (txn: tx1e77662fd3db4f45b0fad-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5798b1b1a8064747b5be8-00665ca39c) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx5798b1b1a8064747b5be8-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txd7d39a78bd7e4c5b820e7-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: txbf3bacf6abe543348a192-00665ca39c) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: txbf3bacf6abe543348a192-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txe01ea115a77c4c2cbfe66-00665ca39c) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: txe01ea115a77c4c2cbfe66-00665ca39c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe01ea115a77c4c2cbfe66-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe01ea115a77c4c2cbfe66-00665ca39c) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txe01ea115a77c4c2cbfe66-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txeefb4e95456d4a87b444d-00665ca39c) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txeefb4e95456d4a87b444d-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txeefb4e95456d4a87b444d-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txeefb4e95456d4a87b444d-00665ca39c) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txeefb4e95456d4a87b444d-00665ca39c) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx9cce2f24c97243648a158-00665ca39c) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx9cce2f24c97243648a158-00665ca39c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx9cce2f24c97243648a158-00665ca39c) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx9cce2f24c97243648a158-00665ca39c) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx9cce2f24c97243648a158-00665ca39c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb5f610e050c34c67bd9af-00665ca39e) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txb5f610e050c34c67bd9af-00665ca39e) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: txb5f610e050c34c67bd9af-00665ca39e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx944f752078524461a38e8-00665ca3a1) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body ... ok test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx944f752078524461a38e8-00665ca3a1) test DEBUG: Setting object count to 1065 (txn: tx944f752078524461a38e8-00665ca3a1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: txc2acea7ff4544574b3952-00665ca3a1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx2a1a0390730d4b999b738-00665ca3a1) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpimen8fe7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpimen8fe7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpimen8fe7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpimen8fe7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0136 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0112 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0103 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0112 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx9f84d2c4e9e641ef84d43-00665ca3a1" "proxy-server 3610488" 0.0017 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:58185/sda1/1/a/c" "tx9f84d2c4e9e641ef84d43-00665ca3a1" "container-server 3610488" 0.0016 "-" 3610488 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:44993/sdb1/1/a/c" "tx9f84d2c4e9e641ef84d43-00665ca3a1" "container-server 3610488" 0.0013 "-" 3610488 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx9f84d2c4e9e641ef84d43-00665ca3a1" "proxy-server 3610488" 0.0195 "-" 3610488 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx9f84d2c4e9e641ef84d43-00665ca3a1" "proxy-server 3610488" 0.0341 "-" 3610488 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/53 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx9f84d2c4e9e641ef84d43-00665ca3a1 - 0.0460 - - 1717347233.582159281 1717347233.628123999 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx731c1790715a4d318f1ab-00665ca3a1" "proxy-server 3610488" 0.0019 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:58185/sda1/1/a1/c1" "tx731c1790715a4d318f1ab-00665ca3a1" "container-server 3610488" 0.0017 "-" 3610488 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:44993/sdb1/1/a1/c1" "tx731c1790715a4d318f1ab-00665ca3a1" "container-server 3610488" 0.0016 "-" 3610488 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx731c1790715a4d318f1ab-00665ca3a1" "proxy-server 3610488" 0.0382 "-" 3610488 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx731c1790715a4d318f1ab-00665ca3a1" "proxy-server 3610488" 0.0259 "-" 3610488 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/53 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx731c1790715a4d318f1ab-00665ca3a1 - 0.0499 - - 1717347233.630040407 1717347233.679900885 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx23f9fa84f30f4a43b4f8d-00665ca3a1" "proxy-server 3610488" 0.0135 "-" 3610488 - /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", test.unit.proxy.test_mem_server.TestAccountController.test_DELETE test.unit.proxy.test_mem_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET test.unit.proxy.test_mem_server.TestAccountController.test_GET ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD test.unit.proxy.test_mem_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT test.unit.proxy.test_mem_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header ... ok /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:58185/sda1/2/a/c1" "tx23f9fa84f30f4a43b4f8d-00665ca3a1" "container-server 3610488" 0.0017 "-" 3610488 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:44993/sdb1/2/a/c1" "tx23f9fa84f30f4a43b4f8d-00665ca3a1" "container-server 3610488" 0.0103 "-" 3610488 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx23f9fa84f30f4a43b4f8d-00665ca3a1" "proxy-server 3610488" 0.0296 "-" 3610488 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx23f9fa84f30f4a43b4f8d-00665ca3a1" "proxy-server 3610488" 0.0458 "-" 3610488 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/53 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx23f9fa84f30f4a43b4f8d-00665ca3a1 - 0.0687 - - 1717347233.681845665 1717347233.750544786 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx4f605ea26fe14e3ba724d-00665ca3a1" "proxy-server 3610488" 0.0113 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:44993/sdb1/1/a/c2" "tx4f605ea26fe14e3ba724d-00665ca3a1" "container-server 3610488" 0.0184 "-" 3610488 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:58185/sda1/1/a/c2" "tx4f605ea26fe14e3ba724d-00665ca3a1" "container-server 3610488" 0.0016 "-" 3610488 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx4f605ea26fe14e3ba724d-00665ca3a1" "proxy-server 3610488" 0.0492 "-" 3610488 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:53 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx4f605ea26fe14e3ba724d-00665ca3a1" "proxy-server 3610488" 0.0371 "-" 3610488 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/53 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx4f605ea26fe14e3ba724d-00665ca3a1 - 0.0708 - - 1717347233.752546549 1717347233.823350668 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpvbnmoswz/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpvbnmoswz/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpvbnmoswz/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpvbnmoswz/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0357 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0335 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0301 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0142 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx6b53cfbfc75848388a8ce-00665ca3a2" "proxy-server 3610488" 0.0035 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:54905/sdb1/1/a/c" "tx6b53cfbfc75848388a8ce-00665ca3a2" "container-server 3610488" 0.0041 "-" 3610488 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:50171/sda1/1/a/c" "tx6b53cfbfc75848388a8ce-00665ca3a2" "container-server 3610488" 0.0024 "-" 3610488 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6b53cfbfc75848388a8ce-00665ca3a2" "proxy-server 3610488" 0.0359 "-" 3610488 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx6b53cfbfc75848388a8ce-00665ca3a2" "proxy-server 3610488" 0.0595 "-" 3610488 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/54 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx6b53cfbfc75848388a8ce-00665ca3a2 - 0.0865 - - 1717347234.894624233 1717347234.981137514 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:54 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txda3f79aeb2884bf5a5b38-00665ca3a2" "proxy-server 3610488" 0.0037 "-" 3610488 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:50171/sda1/1/a1/c1" "txda3f79aeb2884bf5a5b38-00665ca3a2" "container-server 3610488" 0.0040 "-" 3610488 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:54905/sdb1/1/a1/c1" "txda3f79aeb2884bf5a5b38-00665ca3a2" "container-server 3610488" 0.0044 "-" 3610488 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txda3f79aeb2884bf5a5b38-00665ca3a2" "proxy-server 3610488" 0.0386 "-" 3610488 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txda3f79aeb2884bf5a5b38-00665ca3a2" "proxy-server 3610488" 0.0635 "-" 3610488 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/55 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txda3f79aeb2884bf5a5b38-00665ca3a2 - 0.0818 - - 1717347234.984178782 1717347235.065955639 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx19bd80649fef4fa4af7a0-00665ca3a3" "proxy-server 3610488" 0.0146 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:50171/sda1/2/a/c1" "tx19bd80649fef4fa4af7a0-00665ca3a3" "container-server 3610488" 0.0028 "-" 3610488 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:54905/sdb1/2/a/c1" "tx19bd80649fef4fa4af7a0-00665ca3a3" "container-server 3610488" 0.0109 "-" 3610488 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx19bd80649fef4fa4af7a0-00665ca3a3" "proxy-server 3610488" 0.0666 "-" 3610488 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx19bd80649fef4fa4af7a0-00665ca3a3" "proxy-server 3610488" 0.0455 "-" 3610488 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/55 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx19bd80649fef4fa4af7a0-00665ca3a3 - 0.0999 - - 1717347235.068996191 1717347235.168937445 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd55f00a2c0cb4509aa854-00665ca3a3" "proxy-server 3610488" 0.0105 "-" 3610488 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:50171/sda1/1/a/c2" "txd55f00a2c0cb4509aa854-00665ca3a3" "container-server 3610488" 0.0120 "-" 3610488 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:54905/sdb1/1/a/c2" "txd55f00a2c0cb4509aa854-00665ca3a3" "container-server 3610488" 0.0025 "-" 3610488 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd55f00a2c0cb4509aa854-00665ca3a3" "proxy-server 3610488" 0.0446 "-" 3610488 2 test.unit.proxy.test_server.TestECObjectController.test_GET_ec test.unit.proxy.test_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults ... ok test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:53:55 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd55f00a2c0cb4509aa854-00665ca3a3" "proxy-server 3610488" 0.0635 "-" 3610488 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/53/55 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txd55f00a2c0cb4509aa854-00665ca3a3 - 0.1023 - - 1717347235.172264338 1717347235.274588108 2 ---------------------------------------------------------------------- Ran 8924 tests in 334.538s OK (skipped=1814) + stestr slowest Test id Runtime (s) ------------------------------------------------------------------------------------------------------------ ----------- test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges 44.239 test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving 39.475 test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET 34.149 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance 33.908 test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges 27.515 test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state 19.731 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving 17.791 test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock 17.176 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded 15.879 test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile 14.912 + rm -rf .stestr + [ 3.11 = disabled ] + echo+ 3.11 cut -d. -f1 + PYMAJOR=3 + echo ===> Testing with python (python3) ===> Testing with python (python3) + pwd + [ -d /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages ] + [ -z /build/reproducible-path/swift-2.33.0/debian/tmp/usr/lib/python3/dist-packages ] + [ -e .stestr.conf ] + [ -x /usr/bin/python3-stestr ] + STESTR=stestr + rm -rf .stestr + + subunit2pyunit PYTHON=python3.11 stestr run --parallel --subunit --test-path test/unit test\.unit\.(?!common\.test_utils\.TestUtils\.test_get_logger_sysloghandler_plumbing|common\.middleware\.test_cname_lookup\.TestCNAMELookup.*|common\.test_db\.TestDatabaseBroker\.test_get.*|container\.test_sync\.TestContainerSync\.test_init.*|common\.test_utils\.TestPunchHoleReally\.test_punch_a_hole.*|common\.test_utils\.Test_LibcWrapper\.test_argument_plumbing.*|common\.test_utils\.TestUtils\.test_load_pkg_resource_importlib.*|common\.test_db_replicator\.TestHandoffsOnly.*|common\.ring\.test_builder\.TestGetRequiredOverload\.test_perfect_four_zone_four_replica_bad_placement|common\.test_wsgi\.TestWSGI.*|obj\.test_expirer\.TestObjectExpirer\.test_delete_actual_object|common\.test_memcached\.TestMemcached\.test_get_conns_hostname6|common\.test_memcached\.TestMemcached\.test_get_conns_v6|common\.test_memcached\.TestMemcached\.test_get_conns_v6_default|obj\.test_reconstructor\.TestWorkerReconstructor\.test_run_forever_recon_aggregation|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_with_duplicate_but_sufficient_frag_indexes|proxy\.test_mem_server\.TestProxyServer\.test_statsd_prefix|proxy\.controllers\.test_obj\.TestECObjController\.test_GET_read_timeout|proxy\.controllers\.test_base\.TestGetOrHeadHandler\.test_disconnected_logging|obj\.test_expirer\.TestObjectExpirer\.test_failed_delete_continues_on|obj\.test_replicator\.TestObjectReplicator\.test_update) test.unit.cli.test_info.TestPrintObj.test_missing_etag test.unit.cli.test_info.TestPrintObj.test_missing_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid test.unit.cli.test_info.TestPrintObj.test_print_obj_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObj.test_print_obj_valid test.unit.cli.test_info.TestPrintObj.test_print_obj_valid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy test.unit.cli.test_info.TestPrintObj.test_print_obj_with_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_recon.TestScout.test_scout_http_error test.unit.cli.test_recon.TestScout.test_scout_http_error ... ok test.unit.cli.test_recon.TestScout.test_scout_ok test.unit.cli.test_recon.TestScout.test_scout_ok ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error test.unit.cli.test_recon.TestScout.test_scout_server_type_http_error ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok test.unit.cli.test_recon.TestScout.test_scout_server_type_ok ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_server_type_socket_timeout ... ok test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error test.unit.cli.test_recon.TestScout.test_scout_server_type_url_error ... ok test.unit.cli.test_recon.TestScout.test_scout_socket_timeout test.unit.cli.test_recon.TestScout.test_scout_socket_timeout ... ok test.unit.cli.test_recon.TestScout.test_scout_url_error test.unit.cli.test_recon.TestScout.test_scout_url_error ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_404_response ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_412_response ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_cipher_mismatch_for_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_decryption_override ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_different_keys_for_data_and_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_error_in_key_callback ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_metadata_not_encrypted ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_etag_crypto_meta ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_missing_key_callback ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_bad_body_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_ciphertext ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_content_type ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_missing_body_iv ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_no_body_crypto_meta ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_long_body_iv ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multipart_too_short_body_iv ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_multiseg_with_range ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_cipher ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/greenpool.py", line 87, in _spawn_n_impl func(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/x_profile/profile_model.py", line 274, in dump_profile profiler.dump_stats(tmpfn) File "/usr/lib/python3.11/profile.py", line 393, in dump_stats with open(file, 'wb') as f: ^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/tmp/log/swift/profile/default.profile3656228.tmp' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/greenpool.py", line 87, in _spawn_n_impl func(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/x_profile/profile_model.py", line 274, in dump_profile profiler.dump_stats(tmpfn) File "/usr/lib/python3.11/profile.py", line 393, in dump_stats with open(file, 'wb') as f: ^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/tmp/log/swift/profile/default.profile3656228.tmp' test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_override_etag_bad_iv ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_success ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_unencrypted_data_and_encrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_body_key_for_object_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_object_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_bad_key ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_body_key_for_object_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_crypto_meta_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_object_body ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_GET_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_404_response ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_412_response ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_different_keys_for_data_and_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_encrypted_data_and_unencrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_metadata_not_encrypted ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_cipher ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_override_etag_bad_iv ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_success ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_unencrypted_data_and_encrypted_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_bad_key ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_crypto_meta_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_HEAD_with_missing_iv_for_user_metadata ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case test.unit.common.middleware.crypto.test_decrypter.TestDecrypterObjectRequests.test_headers_case ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_get_acl_handler ... ok test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl test.unit.common.middleware.s3api.test_acl_handlers.TestAclHandlers.test_handle_acl ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_error ... ok test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs test.unit.cli.test_ring_builder_analyzer.TestRunScenario.test_it_runs ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_subresource ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceNoAcl.test_service_GET_with_blind_resource ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults test.unit.common.middleware.s3api.test_utils.TestConfig.test_defaults ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete test.unit.common.middleware.s3api.test_utils.TestConfig.test_set_get_delete ... ok test.unit.common.middleware.s3api.test_utils.TestConfig.test_update test.unit.common.middleware.s3api.test_utils.TestConfig.test_update ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_not_encrypted_obj ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_json_with_crypto_override ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_GET_container_success ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_cipher_mismatch ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id test.unit.common.middleware.crypto.test_decrypter.TestDecrypterContainerRequests.test_cont_get_json_req_with_unknown_secret_id ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_owner_permission ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_internal_client.py", line 904, in __call__ raise Exception() Exception test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_dir ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file test.unit.common.middleware.test_dlo.TestDloConfiguration.test_finding_defaults_from_file ... ok test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured test.unit.common.middleware.test_dlo.TestDloConfiguration.test_skip_defaults_if_configured ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_and_HEAD_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_overrides_uploaded ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_GET_registered_with_query_string ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_POST ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_uploaded_with_query_string ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST test.unit.common.middleware.test_helpers.TestFakeSwift.test_PUT_with_query_string_POST ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods test.unit.common.middleware.test_helpers.TestFakeSwift.test_allowed_methods ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered test.unit.common.middleware.test_helpers.TestFakeSwift.test_not_registered ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy test.unit.common.middleware.test_helpers.TestFakeSwift.test_object_GET_updated_with_storage_policy ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range test.unit.common.middleware.test_helpers.TestFakeSwift.test_range ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_ignored ... ok test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift test.unit.common.middleware.test_helpers.TestFakeSwift.test_range_ignore_range_header_old_swift ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_mismatched_location ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_POST_regular_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_DLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_SLO_with_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_object_with_DLO ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_PUT_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_delete_versions_cont ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_fail_object_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_delete_container_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_primary_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_enable_versioning_failed_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_existing_container_has_deprecated_policy_unauthed ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_reserved_container_passthrough ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_get_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_container_with_super_legacy_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_existing_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCase.test_same_policy_as_primary_container ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_ec_symlink_range_unsatisfiable_can_redirect_to_target ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_non_symlink ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_mismatched_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_static_link_to_symlink_fails ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_not_found ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_object ... ok test ERROR: Error decrypting container listing: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: get_keys(): unknown key id: unknown_key test ERROR: Error decrypting container listing: unknown_key test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_with_read_acp_permission ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_to_reserved_symlink ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_symlink_with_account ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object ... ok test.unit.cli.test_reload.TestMain.test_check_failed test.unit.cli.test_reload.TestMain.test_check_failed ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_not_found ... ok test.unit.cli.test_reload.TestMain.test_good test.unit.cli.test_reload.TestMain.test_good ... ok test.unit.cli.test_reload.TestMain.test_needs_pid test.unit.cli.test_reload.TestMain.test_needs_pid ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_get_target_object_range_not_satisfiable ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink ... ok test.unit.cli.test_reload.TestMain.test_timeout test.unit.cli.test_reload.TestMain.test_timeout ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_symlink_with_account ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_head_target_object ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_non_symlink_post ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_GET ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_acl_PUT ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_canned_acl_PUT ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_set_symlink_POST_fail ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_both_acl_header_and_xml_PUT ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_sym_to_sym_to_target ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_change_symloopmax ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_handle_acl_with_invalid_header_string ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_invalid_xml ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_chunked_put_error ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_info ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclNoSetup.test_object_acl_GET ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_aws_chunked ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_format_authorization ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_post_but_fail_at_server ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_account_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_bad_object_header ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_different_account ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_leading_slash ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bad_method_but_method_exists_in_controller ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_non_zero_length ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_bucket_virtual_hosted_style ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_put_with_prevalidated_etag_sysmeta_incomplete ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v2 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_canonical_string_v4 ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_error ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_check_pipeline ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_to_non_existing_object_override ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_cors ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_content_type ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_explicit_content_type ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_etag_target_missing_content_type ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_config ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_logger ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_init_passes_wsgi_conf_file_to_check_pipeline ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_quoted_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_simple_put_with_unmatched_etag ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symlink_too_deep ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_metadata_directive ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_symloop_max_config ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_ssc ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_invalid_format ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_storage_class ... ok test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself test.unit.common.middleware.test_symlink.TestSymlinkMiddleware.test_validate_and_prep_request_headers_points_to_itself ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_invalid_uri ... ok test.unit.common.test_manager.TestManagerModule.test_command_wrapper test.unit.common.test_manager.TestManagerModule.test_command_wrapper ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mfa ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_mixture_param_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_ascii_user ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_non_s3_request_passthrough ... ok test.unit.common.test_manager.TestManagerModule.test_exc test.unit.common.test_manager.TestManagerModule.test_exc ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_notification ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.test_manager.TestManagerModule.test_format_server_name test.unit.common.test_manager.TestManagerModule.test_format_server_name ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_bad_padding ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_GET_without_permission ... ok test.unit.common.test_manager.TestManagerModule.test_safe_kill test.unit.common.test_manager.TestManagerModule.test_safe_kill ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_long ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_too_short ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_create_bad_md5_unreadable ... ok test.unit.common.test_manager.TestManagerModule.test_servers test.unit.common.test_manager.TestManagerModule.test_servers ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_tagging ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_object_virtual_hosted_style ... ok test.unit.common.test_manager.TestManagerModule.test_setup_env test.unit.common.test_manager.TestManagerModule.test_setup_env ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_manager.TestManagerModule.test_verify_server test.unit.common.test_manager.TestManagerModule.test_verify_server ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_path_info_encode ... ok test.unit.common.test_manager.TestManagerModule.test_watch_server_pids test.unit.common.test_manager.TestManagerModule.test_watch_server_pids ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_policy ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access test.unit.common.test_utils.TestAuditLocationGenerator.test_drive_tree_access ... ok test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count test.unit.cli.test_recon_cron.TestReconCron.test_get_async_count ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_defaults ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter test.unit.common.test_utils.TestAuditLocationGenerator.test_error_counter ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_fromstring_with_nonascii_text ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjWeirdPath.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_registered_sensitive_info ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_filters test.unit.common.test_utils.TestAuditLocationGenerator.test_filters ... ok test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag test.unit.cli.test_info.TestPrintObjWeirdPath.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_tostring_with_nonascii_text ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_namespace ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_request_payment ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects test.unit.common.test_utils.TestAuditLocationGenerator.test_find_objects ... ok test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception test.unit.common.middleware.crypto.test_decrypter.TestDecrypter.test_app_exception ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_restore ... ok test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments test.unit.common.middleware.s3api.test_etree.TestS3ApiEtree.test_xml_with_comments ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks test.unit.common.test_utils.TestAuditLocationGenerator.test_hooks ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_error_metric ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_initialization_with_disabled_pipeline_check ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata test.unit.common.test_utils.TestAuditLocationGenerator.test_ignore_metadata ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_mount_check_drive ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_contents ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token ... ok test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive test.unit.common.test_utils.TestAuditLocationGenerator.test_non_dir_drive ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_in_s3acl ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_only_s3_token_v3 ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor test.unit.common.test_utils.TestFsHasFreeSpace.test_bytes_using_file_descriptor ... ok test.unit.common.test_utils.TestFsHasFreeSpace.test_percent test.unit.common.test_utils.TestFsHasFreeSpace.test_percent ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_big ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_brittle ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_empty ... ok test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish test.unit.common.test_utils.TestHashForFileFunction.test_hash_for_file_smallish ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filecount_arg ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_filesize_arg ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_path_arg ... ok test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg test.unit.cli.test_form_signature.TestFormSignature.test_invalid_seconds_arg ... ok test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature test.unit.cli.test_form_signature.TestFormSignature.test_prints_signature ... ok test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args test.unit.cli.test_form_signature.TestFormSignature.test_too_few_args ... ok test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args test.unit.cli.test_info.TestCliInfo.test_parse_get_node_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_many_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis test.unit.cli.test_info.TestCliInfo.test_print_db_info_metadata_with_shard_ranges_bis ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_db_syncs test.unit.cli.test_info.TestCliInfo.test_print_db_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_info test.unit.cli.test_info.TestCliInfo.test_print_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_dashed_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_object_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_container_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring test.unit.cli.test_info.TestCliInfo.test_print_item_locations_account_with_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition test.unit.cli.test_info.TestCliInfo.test_print_item_locations_dashed_ring_name_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_invalid_policy_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object test.unit.cli.test_info.TestCliInfo.test_print_item_locations_policy_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target test.unit.cli.test_info.TestCliInfo.test_print_item_locations_ring_policy_mismatch_no_target ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_invalid_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending test.unit.account.test_backend.TestCommonAccountBroker.test_commit_pending ... ok test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number test.unit.cli.test_info.TestCliInfo.test_print_ring_locations_partition_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_app_exception_v3 ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_filter_v3 ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_from_external_file ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_legacy_key_id ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_get_root_secret_multiple_keys ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_auth_url ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_bad_user_domain_name ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_config_file_and_params ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_connect_failure_auth_url ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_algorithm ... ok test.unit.account.test_server.TestAccountController.test_DELETE_empty test.unit.account.test_server.TestAccountController.test_DELETE_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_empty ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_format ... ok test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition test.unit.account.test_server.TestAccountController.test_DELETE_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_id ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_key_length ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_password ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_invalid_username ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_none_key ... ok test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory test.unit.common.middleware.crypto.test_crypto.TestCrypto.test_filter_factory ... ok test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id test.unit.common.middleware.crypto.test_kms_keymaster.TestKmsKeymaster.test_mocked_castellan_keymanager_nonexistent_key_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorization_nova_toconnect ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorize_with_access_key_in_container ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_error_with_suspended_versioning ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_http ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_error ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_authorized_v3 ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_is_truncated_url_encoded ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_last_modified_rounding ... ok test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails test.unit.account.test_auditor.TestAuditorRealBroker.test_db_validate_fails ... ok test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.account.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_negative_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_non_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_over_32bit_int_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_passthroughs ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_str_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_subdir_with_delimiter_max_keys ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_url_encoded ... ok test.unit.account.test_replicator.TestReplicatorSync.test_local_ids test.unit.account.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_fetch_owner ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_is_truncated ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_passthroughs ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_delimiter_max_keys ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_v2_with_nonascii_queries ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_non_existent_version_id_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_key_marker_and_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_versions_with_version_id_marker ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_delimiter_max_keys ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_nonascii_queries ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_versioning_not_configured ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_GET_with_versions_with_max_keys_and_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_503 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_policy_index_logging ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_complete ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_HEAD_slash_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_missing_key ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_bucket_already_owned_by_you ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_success_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_bucket_upload_part_v4_bad_hash ... ok Reloaded swift-proxy-server test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Use tempauth middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: Use keystone middleware. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test WARNING: s3api middleware requires SLO middleware to support multi-part upload, please add it in pipeline test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: Skip pipeline auth check. test DEBUG: 's3api' is not in list test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Use of the auth_admin_prefix, auth_host, auth_port, auth_protocol, identity_uri, admin_token, admin_user, admin_password, and admin_tenant_name configuration options was deprecated in the Mitaka release in favor of an auth_plugin and its related options. This class may be removed in a future release. Configuring admin URI using auth fragments was deprecated in the Kilo release, and will be removed in the Newton release, use 'identity_uri' instead. Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint test DEBUG: Calling S3Api Middleware Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_s3_token_and_auth_token ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_s3api_with_time_skew ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_server_side_encryption ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4 ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_bad_authorization_string ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_date ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signature_v4_no_payload ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db test.unit.account.test_backend.TestCommonAccountBroker.test_delete_db ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_empty ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_empty test.unit.account.test_server.TestAccountController.test_DELETE_not_empty ... ok test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.container.test_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_batched_reclaim ... ok test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.account.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim test.unit.account.test_backend.TestAccountBroker.test_batched_reclaim ... ok test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration test.unit.account.test_auditor.TestAuditorRealBrokerMigration.test_db_migration ... ok test.unit.account.test_reaper.TestReaper.test_basic_reap_account test.unit.account.test_reaper.TestReaper.test_basic_reap_account ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_info test.unit.account.test_backend.TestCommonAccountBroker.test_get_info ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_chexor ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row test.unit.account.test_backend.TestCommonAccountBroker.test_get_max_row ... ok test.unit.account.test_server.TestAccountController.test_DELETE_not_found test.unit.account.test_server.TestAccountController.test_DELETE_not_found ... ok test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.account.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.account.test_backend.TestAccountBroker.test_chexor test.unit.account.test_backend.TestAccountBroker.test_chexor ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata test.unit.account.test_backend.TestCommonAccountBroker.test_get_raw_metadata ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_creation ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync test.unit.account.test_replicator.TestReplicatorSync.test_sync ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_expired ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs test.unit.account.test_backend.TestCommonAccountBroker.test_get_syncs ... ok test.unit.account.test_reaper.TestReaper.test_creation test.unit.account.test_reaper.TestReaper.test_creation ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_now_empty ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get test.unit.account.test_backend.TestCommonAccountBroker.test_maybe_get ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_container ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_invalid_expire ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_encoding_type_error ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_bad_value ... ok test.unit.account.test_server.TestAccountController.test_DELETE_now_empty test.unit.account.test_server.TestAccountController.test_DELETE_now_empty ... ok test.unit.account.test_backend.TestAccountBroker.test_creation test.unit.account.test_backend.TestAccountBroker.test_creation ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_error ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_default ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_max_parts ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_delete_db_status ... ok test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float test.unit.account.test_server.TestAccountController.test_DELETE_timestamp_not_float ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_access ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_max_parts ... ok test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set test.unit.account.test_reaper.TestReaper.test_delay_reaping_conf_set ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_sign ... ok test.unit.account.test_backend.TestAccountBroker.test_delete_container test.unit.account.test_backend.TestAccountBroker.test_delete_container ... ok test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.account.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_empty ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_application_wildcard ... ok test.unit.account.test_reaper.TestReaper.test_reap_account test.unit.account.test_reaper.TestReaper.test_reap_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_application_wildcard ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_conflicting ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_no_timestamp ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_negative_part_number_marker ... ok test.unit.account.test_backend.TestAccountBroker.test_delete_db_status test.unit.account.test_backend.TestAccountBroker.test_delete_db_status ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting test.unit.account.test_server.TestAccountController.test_GET_accept_conflicting ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_json ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_good test.unit.cli.test_reload.TestValidateManagerPid.test_good ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_exception ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4 ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_no_container test.unit.account.test_reaper.TestReaper.test_reap_account_no_container ... ok test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBroker.test_double_check_trailing_delimiter ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_python test.unit.cli.test_reload.TestValidateManagerPid.test_non_python ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_json test.unit.account.test_server.TestAccountController.test_GET_accept_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_wildcard ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_bad_credential ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_server test.unit.cli.test_reload.TestValidateManagerPid.test_non_server ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_max_parts ... ok test.unit.account.test_backend.TestAccountBroker.test_empty test.unit.account.test_backend.TestAccountBroker.test_empty ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_xml ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change test.unit.account.test_backend.TestCommonAccountBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_account_with_sharding ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid test.unit.account.test_server.TestAccountController.test_GET_accept_not_valid ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.account.test_backend.TestAccountBroker.test_exception test.unit.account.test_backend.TestAccountBroker.test_exception ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard test.unit.account.test_server.TestAccountController.test_GET_accept_wildcard ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_algorithm ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_json ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift test.unit.cli.test_reload.TestValidateManagerPid.test_non_swift ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_over_part_number_marker ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_plain ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_open_error test.unit.cli.test_reload.TestValidateManagerPid.test_open_error ... ok test.unit.account.test_reaper.TestReaper.test_reap_container test.unit.account.test_reaper.TestReaper.test_reap_container ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info test.unit.account.test_backend.TestAccountBroker.test_get_info ... ok test.unit.account.test_server.TestAccountController.test_GET_accept_xml test.unit.account.test_server.TestAccountController.test_GET_accept_xml ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_invalid_credentials ... ok test.unit.cli.test_reload.TestValidateManagerPid.test_worker test.unit.cli.test_reload.TestValidateManagerPid.test_worker ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp test.unit.account.test_backend.TestCommonAccountBroker.test_put_timestamp ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_json test.unit.account.test_server.TestAccountController.test_GET_empty_account_json ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail test.unit.account.test_reaper.TestReaper.test_reap_container_full_fail ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBroker.test_get_info_no_stale_reads ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain test.unit.account.test_server.TestAccountController.test_GET_empty_account_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_empty_account_xml ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_max_parts ... ok test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at test.unit.account.test_backend.TestCommonAccountBroker.test_status_changed_at ... ok test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_bad_composite_builder_file ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_get_policy_stats ... ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBroker.test_get_info_stale_read_ok ... ok test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml test.unit.account.test_server.TestAccountController.test_GET_empty_account_xml ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signature ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose test.unit.cli.test_ringcomposer.TestCommands.test_compose ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_fail ... ok test.unit.account.test_server.TestAccountController.test_GET_invalid_accept test.unit.account.test_server.TestAccountController.test_GET_invalid_accept ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_initialize_fail ... ok test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter test.unit.account.test_server.TestAccountController.test_GET_leading_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_str_part_number_marker ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_signed_headers ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_container_get_object_timeout ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_is_status_deleted ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing test.unit.cli.test_ringcomposer.TestCommands.test_compose_existing ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json test.unit.account.test_server.TestAccountController.test_GET_limit_marker_json ... ok test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats test.unit.account.test_backend.TestAccountBroker.test_get_policy_stats ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_signed_urls_v4_missing_x_amz_date ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_json ... ok test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_container_non_exist_policy_index ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_with_part_number_marker ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_builder_file ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_initialize_fail test.unit.account.test_backend.TestAccountBroker.test_initialize_fail ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_list_parts_without_encoding_type ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_plain ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_container_iter_with_reserved_name ... ok test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain test.unit.account.test_server.TestAccountController.test_GET_limit_marker_plain ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_tagging ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit_marker_xml ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_fails_to_write_composite_ring_file ... ok test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted test.unit.account.test_backend.TestAccountBroker.test_is_status_deleted ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_crypto_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml test.unit.account.test_server.TestAccountController.test_GET_limit_marker_xml ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files test.unit.cli.test_ringcomposer.TestCommands.test_compose_insufficient_component_builder_files ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_token_generation ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBroker.test_list_container_iter_with_reserved_name ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv4 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_torrent ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter test.unit.account.test_backend.TestAccountBroker.test_list_containers_iter ... ok test.unit.account.test_server.TestAccountController.test_GET_log_requests_false test.unit.account.test_server.TestAccountController.test_GET_log_requests_false ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_curl_command_ipv6 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_containers_iter ... ok test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_compose_nonexistent_component_builder_file ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_unsupported_method ... ok test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBroker.test_load_old_pending_puts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_meta_and_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_backend.TestAccountBroker.test_merge_items test.unit.account.test_backend.TestAccountBroker.test_merge_items ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_load_old_pending_puts ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show test.unit.cli.test_ringcomposer.TestCommands.test_show ... ok test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBroker.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website ... ok test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file test.unit.cli.test_ringcomposer.TestCommands.test_show_nonexistent_composite_builder_file ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_no_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multi_upload_empty_body ... ok test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command test.unit.cli.test_ringcomposer.TestCommands.test_unknown_command ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items ... ok test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location test.unit.common.middleware.s3api.test_s3api.TestS3ApiMiddleware.test_website_redirect_location ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_app_exception ... ok test.unit.account.test_backend.TestAccountBroker.test_newid test.unit.account.test_backend.TestAccountBroker.test_newid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_can_only_configure_secret_in_one_place ... ok test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBroker.test_policy_stats_tracking ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_bad_reply ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_chained_keymasters ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_fail_to_connect_to_keystone ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch test.unit.cli.test_info.TestPrintObjFullMeta.test_print_obj_policy_name_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.account.test_backend.TestAccountBroker.test_put_container test.unit.account.test_backend.TestAccountBroker.test_put_container ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_container_path ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestDeferredAuth.test_unauthorized_token ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_newid ... ok test.unit.account.test_backend.TestAccountBroker.test_reclaim test.unit.account.test_backend.TestAccountBroker.test_reclaim ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_unchecked_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_correct_root_secret_used ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_policy_stats_tracking ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_put_container ... ok test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBroker.test_reverse_prefix_delim ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_filter ... ok test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag test.unit.cli.test_info.TestPrintObjFullMeta.test_valid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers test.unit.common.middleware.crypto.test_decrypter.TestModuleMethods.test_purge_crypto_sysmeta_headers ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_error ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_acl_GET ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_canned_acl_PUT_with_s3acl ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_invalid_root_secret ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reclaim ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_key_uniqueness ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keymaster_config_path ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_no_content_type ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_matching_ending_not_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_container ... ok test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_keys_cached ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_account_with_path_root_unicode_container ... ok test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder test.unit.cli.test_dispersion_report.TestDispersionReport.test_placeholder ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_old_content_type ... ok test.unit.cli.test_recon.TestReconCommands.test_disk_usage test.unit.cli.test_recon.TestReconCommands.test_disk_usage ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_missing_conf_section ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5 ... ok test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch test.unit.cli.test_recon.TestReconCommands.test_get_swiftconfmd5_mismatch ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_batched_reclaim ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.cli.test_recon.TestReconCommands.test_load_check test.unit.cli.test_recon.TestReconCommands.test_load_check ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets ... ok test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types test.unit.cli.test_recon.TestReconCommands.test_multiple_server_types ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_missing_active_root_secret_id ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_id ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_chexor ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_container_table_migration ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_multiple_root_secrets_with_invalid_secret ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_creation ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_batched_reclaim ... ok test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check test.unit.cli.test_recon.TestReconCommands.test_object_auditor_check ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_container ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_no_root_secret ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_chexor ... ok test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check test.unit.cli.test_recon.TestReconCommands.test_reconstruction_check ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_paginated ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_delete_db_status ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_object_path ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.cli.test_recon.TestReconCommands.test_replication_check test.unit.cli.test_recon.TestReconCommands.test_replication_check ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.cli.test_recon.TestReconCommands.test_server_type_check test.unit.cli.test_recon.TestReconCommands.test_server_type_check ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_empty ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_exception ... ok test.unit.cli.test_recon.TestReconCommands.test_sharding_check test.unit.cli.test_recon.TestReconCommands.test_sharding_check ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_creation ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_root_secret_path_invalid_secret ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_unicode_object_path ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check test.unit.cli.test_recon.TestReconCommands.test_time_check ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_complete_zero_segments ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v1_keys_with_weird_paths ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_jitter_mismatch ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys test.unit.common.middleware.crypto.test_keymaster.TestKeymaster.test_v2_keys ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_get_policy_stats ... ok test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch test.unit.cli.test_recon.TestReconCommands.test_time_check_mismatch ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_get_object_lock ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_half_upgraded_database ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check test.unit.cli.test_recon.TestReconCommands.test_version_check ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock test.unit.common.middleware.s3api.test_object_lock.TestS3ApiObjectLock.test_put_object_lock ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.cli.test_recon.TestReconCommands.test_version_check_differs test.unit.cli.test_recon.TestReconCommands.test_version_check_differs ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_md5 ... ok test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_bulk.TestSwiftInfo.test_registered_defaults ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_initialize_fail ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_add ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_invalid_sha256 ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_is_status_deleted ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_bad_destination_account ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_overload ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_destination_leading_slash ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_container_iter_with_reserved_name ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_part_power ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_containers_iter ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_container ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_random_seed ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_complete ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_remove ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_newest ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_delete_db_status ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_no_object_in_destination ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_multipart_uploads_initiate ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_replicas ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_missing_key ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_load_old_pending_puts ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_not_found_reading_source ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_double_check_trailing_delimiter ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_rounds ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_server_error_reading_source ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_save ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_success_mixed_policy ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_empty ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_larger_than_max_file_size ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_bad_set_weight ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_good ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_exception ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_with_slashes_in_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_bucket_upload_part_v4_bad_hash ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_newid ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_upload_upper_sha256 ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_invalid_json ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_source_zero_content_length ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_stats_tracking ... ok test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object test.unit.cli.test_ring_builder_analyzer.TestParseScenario.test_json_not_object ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_account_with_metadata ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_object_upload_part_error ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_no_stale_reads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_camel_to_snake ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_across_containers ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_info_stale_read_ok ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_encoding_type_error ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_destination_leading_slash ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_get_policy_stats ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_mktime_weird_tz ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_error ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_match ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_initialize_fail ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_snake_to_camel ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_max_parts ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_in_OPTIONS_response_CORS ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_newest ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_headers_with_not_match ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_is_status_deleted ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_max_parts ... ok test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false test.unit.common.middleware.s3api.test_utils.TestS3ApiUtils.test_validate_bucket_name_with_dns_compliant_bucket_names_false ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '462'} test DEBUG: Grant test:full_control FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '446'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_container_iter_with_reserved_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_other_owner ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_negative_part_number_marker ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_defaults ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_containers_iter ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_destination_header ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_invalid ... ok test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_domain_remap.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_list_objects_iter_order_and_reverse ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_no_object_in_destination ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_32bit_int_part_number_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadNonUTC.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_load_old_pending_puts ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_not_found_reading_source ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_containers_prefix ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_dict ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestAccountOperations.test_list_orphan_hidden_containers ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_server_error_reading_source ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_max_parts ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_obj_is_unversioned ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_over_part_number_marker ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_merge_items_overwrite_unicode ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_no_updates ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_same_max_marts_as_objects_num ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_migrate_add_storage_policy_index_fail ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_headers_with_updates ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_max_parts ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_current_version_is_delete_marker ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_newid ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_larger_than_max_file_size ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_str_part_number_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_not_current_version ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_impossible ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_metadata ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_with_part_number_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_DELETE_null_version ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_deleted ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_destination_leading_slash ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_list_parts_without_encoding_type ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_404 ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_with_slashes_in_name ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_and_HEAD ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_source_zero_content_length ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_metadata ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_404 ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_GET_null_id_versioned_obj ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_with_single_range ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_no_updates ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_delete_marker ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_COPY_x_fresh_metadata_with_updates ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_s3acl_without_segment_bucket_mixed_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_HEAD_null_id ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_with_segment_bucket_mixed_policy ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_POST_error ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_COPY_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_make_public ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_acls ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_mpu_initiate_without_segment_bucket_same_policy ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_DLO_manifest ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_container_not_found ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_invalid ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_not_found ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_PUT_version_with_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_404_on_marker_delete ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_basic_put_with_x_copy_from_across_container_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_409_on_marker_delete ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestVersionAPI.test_fail_non_versioned_container ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_error ... ok test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawOldManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_non_zero_content_length_with_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_failure_with_heartbeat ... ok test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning test.unit.common.middleware.test_tempurl.TestTempurlWarning.test_deprecation_warning ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_in_future ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_marker_ts_now ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download test.unit.common.middleware.test_xprofile.Test_html_viewer.test_download ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_mixed_policy ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code test.unit.common.middleware.test_xprofile.Test_html_viewer.test_format_source_code ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_not_found_reading_source_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_no_content_type ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param test.unit.common.middleware.test_xprofile.Test_html_viewer.test_get_param ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_non_ascii ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_server_error_reading_source_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_old_content_type ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_source_larger_than_max_file_size ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all test.unit.common.middleware.test_xprofile.Test_html_viewer.test_index_all ... ok test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot test.unit.common.middleware.test_xprofile.Test_html_viewer.test_plot ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_other_headers ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_bad_x_copy_from_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_segment_too_small ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_and_slashes_in_x_copy_from_acct ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_single_zero_length_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_weird_host_name ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_with_heartbeat ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_length_final_segment ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_complete_zero_segments ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_s3acl_with_cached_seg_buck ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_with_segment_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_bucket ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_leading_slash_in_x_copy_from_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_initiate_without_segment_bucket ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_no_object_in_x_copy_from_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_md5 ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_object_metadata_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_invalid_sha256 ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_missing_part_with_heartbeat ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_slashes_in_x_copy_from_and_account ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_etag_mismatch ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_mixed_policy ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_spaces_in_x_copy_from_and_account ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_encrypted_to_unencrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_copy_with_unicode ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_nothing_there ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_no_object_in_path_pass_through ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_copy_unencrypted_to_encrypted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_retry_complete_upload_id_mismatch ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_object_pass_through_methods ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_upload_upper_sha256 ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_UTF8_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_multipart_uploads_list ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_crypto_max_length_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest test.unit.common.middleware.test_copy.TestServerSideCopyMiddleware.test_static_large_object_manifest ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_disable_encryption_config_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_object_upload_part_error ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_match ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_headers_with_not_match ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_ondisk_data_after_write_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_disabled ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_put_without_crypto_post_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_invalid ... ok test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled test.unit.common.middleware.test_healthcheck.TestHealthCheck.test_healthcheck_pass_not_disabled ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_and_override_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUpload.test_upload_part_copy_range_unsatisfiable ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_get ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req test.unit.common.middleware.s3api.test_s3request.TestRequest.test_authenticate_delete_Authorization_from_s3req ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderProject.test_reader_put ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv2 ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_account_unknown_domain ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_known_domain ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_different_root_secrets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_canonical_uri_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_multi_bytes_secret_failure ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2 ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_existing_v2_account ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv2_unicode_string ... ok test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account test.unit.common.middleware.test_keystoneauth.TestSetProjectDomain.test_set_project_domain_id_new_account ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_signature_sigv4_url_encode ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_check_sigv4_req_zero_content_length_sha256 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_expired ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_missing ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_endpoint ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_get_object_ring ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_expired ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid test.unit.common.middleware.s3api.test_s3request.TestRequest.test_date_header_with_x_amz_date_valid ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_container_info ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_container_permission ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_account_that_looks_like_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_no_version_specified ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_invalid_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_no_account ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_path_with_valid_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_response_version ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_parse_version_that_looks_like_account ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v1_response ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_non_obj_response ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_v2_obj_response ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_with_check_object_permission ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_duplication_HEAD_request ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_match_ACL_MAP ... ok test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response test.unit.common.middleware.test_list_endpoints.TestListEndpoints.test_version_account_response ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail test.unit.common.middleware.test_quotas.TestContainerQuotas.test_auth_fail ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_delete_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_invalid_quotas ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container test.unit.common.middleware.test_quotas.TestContainerQuotas.test_missing_container ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_no_quotas ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_response_without_s3_acl ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param test.unit.common.middleware.s3api.test_s3request.TestRequest.test_get_validate_param ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_exceed_counts_quota ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled test.unit.common.middleware.test_quotas.TestContainerQuotas.test_not_handled ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment test.unit.common.middleware.test_quotas.TestContainerQuotas.test_split_path_empty_container_path_segment ... ok test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas test.unit.common.middleware.test_quotas.TestContainerQuotas.test_valid_quotas ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges test.unit.common.middleware.test_slo.TestSloGetRawManifest.test_get_raw_manifest_passthrough_with_ranges ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_first_byte ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_headers_to_sign_sigv4 ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_manifest_unsatisfiable ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names test.unit.account.test_utils.TestAccountUtils.test_account_listing_reserved_names ... ok test.unit.account.test_utils.TestAccountUtils.test_account_listing_response test.unit.account.test_utils.TestAccountUtils.test_account_listing_response ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_empty_memory_broker ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_fake_broker ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_data ... ok test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data test.unit.account.test_utils.TestAccountUtils.test_get_response_headers_with_legacy_data ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_info ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequestsOldManifest.test_range_get_subrange_manifest ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilder_states ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_cobuilders_calls ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_rebalance_respects_cobuilder_part_moves ... ok test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load test.unit.common.ring.test_composite_builder.TestCooperativeRingBuilder.test_save_then_load ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_default_constraints ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv2 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 test.unit.common.middleware.s3api.test_s3request.TestRequest.test_request_timestamp_sigv4 ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_Authorization_not_exist_in_swreq ... ok test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log test.unit.common.middleware.s3api.test_s3request.TestRequest.test_to_swift_req_subrequest_proxy_access_log ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo test.unit.common.middleware.s3api.test_s3response.TestResponse.test_from_swift_resp_slo ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_from_swift3_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_sysmeta_headers_ignore_other_sysmeta ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_s3api_user_meta_headers ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_get_policy_stats ... ok test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_list_containers_iter ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_with_disable_encryption_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_effective_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints test.unit.common.test_constraints.TestConstraintsConfig.test_override_constraints ... ok test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset test.unit.common.test_constraints.TestConstraintsConfig.test_reload_reset ... ok test.unit.common.test_db.TestExampleBroker.test_commit_pending test.unit.common.test_db.TestExampleBroker.test_commit_pending ... ok test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta test.unit.common.middleware.s3api.test_s3response.TestResponse.test_response_swift3_sysmeta_does_not_overwrite_s3api_sysmeta ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_client_error_fetching_first_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_fetching_second_segment ... ok test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_with_crypto_read_without_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec test.unit.common.middleware.crypto.test_encryption.TestCryptoPipelineChanges.test_write_without_crypto_read_with_crypto_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata test.unit.account.test_utils.TestFakeAccountBroker.test_fake_broker_metadata ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_active_key ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_algorithm ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_HEAD ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_first_listing_request ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_listing_container_second_listing_request ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_error_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes test.unit.common.middleware.test_dlo.TestDloGetManifest.test_etag_comparison_ignores_quotes ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies test.unit.cli.test_relinker.TestRelinker.test_cleanup_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_conflicting_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_data_file_existing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_older_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_big_manifest ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_bad_key_length ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_filter_section ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_config_in_separate_file ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_multi_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_manifest_passthrough ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_conflicting_ts_is_linked_to_part_power_minus_2_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_non_object_passthrough ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_oversize_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_first_byte ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_last_byte ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted test.unit.cli.test_relinker.TestRelinker.test_cleanup_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiability_unknown ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_many_segments_satisfiable ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_on_segment_boundaries ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_cleanup_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_overlapping_end ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_range_unsatisfiable ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_suffix_range_many_segments ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_taking_too_long ... ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing test.unit.cli.test_relinker.TestRelinker.test_cleanup_diskfile_error_new_file_missing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second test.unit.cli.test_relinker.TestRelinker.test_cleanup_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_first_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error test.unit.cli.test_relinker.TestRelinker.test_cleanup_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_both_relinks_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_and_meta_file_one_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_existing_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_data_missing_ts_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_file_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_meta_missing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_missing_ts_missing_data_missing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_new_does_not_exist_and_relink_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_cleanup_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low test.unit.cli.test_relinker.TestRelinker.test_cleanup_not_yet_relinked_low ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_careful_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_lock_taken_between_get_hashes_and_rm ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_partition_lock_taken_during_get_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_replication_lock_taken ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust test.unit.cli.test_relinker.TestRelinker.test_cleanup_old_part_robust ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_older_object_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable test.unit.cli.test_relinker.TestRelinker.test_cleanup_reapable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok test.unit.cli.test_relinker.TestRelinker.test_cleanup_relinked_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails test.unit.cli.test_relinker.TestRelinker.test_cleanup_remove_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition test.unit.cli.test_relinker.TestRelinker.test_cleanup_same_object_different_inode_in_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_cleanup_second_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_cleanup_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking test.unit.cli.test_relinker.TestRelinker.test_cleanup_two_files_need_linking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_conf_file test.unit.cli.test_relinker.TestRelinker.test_conf_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering test.unit.cli.test_relinker.TestRelinker.test_devices_filter_filtering ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking test.unit.cli.test_relinker.TestRelinker.test_hook_pre_post_device_locking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_rehashing test.unit.cli.test_relinker.TestRelinker.test_rehashing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies test.unit.cli.test_relinker.TestRelinker.test_relink_all_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_both_in_older_part_powers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err test.unit.cli.test_relinker.TestRelinker.test_relink_conflicting_ts_is_linked_to_part_power_minus_2_err ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_data_files_no_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_existing_older_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_file test.unit.cli.test_relinker.TestRelinker.test_relink_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_newer_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_existing_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_data_ts_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid test.unit.cli.test_relinker.TestRelinker.test_relink_device_filter_invalid ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_older_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files test.unit.cli.test_relinker.TestRelinker.test_relink_existing_data_meta_ts_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second test.unit.cli.test_relinker.TestRelinker.test_relink_files_per_second ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_first_quartile_no_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode test.unit.cli.test_relinker.TestRelinker.test_relink_link_already_exists_but_different_inode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears test.unit.cli.test_relinker.TestRelinker.test_relink_link_target_disappears ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error test.unit.cli.test_relinker.TestRelinker.test_relink_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file test.unit.cli.test_relinker.TestRelinker.test_relink_meta_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy test.unit.cli.test_relinker.TestRelinker.test_relink_no_applicable_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted test.unit.cli.test_relinker.TestRelinker.test_relink_not_mounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter test.unit.cli.test_relinker.TestRelinker.test_relink_partition_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option test.unit.cli.test_relinker.TestRelinker.test_relink_policy_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash test.unit.cli.test_relinker.TestRelinker.test_relink_second_quartile_does_rehash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_data_meta_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files test.unit.cli.test_relinker.TestRelinker.test_relink_ts_existing_newer_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file test.unit.cli.test_relinker.TestRelinker.test_relink_ts_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges test.unit.cli.test_relinker.TestRelinker.test_relinker_drop_privileges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub test.unit.cli.test_relinker.TestRelinker.test_relinker_utils_get_hub ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_state_file test.unit.cli.test_relinker.TestRelinker.test_state_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_workers_children test.unit.cli.test_relinker.TestRelinker.test_workers_children ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_workers_parent test.unit.cli.test_relinker.TestRelinker.test_workers_parent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors test.unit.cli.test_relinker.TestRelinker.test_workers_parent_bubbles_up_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_etag_is_at_not_duplicated ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_GET_if_none_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_etag_is_at_not_duplicated ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_HEAD_if_none_match_with_existing_etag_is_at_header ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_constraints_checking ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_no_user_metadata ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_POST_req ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_app_exception ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_constraints_checking ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_encryption_override ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_error_in_key_callback ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_missing_key_callback ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_bad_client_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_good_client_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_multiseg_no_client_etag ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_no_user_metadata ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_nothing_read ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_req ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_response_inconsistent_etag_is_not_replaced ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_headers_and_other_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_bad_etag_in_other_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_footers_no_body ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_empty_etag_override_in_headers_no_body ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_of_empty_body_override_in_headers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_etag_override_in_headers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_with_other_footers_and_etag_of_empty_body ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_PUT_zero_size_object ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_config_true_value_on_disable_encryption ... ok test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val test.unit.common.middleware.crypto.test_encrypter.TestEncrypter.test_encrypt_header_val ... ok test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header test.unit.common.middleware.s3api.test_acl_utils.TestS3ApiAclUtils.test_canned_acl_header ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_GET_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_error ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_bucket_versioning_PUT_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_not_configured ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_GET_suspended ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_enabled ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_error ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_not_implemented ... ok test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended test.unit.common.middleware.s3api.test_versioning.TestS3ApiVersioning.test_object_versioning_PUT_suspended ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_content_type ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_call_and_content_type ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_get_objs ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_body ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_unicode ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_get ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_none ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_elsewhere_fails ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own test.unit.common.middleware.test_keystoneauth.TestAuthorizeReaderSystem.test_reader_put_to_own ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_tombstone ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_COPY_overwrite_version_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_new_version_different_account ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestCopy.test_copy_object_versioning_disabled ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing test.unit.common.test_internal_client.TestInternalClient.test_base_full_listing ... ok test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout test.unit.common.test_internal_client.TestInternalClient.test_base_request_timeout ... ok test.unit.common.test_internal_client.TestInternalClient.test_container_exists test.unit.common.test_internal_client.TestInternalClient.test_container_exists ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_account test.unit.common.test_internal_client.TestInternalClient.test_create_account ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container test.unit.common.test_internal_client.TestInternalClient.test_create_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_create_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_account test.unit.common.test_internal_client.TestInternalClient.test_delete_account ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_container test.unit.common.test_internal_client.TestInternalClient.test_delete_container ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing test.unit.common.test_internal_client.TestInternalClient.test_delete_container_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_delete_object test.unit.common.test_internal_client.TestInternalClient.test_delete_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded test.unit.common.test_internal_client.TestInternalClient.test_gatekeeper_not_loaded ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info test.unit.common.test_internal_client.TestInternalClient.test_get_account_info ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 test.unit.common.test_internal_client.TestInternalClient.test_get_account_info_404 ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadadata_with_acceptable_status ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_extra_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status test.unit.common.test_internal_client.TestInternalClient.test_get_metadata_invalid_status ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_object test.unit.common.test_internal_client.TestInternalClient.test_get_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_get_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_init test.unit.common.test_internal_client.TestInternalClient.test_init ... ok test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline test.unit.common.test_internal_client.TestInternalClient.test_init_allow_modify_pipeline ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_containers test.unit.common.test_internal_client.TestInternalClient.test_iter_containers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable test.unit.common.test_internal_client.TestInternalClient.test_iter_item_read_response_if_status_is_acceptable ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items test.unit.common.test_internal_client.TestInternalClient.test_iter_items ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix test.unit.common.test_internal_client.TestInternalClient.test_iter_items_with_markers_and_prefix ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_404 ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object test.unit.common.test_internal_client.TestInternalClient.test_iter_object_lines_compressed_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_iter_objects test.unit.common.test_internal_client.TestInternalClient.test_iter_objects ... ok test.unit.common.test_internal_client.TestInternalClient.test_load_from_config test.unit.common.test_internal_client.TestInternalClient.test_load_from_config ... ok test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf test.unit.common.test_internal_client.TestInternalClient.test_load_from_config_with_global_conf ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path test.unit.common.test_internal_client.TestInternalClient.test_make_path ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception test.unit.common.test_internal_client.TestInternalClient.test_make_path_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx test.unit.common.test_internal_client.TestInternalClient.test_make_request_acceptable_status_not_2xx ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try test.unit.common.test_internal_client.TestInternalClient.test_make_request_calls_fobj_seek_each_try ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app test.unit.common.test_internal_client.TestInternalClient.test_make_request_clears_txn_id_after_calling_app ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes test.unit.common.test_internal_client.TestInternalClient.test_make_request_codes ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header test.unit.common.test_internal_client.TestInternalClient.test_make_request_defaults_replication_network_header ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case test.unit.common.test_internal_client.TestInternalClient.test_make_request_error_case ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers test.unit.common.test_internal_client.TestInternalClient.test_make_request_method_path_headers ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception test.unit.common.test_internal_client.TestInternalClient.test_make_request_request_exception ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries test.unit.common.test_internal_client.TestInternalClient.test_make_request_retries ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_query_string ... ok test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent test.unit.common.test_internal_client.TestInternalClient.test_make_request_sets_user_agent ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_account_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_container_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata ... ok test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing test.unit.common.test_internal_client.TestInternalClient.test_set_object_metadata_plumbing ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object test.unit.common.test_internal_client.TestInternalClient.test_upload_object ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked test.unit.common.test_internal_client.TestInternalClient.test_upload_object_not_chunked ... ok test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing test.unit.common.test_internal_client.TestInternalClient.test_upload_object_plumbing ... ok test.unit.common.test_splice.TestTee.test_available test.unit.common.test_splice.TestTee.test_available ... ok test.unit.common.test_splice.TestTee.test_errno test.unit.common.test_splice.TestTee.test_errno ... ok test.unit.common.test_db.TestExampleBroker.test_delete_db test.unit.common.test_db.TestExampleBroker.test_delete_db ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_multipart ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 200, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Connecting with tenant: PROJECT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestV3.test_bad_reply_missing_parts ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_HEAD_with_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_bogus_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_passthrough ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_catcherrors_with_unexpected_error ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_error_in_iterator ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_long_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_multiple_content_lengths ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_no_content_length ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_short_response_body ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_length_limit ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_extra_quoted ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_fail ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_pass ... ok test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix test.unit.common.middleware.test_catch_errors.TestCatchErrors.test_trans_id_header_suffix ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_copy_from ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_bad_destination ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_copy_from ... ok test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination test.unit.common.middleware.test_copy.TestCopyConstraints.test_validate_destination ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_first ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_is_last_response_wins ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_next_response_keeps_current_registered_response ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_resets ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_forever ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins test.unit.common.middleware.test_helpers.TestFakeSwiftMultipleResponses.test_register_response_is_last_response_wins ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_bytes_count ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_container_listing_translation ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_missing ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_404_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delete_markers ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_delimiter ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_primary ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_empty_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_error_versions_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_invalid_delimiter ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_multipart ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_marker_missing_marker ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_list_versions_unversioned ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 test.unit.common.middleware.test_object_versioning.ObjectVersioningTestContainerOperations.test_listing_translation_utf8 ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_failures ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAccountAcls.test_account_acl_success ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acl_syntax_verification ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAccountAcls.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_admin_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAccountAcls.test_bad_acls_get_denied ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAccountAcls.test_readwrite_privileges ... ok test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAccountAcls.test_user_gets_best_acl ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_custom_url_config ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_multiple_reseller ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_no_prefix_reseller ... ok test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller test.unit.common.middleware.test_tempauth.TestGetUserGroups.test_single_reseller ... ok test.unit.common.ring.test_ring.TestRingData.test_attrs test.unit.common.ring.test_ring.TestRingData.test_attrs ... ok test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization test.unit.common.ring.test_ring.TestRingData.test_byteswapped_serialization ... ok test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data test.unit.common.ring.test_ring.TestRingData.test_can_load_pickled_ring_data ... ok test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization test.unit.common.ring.test_ring.TestRingData.test_deterministic_serialization ... ok test.unit.common.ring.test_ring.TestRingData.test_load_closes_file test.unit.common.ring.test_ring.TestRingData.test_load_closes_file ... ok test.unit.common.ring.test_ring.TestRingData.test_permissions test.unit.common.ring.test_ring.TestRingData.test_permissions ... ok test.unit.common.ring.test_ring.TestRingData.test_replica_count test.unit.common.ring.test_ring.TestRingData.test_replica_count ... ok test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization test.unit.common.ring.test_ring.TestRingData.test_roundtrip_serialization ... ok test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm test.unit.common.test_digest.TestDigestUtils.test_extract_digest_and_algorithm ... ok test.unit.common.test_db.TestExampleBroker.test_get_info test.unit.common.test_db.TestExampleBroker.test_get_info ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration ... ok test.unit.common.test_db.TestExampleBroker.test_get_max_row test.unit.common.test_db.TestExampleBroker.test_get_max_row ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_per_policy_cont_count_migration_with_single_policy ... ok test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata test.unit.common.test_db.TestExampleBroker.test_get_raw_metadata ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_policy_table_migration_in_get_policy_stats ... ok test.unit.common.test_db.TestExampleBroker.test_get_syncs test.unit.common.test_db.TestExampleBroker.test_get_syncs ... ok test.unit.common.test_db.TestExampleBroker.test_maybe_get test.unit.common.test_db.TestExampleBroker.test_maybe_get ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_stats_tracking ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_delete_with_objects ... ok test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Cipher must be AES_CTR_256 test DEBUG: No decryption is necessary because of override test DEBUG: No decryption is necessary because of override test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_decrypter.py", line 902, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: Error decrypting header X-Object-Sysmeta-Crypto-Etag: Missing crypto meta in value fgBd78GL6kACmfmuAxTz9B7eJCFTfjxotcr7trw/5Qo= test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Error decrypting object: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting object: Bad crypto meta: Missing 'cipher' test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting object: Key must be length 32 bytes test ERROR: Error decrypting object: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting object: Missing 'body_key' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting object: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: Cipher must be AES_CTR_256 test ERROR: Error decrypting header X-Object-Sysmeta-Container-Update-Override-Etag: Bad crypto meta: IV must be length 16 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: IV must be length 16 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Missing crypto meta in value LwwJqI3L+FBfmg== test ERROR: Error decrypting header X-Object-Transient-Sysmeta-Crypto-Meta-Test: Bad crypto meta: Missing 'iv' test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/service.py", line 35, in GET resp = req.get_response(self.app, query={'format': 'json'}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify memory limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? test WARNING: Skipping /tmp/tmpl7tgfj0i/devices3/device_is_file/data because [Errno 20] Not a directory: '/tmp/tmpl7tgfj0i/devices3/device_is_file/data' test WARNING: Skipping /tmp/tmpl7tgfj0i/devices4/device/data because test WARNING: Skipping device as it is not mounted test WARNING: Skipping /tmp/tmpyrl7q48x/asdf/data because [Errno 20] Not a directory: '/tmp/tmpyrl7q48x/asdf/data' test WARNING: Skipping /tmp/tmpyrl7q48x/asdf/data because [Errno 20] Not a directory: '/tmp/tmpyrl7q48x/asdf/data' test WARNING: Skipping drive as it is not mounted test WARNING: Skipping asdf as it is not mounted test WARNING: Skipping /tmp/tmpo5i9h6qh/asdf/data because [Errno 20] Not a directory: '/tmp/tmpo5i9h6qh/asdf/data' test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00049 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpv8tv3qyh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00683 seconds (146.36525/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test DEBUG: Successfully deleted db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test DEBUG: Not deleting db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (2/3 success) test DEBUG: Not deleting db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db (requires sharding, state unsharded) test DEBUG: Not deleting db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347264.64981.db (requires sharding, state sharding) test DEBUG: Not deleting db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347264.64981.db (2/3 success) test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate test.unit.container.test_replicator.TestReplicatorSync.test_cleanup_post_replicate ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_do_migrations ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_pre_storage_policy_replication ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_delete ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_simple_recreate ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_policy_table_cont_count_update_get_stats ... ok test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change test.unit.common.test_db.TestExampleBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.common.test_db.TestExampleBroker.test_put_timestamp test.unit.common.test_db.TestExampleBroker.test_put_timestamp ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_put_container ... ok test.unit.common.test_db.TestExampleBroker.test_status_changed_at test.unit.common.test_db.TestExampleBroker.test_status_changed_at ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_increment test.unit.common.test_error_limiter.TestErrorLimiter.test_increment ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_bad_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config test.unit.common.test_error_limiter.TestErrorLimiter.test_init_config ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited test.unit.common.test_error_limiter.TestErrorLimiter.test_is_limited ... ok test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key test.unit.common.test_error_limiter.TestErrorLimiter.test_node_key ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit test.unit.common.test_request_helpers.TestRequestHelpers.test_constrain_req_limit ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset test.unit.common.test_request_helpers.TestRequestHelpers.test_copy_header_subset ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port test.unit.common.test_request_helpers.TestRequestHelpers.test_get_ip_port ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_replication ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req test.unit.common.test_request_helpers.TestRequestHelpers.test_get_name_and_placement_object_req ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_get_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_names_in_system_accounts ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_reserved_names ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_invalid_split_reserved_name ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_put_container ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_object_transient_sysmeta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_sys_or_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network test.unit.common.test_request_helpers.TestRequestHelpers.test_is_use_replication_network ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta test.unit.common.test_request_helpers.TestRequestHelpers.test_is_user_meta ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items test.unit.common.test_request_helpers.TestRequestHelpers.test_remove_items ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name test.unit.common.test_request_helpers.TestRequestHelpers.test_split_reserved_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_object_transient_sysmeta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_sys_meta_prefix ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix test.unit.common.test_request_helpers.TestRequestHelpers.test_strip_user_meta_prefix ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reclaim ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_container_params ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_account ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_container ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_name ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_internal_object ... ok test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params test.unit.common.test_request_helpers.TestRequestHelpers.test_validate_params ... ok test.unit.common.test_swob.TestMatch.test_match test.unit.common.test_swob.TestMatch.test_match ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reclaim ... ok test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space test.unit.common.test_swob.TestMatch.test_match_lots_of_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space test.unit.common.test_swob.TestMatch.test_match_no_optional_white_space ... ok test.unit.common.test_swob.TestMatch.test_match_noquote test.unit.common.test_swob.TestMatch.test_match_noquote ... ok test.unit.common.test_swob.TestMatch.test_match_star test.unit.common.test_swob.TestMatch.test_match_star ... ok test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation test.unit.common.test_utils.TestStatsdLogging.test_bad_hostname_instantiation ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_defaults ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_non_defaults ... ok test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified test.unit.common.test_utils.TestStatsdLogging.test_get_logger_statsd_client_not_specified ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv4_instantiation_and_socket_creation ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 test.unit.common.test_utils.TestStatsdLogging.test_ipv4_or_ipv6_hostname_defaults_to_ipv4 ... ok test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation test.unit.common.test_utils.TestStatsdLogging.test_ipv6_instantiation_and_socket_creation ... ok test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_memcached_timing_stats ... ok test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet test.unit.common.test_utils.TestStatsdLogging.test_no_exception_when_cant_send_udp_packet ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates test.unit.common.test_utils.TestStatsdLogging.test_sample_rates ... ok test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor test.unit.common.test_utils.TestStatsdLogging.test_sample_rates_with_sample_rate_factor ... ok test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 test.unit.common.test_utils.TestStatsdLogging.test_sending_ipv6 ... ok test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation test.unit.common.test_utils.TestStatsdLogging.test_statsd_set_prefix_deprecation ... ok test.unit.common.test_utils.TestStatsdLogging.test_timing_stats test.unit.common.test_utils.TestStatsdLogging.test_timing_stats ... ok test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforePerPolicyContainerTrack.test_reverse_prefix_delim ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_add_device_already_exists ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_add_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_add_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region test.unit.cli.test_ringbuilder.TestCommands.test_add_device_old_missing_region ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_add_device_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices test.unit.cli.test_ringbuilder.TestCommands.test_add_duplicate_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring test.unit.cli.test_ringbuilder.TestCommands.test_create_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_create_ring_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default test.unit.cli.test_ringbuilder.TestCommands.test_default ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception test.unit.cli.test_ringbuilder.TestCommands.test_default_no_device_ring_without_exception ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output test.unit.cli.test_ringbuilder.TestCommands.test_default_output ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned test.unit.cli.test_ringbuilder.TestCommands.test_default_output_id_assigned ... ok test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim test.unit.account.test_backend.TestAccountBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name test.unit.cli.test_container_deleter.TestContainerDeleter.test_init_internal_client_log_name ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check test.unit.cli.test_ringbuilder.TestCommands.test_default_ringfile_check ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed test.unit.cli.test_ringbuilder.TestCommands.test_default_show_removed ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_native_utf8 ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 test.unit.cli.test_container_deleter.TestContainerDeleter.test_make_delete_jobs_unicode_utf8 ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_no_yield ... ok test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output test.unit.cli.test_ringbuilder.TestCommands.test_default_sorted_output ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_empty_with_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_one_update_no_yield ... ok test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield test.unit.cli.test_container_deleter.TestContainerDeleter.test_mark_for_deletion_two_updates_with_yield ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_check_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_decryption_context_non_zero_offset ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_encryption_context ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_iv ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_create_random_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_decrypt_range ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_large_chunks ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_enc_dec_small_chunks ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_get_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_unwrap_bad_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key test.unit.common.middleware.crypto.test_crypto_utils.TestCrypto.test_wrap_unwrap_key ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_append_then_extract_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_dump_then_load_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_extract_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestModuleMethods.test_load_crypto_meta ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_init ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_not_ratelimited ... ok test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate test.unit.cli.test_ringbuilder.TestCommands.test_dispersion_command_recalculate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_empty_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_find_parts test.unit.cli.test_ringbuilder.TestCommands.test_find_parts ... ok test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name test.unit.cli.test_ringbuilder.TestCommands.test_invalid_device_name ... ok test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output test.unit.cli.test_ringbuilder.TestCommands.test_ipv6_output ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_no_matching ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_ratelimited ... ok test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request test.unit.common.middleware.test_backend_ratelimit.TestBackendRatelimitMiddleware.test_unhandled_request ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_off_overrides_cluster_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_account_on_overrides_cluster_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_already_weak_but_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_cluster_wide ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_account_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_off_overrides_cluster_on ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_account_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_container_on_overrides_cluster_off ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_account_info ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_container_info ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_no_etag ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_object_request ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_non_swift_path ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_only_half_valid ... ok test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info test.unit.common.middleware.test_etag_quoter.TestEtagQuoter.test_swift_info ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_plus_other_reseller ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_blank_reseller ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_default ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_keystoneauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_get_nonmanifest_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_partially_satisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_starting_beyond_multipart_resp_length ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_multi_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_passthrough ... ok test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough test.unit.common.middleware.test_slo.TestNonSloPassthrough.test_non_slo_range_unsatisfiable_passthrough ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_new_target ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_target ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_copy_symlink_with_slo_query ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/54/23 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0013 - - 1717347263.600342035 1717347263.601608276 0 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/54/23 HEAD /junk HTTP/1.0 200 - Mozzarella%20Foxfire - - - - - - 0.0010 - - 1717347263.602887154 1717347263.603839159 1 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_first_put_fail ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_canned_acl ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_error ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_new_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_old_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_really_old_slo_manifest ... ok test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag test.unit.common.middleware.test_symlink.SymlinkCopyingTestCase.test_static_link_to_slo_manifest_slo_etag ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_blank_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_container_allow_versioned_writes_false ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_doubled_up_markers_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_is_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_latest_version_restores_marker_success ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_mixed_case_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketNoACL.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config test.unit.common.middleware.s3api.test_cfg.TestS3ApiCfg.test_config ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_no_versions_container_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_delete_single_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_denied_PUT_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_empty_versions_location ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_get_head ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_head_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_no_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_history_delete_marker_over_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_get_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_put_errors ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_new_version_sysmeta_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_both_headers ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_container_history_header ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_first_object_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_request_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_version_is_dlo_manifest_with_container_config_true ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_put_versioned_object_including_url_encoded_name_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_add_versions_precedence ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers test.unit.common.middleware.test_versioned_writes.VersionedWritesTestCase.test_remove_headers ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev test.unit.common.ring.test_builder.TestRingBuilder.test_add_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_add_rebalance_add_rebalance_delete_rebalance ... ok test ERROR: get_keys(): from callback: Testing: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_encrypter.py", line 913, in raise_exc raise Exception('Testing') Exception: Testing test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: User tenant_id has system reader authorizing test DEBUG: tenant mismatch: AUTH_tenant_id2 != tenant_id test DEBUG: allow user with role(s) swiftoperator,admin as account admin test-ic DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) fake-swift INFO: - - 02/Jun/2024/16/54/30 GET /cont/obj HTTP/1.0 200 - some_agent - - 19 - - - 0.0003 - - 1717347270.875485897 1717347270.875816822 - fake-swift INFO: - - 02/Jun/2024/16/54/30 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717347270.877341270 1717347270.877545357 - fake-swift INFO: - - 02/Jun/2024/16/54/30 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717347270.878454447 1717347270.878682375 - fake-swift INFO: - - 02/Jun/2024/16/54/30 GET /cont/obj HTTP/1.0 503 - some_agent - - 19 - - - 0.0002 - - 1717347270.879508018 1717347270.879717112 - test DEBUG: Before request test DEBUG: Inside of request (txn: foo) test DEBUG: After request fake-swift INFO: - - 02/Jun/2024/16/54/30 DELETE /container HTTP/1.0 409 - some_agent - - 19 - - - 0.0001 - - 1717347270.889113903 1717347270.889250517 - Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor Error while closing FD Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_splice.py", line 52, in safe_close os.close(fd) OSError: [Errno 9] Bad file descriptor test.unit.common.test_splice.TestTee.test_fileno test.unit.common.test_splice.TestTee.test_fileno ... ok test.unit.common.test_splice.TestTee.test_flags_list test.unit.common.test_splice.TestTee.test_flags_list ... ok test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe test.unit.common.test_splice.TestTee.test_tee_pipe_to_pipe ... ok test.unit.common.test_splice.TestTee.test_unavailable test.unit.common.test_splice.TestTee.test_unavailable ... ok test.unit.common.test_splice.TestTee.test_unavailable_in_libc test.unit.common.test_splice.TestTee.test_unavailable_in_libc ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_eq test.unit.common.test_utils.TestNamespaceBoundList.test_eq ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace test.unit.common.test_utils.TestNamespaceBoundList.test_get_namespace ... ok test.unit.common.test_utils.TestNamespaceBoundList.test_parse test.unit.common.test_utils.TestNamespaceBoundList.test_parse ... ok test.unit.common.test_utils.TestPairs.test_pairs test.unit.common.test_utils.TestPairs.test_pairs ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_closes test.unit.common.test_utils.TestReiterate.test_reiterate_closes ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first test.unit.common.test_utils.TestReiterate.test_reiterate_consumes_first ... ok test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple test.unit.common.test_utils.TestReiterate.test_reiterate_list_or_tuple ... ok test.unit.common.test_utils.TestShardName.test test.unit.common.test_utils.TestShardName.test ... ok test.unit.common.test_utils.TestShardName.test_bad_create test.unit.common.test_utils.TestShardName.test_bad_create ... ok test.unit.common.test_utils.TestShardName.test_bad_parse test.unit.common.test_utils.TestShardName.test_bad_parse ... ok test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names test.unit.common.test_utils.TestShardName.test_realistic_shard_range_names ... ok test.unit.common.test_utils.TestShardName.test_root_has_hyphens test.unit.common.test_utils.TestShardName.test_root_has_hyphens ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_decoding ... ok test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding test.unit.common.utils.test_timestamp.TestTimestampEncoding.test_encoding ... ok test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim test.unit.container.test_backend.TestContainerBroker.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_chexor test.unit.container.test_backend.TestContainerBroker.test_chexor ... ok test.unit.container.test_backend.TestContainerBroker.test_create_broker test.unit.container.test_backend.TestContainerBroker.test_create_broker ... ok test.unit.container.test_backend.TestContainerBroker.test_creation test.unit.container.test_backend.TestContainerBroker.test_creation ... ok test.unit.container.test_backend.TestContainerBroker.test_db_file test.unit.container.test_backend.TestContainerBroker.test_db_file ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBroker.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBroker.test_delete_object test.unit.container.test_backend.TestContainerBroker.test_delete_object ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBroker.test_double_check_trailing_delimiter_non_slash ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight test.unit.common.ring.test_builder.TestRingBuilder.test_add_replicas_then_rebalance_respects_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_adding_region_slowly_with_unbalanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region test.unit.common.ring.test_builder.TestRingBuilder.test_avoid_tier_change_new_region ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_cancel_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_create_add_dev_add_replica_rebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy test.unit.common.ring.test_builder.TestRingBuilder.test_deepcopy ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts test.unit.common.ring.test_builder.TestRingBuilder.test_dispersion_with_zero_weight_devices_with_parts ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload test.unit.common.ring.test_builder.TestRingBuilder.test_effective_overload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced test.unit.common.ring.test_builder.TestRingBuilder.test_ever_rebalanced ... ok test INFO: - - - [02/Jun/2024:16:54:23 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 1.3486 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:23 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.5313 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:23 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5553 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.2014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0352 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0335 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0025 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0143 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0322 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0264 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0002 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0436 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0143 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0218 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0182 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0292 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0031 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0167 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0276 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0072 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0231 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0227 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0208 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0241 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0233 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0028 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0172 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0260 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0040 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0005 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0191 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0128 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0150 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0163 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0228 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0236 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0252 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0194 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0272 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0280 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0181 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0171 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0188 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0049 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0046 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0060 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0286 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0037 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0189 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0351 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0191 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0211 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0165 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0048 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0282 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0114 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0139 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0159 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0106 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0166 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0029 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0233 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0029 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0275 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0150 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0251 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0159 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0162 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0046 "-" 3656197 - test.unit.account.test_server.TestAccountController.test_GET_log_requests_true test.unit.account.test_server.TestAccountController.test_GET_log_requests_true ... ok test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter test.unit.account.test_server.TestAccountController.test_GET_multichar_delimiter ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_json test.unit.account.test_server.TestAccountController.test_GET_not_found_json ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_plain test.unit.account.test_server.TestAccountController.test_GET_not_found_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_not_found_xml test.unit.account.test_server.TestAccountController.test_GET_not_found_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_over_limit test.unit.account.test_server.TestAccountController.test_GET_over_limit ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_json ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestAccountController.test_GET_prefix_delimiter_xml ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'X-Amz-Acl': 'public-read', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Grant AllUsers READ permission on the bucket /bucket test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_bucket_fails_with_neither_acl_header_nor_xml_PUT ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_finalize_increase_partition_power ... ok test INFO: - - - [02/Jun/2024:16:54:22 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.9837 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:23 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.9126 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:23 +0000] "DELETE /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5550 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.2016 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0353 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0307 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0051 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 204 - "-" "-" "-" 0.0149 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0286 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0226 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "DELETE /sda1/p/a" 400 26 "-" "-" "-" 0.0002 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0303 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0216 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0211 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0033 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0170 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0188 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0017 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 87 "-" "-" "-" 0.0221 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0162 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 406 - "-" "-" "-" 0.0003 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0208 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0157 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0240 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0049 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:24 +0000] "GET /sda1/p/a" 200 196 "-" "-" "-" 0.0319 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0279 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0061 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0207 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0272 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0172 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0" 201 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0.0" 201 - "-" "-" "-" 0.0129 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0.1" 201 - "-" "-" "-" 0.0141 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.0.2" 201 - "-" "-" "-" 0.0178 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1" 201 - "-" "-" "-" 0.0143 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1.0" 201 - "-" "-" "-" 0.0128 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1.1" 201 - "-" "-" "-" 0.0148 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.1.2" 201 - "-" "-" "-" 0.0111 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2" 201 - "-" "-" "-" 0.0195 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2.0" 201 - "-" "-" "-" 0.0135 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2.1" 201 - "-" "-" "-" 0.0134 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/.sub.2.2" 201 - "-" "-" "-" 0.0161 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0180 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 45 "-" "-" "-" 0.0041 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 27 "-" "-" "-" 0.0052 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0200 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0165 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0298 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0191 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0141 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 261 "-" "-" "-" 0.0185 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0034 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0241 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0028 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0149 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0165 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0295 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0153 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 9 "-" "-" "-" 0.0161 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0193 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c0" 201 - "-" "-" "-" 0.0037 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0207 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0182 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c3" 201 - "-" "-" "-" 0.0140 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c4" 201 - "-" "-" "-" 0.0100 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 452 "-" "-" "-" 0.0197 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0029 "-" 3656221 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_not_found_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix_delimiter_xml ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_fractional_replicas_rebalance ... ok test.unit.container.test_backend.TestContainerBroker.test_empty test.unit.container.test_backend.TestContainerBroker.test_empty ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_get_part_devices_partial_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring test.unit.common.ring.test_builder.TestRingBuilder.test_get_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id test.unit.common.ring.test_builder.TestRingBuilder.test_id ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from test.unit.common.ring.test_builder.TestRingBuilder.test_id_copy_from ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file test.unit.common.ring.test_builder.TestRingBuilder.test_id_legacy_builder_file ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors test.unit.common.ring.test_builder.TestRingBuilder.test_id_not_initialised_errors ... ok test INFO: Begin account audit "once" mode test ERROR: Audit Failed for /tmp/tmptl_17mwd/sda/accounts/0/0/0/test.db: The total container_count for the account a (3) does not match the sum of container_count across policies (2) test INFO: Account audit "once" mode completed: 0.00s test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID"}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_tenant_name ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_token_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_dict ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_name ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_missing_user_roles ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_not_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_bad_reply_valid_but_bad_json ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_fail_to_connect_to_keystone ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_no_s3_creds_defers_to_auth_middleware ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestBad.test_unauthorized_token ... ok test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info test.unit.common.middleware.test_acl.TestACL.test_acls_from_account_info ... ok test.unit.common.middleware.test_acl.TestACL.test_clean_acl test.unit.common.middleware.test_acl.TestACL.test_clean_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl test.unit.common.middleware.test_acl.TestACL.test_format_v1_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl test.unit.common.middleware.test_acl.TestACL.test_format_v2_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_parse_acl test.unit.common.middleware.test_acl.TestACL.test_parse_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl test.unit.common.middleware.test_acl.TestACL.test_parse_v2_acl ... ok test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed test.unit.common.middleware.test_acl.TestACL.test_referrer_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_auth_scheme ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_blank_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_confirmed_identity_is_not_authorized ... ok test INFO: Begin account audit "once" mode test INFO: Account audit "once" mode completed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, 1 containers deleted, 1 objects deleted, 1 containers remaining, 1 objects remaining, 1 containers possibly remaining, 1 objects possibly remaining, return codes: 1 2xxs, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper INFO: Beginning pass on account a test-reaper INFO: Completed pass on account a, elapsed: 0.00s test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda3: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 370, in reap_container headers, objects = direct_get_container( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 171, in fake_direct_get_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.10.10.1:6202/sda3 test-reaper ERROR: ERROR: invalid storage policy index: 2 test-reaper ERROR: Exception with objects for container c for account a: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 393, in reap_container policy_index = headers.get('X-Backend-Storage-Policy-Index', 0) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail test.unit.account.test_reaper.TestReaper.test_reap_container_partial_fail ... ok test.unit.account.test_reaper.TestReaper.test_reap_delay test.unit.account.test_reaper.TestReaper.test_reap_delay ... ok test.unit.account.test_reaper.TestReaper.test_reap_device test.unit.account.test_reaper.TestReaper.test_reap_device ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip test.unit.account.test_reaper.TestReaper.test_reap_device_with_not_my_ip ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices test.unit.account.test_reaper.TestReaper.test_reap_device_with_sharding_and_various_devices ... ok test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts test.unit.account.test_reaper.TestReaper.test_reap_device_with_ts ... ok test.unit.account.test_reaper.TestReaper.test_reap_object test.unit.account.test_reaper.TestReaper.test_reap_object ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_fail test.unit.account.test_reaper.TestReaper.test_reap_object_fail ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index test.unit.account.test_reaper.TestReaper.test_reap_object_non_exist_policy_index ... ok test.unit.account.test_reaper.TestReaper.test_reap_object_timeout test.unit.account.test_reaper.TestReaper.test_reap_object_timeout ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_bad_value ... ok test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set test.unit.account.test_reaper.TestReaper.test_reap_warn_after_conf_set ... ok test.unit.account.test_reaper.TestReaper.test_reset_stats test.unit.account.test_reaper.TestReaper.test_reset_stats ... ok test.unit.account.test_reaper.TestReaper.test_run_forever test.unit.account.test_reaper.TestReaper.test_run_forever ... ok test.unit.account.test_reaper.TestReaper.test_run_once test.unit.account.test_reaper.TestReaper.test_run_once ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_full ... ok test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space test.unit.cli.test_drive_full_checker.TestContainerDeleter.test_drive_with_space ... ok test.unit.cli.test_recon.TestRecon.test_async_check test.unit.cli.test_recon.TestRecon.test_async_check ... ok test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent test.unit.cli.test_recon.TestRecon.test_calculate_least_and_most_recent ... ok test.unit.cli.test_recon.TestRecon.test_drive_audit_check test.unit.cli.test_recon.TestRecon.test_drive_audit_check ... ok test.unit.cli.test_recon.TestRecon.test_gen_stats test.unit.cli.test_recon.TestRecon.test_gen_stats ... ok test.unit.cli.test_recon.TestRecon.test_get_error_ringnames test.unit.cli.test_recon.TestRecon.test_get_error_ringnames ... ok test.unit.cli.test_recon.TestRecon.test_get_hosts test.unit.cli.test_recon.TestRecon.test_get_hosts ... ok test.unit.cli.test_recon.TestRecon.test_get_ring_names test.unit.cli.test_recon.TestRecon.test_get_ring_names ... ok test.unit.cli.test_recon.TestRecon.test_get_ringmd5 test.unit.cli.test_recon.TestRecon.test_get_ringmd5 ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_alias ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_all_policies ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_invalid ... ok test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu test.unit.cli.test_recon.TestRecon.test_main_object_hosts_default_unu ... ok test.unit.cli.test_recon.TestRecon.test_ptime test.unit.cli.test_recon.TestRecon.test_ptime ... ok test.unit.cli.test_recon.TestRecon.test_quarantine_check test.unit.cli.test_recon.TestRecon.test_quarantine_check ... ok test.unit.cli.test_recon.TestRecon.test_umount_check test.unit.cli.test_recon.TestRecon.test_umount_check ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_cached_dispersion_value ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_increase_partition_power_failed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_init test.unit.common.ring.test_builder.TestRingBuilder.test_init ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance test.unit.common.ring.test_builder.TestRingBuilder.test_initial_balance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_insufficient_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_load test.unit.common.ring.test_builder.TestRingBuilder.test_load ... ok [2024-06-02 16:54:27] Checking auditor stats =============================================================================== test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile123 Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile3656248 Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile123 Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile3656248 Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile456 30 function calls in 0.002 seconds Ordered by: call count ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 6 0.000 0.000 0.000 0.000 :0(setprofile) 0 0.000 0.000 profile:0(profiler) Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile123 Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile3656248 Sun Jun 2 04:54:28 2024 /tmp/tmp4gp5xuo2/unittest.profile456 30 function calls in 0.002 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.002 0.000 profile:0(import os;os.getcwd();) 6 0.001 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 5 function calls in 0.000 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 :0(exec) 1 0.000 0.000 0.000 0.000 profile:0(import os;os.getcwd();) 1 0.000 0.000 0.000 0.000 :1() 1 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile3656248 Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile3656248 Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile123 Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile3656248 Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile456 30 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 6 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 6 0.000 0.000 0.001 0.000 :0(exec) 6 0.000 0.000 0.000 0.000 :1() 6 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 6 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile3656248 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile3656248 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render test.unit.common.middleware.test_xprofile.Test_html_viewer.test_render ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_closed_response ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_get_expect ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_http_connect ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_nonstr_header_values ... ok test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values test.unit.common.test_bufferedhttp.TestBufferedHTTP.test_unicode_values ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_db_exists ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_old_file_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_rename ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input test.unit.common.test_db_replicator.TestDBReplicator.test_complete_rsync_with_bad_input ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_creation test.unit.common.test_db_replicator.TestDBReplicator.test_creation ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db test.unit.common.test_db_replicator.TestDBReplicator.test_delete_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_no_arg_pop ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_complete_rsync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_operation_unexpected ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_dispatch_unexpected_operation_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup test.unit.common.test_db_replicator.TestDBReplicator.test_empty_suffix_and_hash_dirs_get_cleanedup ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device test.unit.common.test_db_replicator.TestDBReplicator.test_extract_device ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete test.unit.common.test_db_replicator.TestDBReplicator.test_handoff_delete ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect test.unit.common.test_db_replicator.TestDBReplicator.test_http_connect ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync test.unit.common.test_db_replicator.TestDBReplicator.test_in_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items test.unit.common.test_db_replicator.TestDBReplicator.test_merge_items ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs test.unit.common.test_db_replicator.TestDBReplicator.test_merge_syncs ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica test.unit.common.ring.test_builder.TestRingBuilder.test_min_part_hours_zero_will_move_one_replica ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev test.unit.common.ring.test_builder.TestRingBuilder.test_more_devices_than_replicas_validation_when_removed_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_multiple_duplicate_device_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_dont_move_too_many_replicas ... ok test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"name": "PROJECT_NAME", "id": "PROJECT_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' test DEBUG: Error on keystone reply: 201 b'{"token": {"user": {"domain": {"name": "Default", "id": "default"}, "name": "S3_USER", "id": "USER_ID"}, "project": {"domain": {"name": "PROJECT_DOMAIN_NAME", "id": "PROJECT_DOMAIN_ID"}, "name": "PROJECT_NAME", "id": "PROJECT_ID"}}}' - rejecting request test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests test.unit.common.test_digest.TestDigestUtils.test_get_allowed_digests ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac test.unit.common.test_digest.TestDigestUtils.test_get_hmac ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest test.unit.common.test_digest.TestDigestUtils.test_get_hmac_digest ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range ... ok test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type test.unit.common.test_digest.TestDigestUtils.test_get_hmac_ip_range_non_binary_type ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_bad_request_line ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_message_no_logger ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_log_request ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy test.unit.common.test_http_protocol.TestSwiftHttpProtocol.test_swift_http_protocol_parse_request_no_proxy ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_absolute_target ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_app_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_bad_request_server_logging ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_chunked_with_content_length ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_leading_slashes ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_messy ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_quoted ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_request_lines ... ok test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple test.unit.common.test_http_protocol.TestSwiftHttpProtocolSomeMore.test_simple ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default test.unit.common.test_memcached.TestLoadMemcache.test_conf_default ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_default_read ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_option ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section test.unit.common.test_memcached.TestLoadMemcache.test_conf_extra_no_section ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_extra_conf_bad_max_conn ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_maxc_from_extra_conf ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf test.unit.common.test_memcached.TestLoadMemcache.test_conf_from_inline_and_sers_from_extra_conf ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_item_warning_threshold ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_bad_max_conn ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_expansion_more_devices ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_other_max_conn ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_ratelimiting ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls test.unit.common.test_memcached.TestLoadMemcache.test_conf_inline_tls ... ok test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read test.unit.common.test_memcached.TestLoadMemcache.test_conf_set_no_read ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains test.unit.common.test_swob.TestHeaderEnvironProxy.test_contains ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_del test.unit.common.test_swob.TestHeaderEnvironProxy.test_del ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_ignored_keys ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys test.unit.common.test_swob.TestHeaderEnvironProxy.test_keys ... ok test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy test.unit.common.test_swob.TestHeaderEnvironProxy.test_proxy ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized test.unit.common.test_utils.TestGreenthreadSafeIterator.test_access_is_serialized ... ok test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works test.unit.common.test_utils.TestGreenthreadSafeIterator.test_setup_works ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_full ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_overfull ... ok test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_old_style_shard_container ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_0_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_part_moves_with_positive_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial test.unit.common.ring.test_builder.TestRingBuilder.test_multitier_partial ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours test.unit.common.ring.test_builder.TestRingBuilder.test_negative_min_part_hours ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_overlarge_part_powers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_old_style_shard_container ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload test.unit.common.ring.test_builder.TestRingBuilder.test_overload ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_initially ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance test.unit.common.ring.test_builder.TestRingBuilder.test_overload_keeps_balanceable_things_balanced_on_rebalance ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers test.unit.common.ring.test_builder.TestRingBuilder.test_oversmall_part_powers ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem test.unit.common.ring.test_builder.TestRingBuilder.test_part_swapping_problem ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed test.unit.common.ring.test_builder.TestRingBuilder.test_prepare_increase_partition_power_failed ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_part_on_deleted_other_part_on_drained ... ok proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpgslv8d04/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpgslv8d04/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpgslv8d04/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpgslv8d04/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:23 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 1.1042 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:23 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.7038 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.5950 "-" 3656192 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0272 "-" 3656192 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txed2879c51d93432192c11-00665ca3c0" "proxy-server 3656192" 0.0020 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:34141/sda1/1/a/c" "txed2879c51d93432192c11-00665ca3c0" "container-server 3656192" 0.0021 "-" 3656192 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:45249/sdb1/1/a/c" "txed2879c51d93432192c11-00665ca3c0" "container-server 3656192" 0.0023 "-" 3656192 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txed2879c51d93432192c11-00665ca3c0" "proxy-server 3656192" 0.0730 "-" 3656192 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txed2879c51d93432192c11-00665ca3c0" "proxy-server 3656192" 0.0437 "-" 3656192 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/24 PUT /v1/a/c HTTP/1.0 201 - - t - - - txed2879c51d93432192c11-00665ca3c0 - 0.0880 - - 1717347264.568399191 1717347264.656402349 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx8e4566f5cbd44688aca8a-00665ca3c0" "proxy-server 3656192" 0.0023 "-" 3656192 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:45249/sdb1/1/a1/c1" "tx8e4566f5cbd44688aca8a-00665ca3c0" "container-server 3656192" 0.0027 "-" 3656192 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:34141/sda1/1/a1/c1" "tx8e4566f5cbd44688aca8a-00665ca3c0" "container-server 3656192" 0.0071 "-" 3656192 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx8e4566f5cbd44688aca8a-00665ca3c0" "proxy-server 3656192" 0.0529 "-" 3656192 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx8e4566f5cbd44688aca8a-00665ca3c0" "proxy-server 3656192" 0.0861 "-" 3656192 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/24 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx8e4566f5cbd44688aca8a-00665ca3c0 - 0.1012 - - 1717347264.659126282 1717347264.760277987 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx50be1c5163434ba085c4d-00665ca3c0" "proxy-server 3656192" 0.0151 "-" 3656192 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:45249/sdb1/2/a/c1" "tx50be1c5163434ba085c4d-00665ca3c0" "container-server 3656192" 0.0137 "-" 3656192 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:34141/sda1/2/a/c1" "tx50be1c5163434ba085c4d-00665ca3c0" "container-server 3656192" 0.0019 "-" 3656192 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx50be1c5163434ba085c4d-00665ca3c0" "proxy-server 3656192" 0.0452 "-" 3656192 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx50be1c5163434ba085c4d-00665ca3c0" "proxy-server 3656192" 0.0666 "-" 3656192 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/24 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx50be1c5163434ba085c4d-00665ca3c0 - 0.0953 - - 1717347264.763113499 1717347264.858419895 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx66c78ead104a4ca89762b-00665ca3c0" "proxy-server 3656192" 0.0186 "-" 3656192 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:34141/sda1/1/a/c2" "tx66c78ead104a4ca89762b-00665ca3c0" "container-server 3656192" 0.0167 "-" 3656192 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:45249/sdb1/1/a/c2" "tx66c78ead104a4ca89762b-00665ca3c0" "container-server 3656192" 0.0024 "-" 3656192 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx66c78ead104a4ca89762b-00665ca3c0" "proxy-server 3656192" 0.0731 "-" 3656192 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:24 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx66c78ead104a4ca89762b-00665ca3c0" "proxy-server 3656192" 0.0490 "-" 3656192 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/24 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx66c78ead104a4ca89762b-00665ca3c0 - 0.1054 - - 1717347264.861158133 1717347264.966510534 2 Something useful Also useful Boring, but not secret test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_logger_manipulations ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_missing_key_id ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_filter_section ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_multikey_config_in_separate_file ... ok test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir test.unit.common.middleware.crypto.test_kmip_keymaster.TestKmipKeymaster.test_proxy_server_conf_dir ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_with_write_permission ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_bucket_list ... ok test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_subresource ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_post_upgrade ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed test.unit.common.ring.test_builder.TestRingBuilder.test_rebalance_with_seed ... ok test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container test.unit.container.test_backend.TestContainerBroker.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBroker.test_enable_sharding test.unit.container.test_backend.TestContainerBroker.test_enable_sharding ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device test.unit.common.ring.test_builder.TestRingBuilder.test_reduce_replicas_after_remove_device ... ok test.unit.container.test_backend.TestContainerBroker.test_exception test.unit.container.test_backend.TestContainerBroker.test_exception ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_balanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring test.unit.common.ring.test_builder.TestRingBuilder.test_region_fullness_with_unbalanceable_ring ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot test.unit.common.ring.test_builder.TestRingBuilder.test_remove_a_lot ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev test.unit.common.ring.test_builder.TestRingBuilder.test_remove_dev ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight test.unit.common.ring.test_builder.TestRingBuilder.test_remove_last_partition_from_zero_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted test.unit.common.ring.test_builder.TestRingBuilder.test_remove_zero_weighted ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance test.unit.common.ring.test_builder.TestRingBuilder.test_rerebalance ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: multipart: No SLO middleware in pipeline test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - deferring rejection downstream test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, deferring rejection based on error: 400 Bad Request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, deferring rejection based on error: 401 Unauthorized {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'AUTH_a.example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007251630>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/dontchange', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.aexample.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/ä½\xa0好', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/FOO_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007251630>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_add_prefix_already_there ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_bad_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_empty_storage_domain ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_no_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_configured_with_prefixes ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_extra_subdomains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_passthrough ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_redirect ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_multiple_storage_domains ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format test.unit.common.middleware.test_domain_remap.TestDomainRemap.test_storage_domains_conf_format ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_symlink ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_POST_unversioned_obj ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_overwrite_null_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_PUT_with_recent_versioned_marker_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_delete_object_with_versioning_disabled ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_with_id ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_disable_versioning ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_get_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_head_container ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDisabled.test_put_object_versioning_disabled ... ok test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_ratelimit.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRanges.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_modified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id test.unit.common.ring.test_builder.TestRingBuilder.test_reuse_of_dev_holes_without_id ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetNewManifest.test_range_resume_download ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments test.unit.common.middleware.test_slo.TestSloDataSegments.test_bunches_of_data_segments ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich test.unit.common.middleware.test_slo.TestSloDataSegments.test_data_segment_sandwich ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_leading_data_segment ... ok test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment test.unit.common.middleware.test_slo.TestSloDataSegments.test_trailing_data_segment ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_blank_prefix ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_default_prefix ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_multiple_prefixes ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_errors ... ok test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix test.unit.common.middleware.test_tempauth.TestDefinitiveAuth.test_noreseller_prefix ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_call ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_combine_body_qs ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_dump_checkpoint ... ok test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile test.unit.common.middleware.test_xprofile.TestProfileMiddleware.test_renew_profile ... ok test.unit.common.ring.test_ring.TestRing.test_creation test.unit.common.ring.test_ring.TestRing.test_creation ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes test.unit.common.ring.test_ring.TestRing.test_get_more_nodes ... ok test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region test.unit.common.ring.test_ring.TestRing.test_get_more_nodes_with_zero_weight_region ... ok test.unit.common.ring.test_ring.TestRing.test_get_nodes test.unit.common.ring.test_ring.TestRing.test_get_nodes ... ok test.unit.common.ring.test_ring.TestRing.test_get_part test.unit.common.ring.test_ring.TestRing.test_get_part ... ok test.unit.common.ring.test_ring.TestRing.test_get_part_nodes test.unit.common.ring.test_ring.TestRing.test_get_part_nodes ... ok test.unit.common.ring.test_ring.TestRing.test_has_changed test.unit.common.ring.test_ring.TestRing.test_has_changed ... ok test.unit.common.ring.test_ring.TestRing.test_reload test.unit.common.ring.test_ring.TestRing.test_reload ... ok test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring test.unit.common.ring.test_ring.TestRing.test_reload_old_style_pickled_ring ... ok test.unit.common.ring.test_ring.TestRing.test_reload_without_replication test.unit.common.ring.test_ring.TestRing.test_reload_without_replication ... ok test.unit.common.ring.test_ring.TestRing.test_replica_count test.unit.common.ring.test_ring.TestRing.test_replica_count ... ok test.unit.common.test_constraints.TestConstraints.test_check_account_format test.unit.common.test_constraints.TestConstraints.test_check_account_format ... ok 2 RLock(s) were not greened, to fix this error make sure you run eventlet.monkey_patch() before importing any other modules. test.unit.common.test_constraints.TestConstraints.test_check_container_format test.unit.common.test_constraints.TestConstraints.test_check_container_formatLoaded db broker for a/c ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_delete_headers ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_removes_delete_after ... ok test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at test.unit.common.test_constraints.TestConstraints.test_check_delete_headers_sets_delete_at ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path test.unit.common.test_constraints.TestConstraints.test_check_drive_invalid_path ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir test.unit.common.test_constraints.TestConstraints.test_check_drive_isdir ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error test.unit.common.ring.test_builder.TestRingBuilder.test_round_off_error ... ok test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount test.unit.common.test_constraints.TestConstraints.test_check_drive_ismount ... ok test.unit.common.test_constraints.TestConstraints.test_check_float test.unit.common.test_constraints.TestConstraints.test_check_float ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_count test.unit.common.test_constraints.TestConstraints.test_check_metadata_count ... ok Loaded db broker for a/c test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty test.unit.common.test_constraints.TestConstraints.test_check_metadata_emptyLoaded db broker for a/c ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name test.unit.common.test_constraints.TestConstraints.test_check_metadata_empty_name ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_good test.unit.common.test_constraints.TestConstraints.test_check_metadata_good ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 test.unit.common.test_constraints.TestConstraints.test_check_metadata_non_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_size test.unit.common.test_constraints.TestConstraints.test_check_metadata_size ... ok test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length test.unit.common.test_constraints.TestConstraints.test_check_metadata_value_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers test.unit.common.test_constraints.TestConstraints.test_check_object_creation_bad_delete_headers ... ok test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBroker.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type test.unit.common.test_constraints.TestConstraints.test_check_object_creation_content_type ... ok test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length test.unit.common.test_constraints.TestConstraints.test_check_object_creation_name_length ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8 test.unit.common.test_constraints.TestConstraints.test_check_utf8 ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal test.unit.common.test_constraints.TestConstraints.test_check_utf8_internal ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates test.unit.common.test_constraints.TestConstraints.test_check_utf8_lone_surrogates ... ok test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical test.unit.common.test_constraints.TestConstraints.test_check_utf8_non_canonical ... ok test.unit.common.test_constraints.TestConstraints.test_valid_api_version test.unit.common.test_constraints.TestConstraints.test_valid_api_version ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save test.unit.common.ring.test_builder.TestRingBuilder.test_save ... ok test.unit.common.test_constraints.TestConstraints.test_valid_timestamp test.unit.common.test_constraints.TestConstraints.test_valid_timestamp ... ok test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta test.unit.common.test_constraints.TestConstraints.test_validate_bad_meta ... ok test.unit.common.test_constraints.TestConstraints.test_validate_constraints test.unit.common.test_constraints.TestConstraints.test_validate_constraints ... ok test.unit.common.test_db_auditor.TestAuditor.test_database_auditor test.unit.common.test_db_auditor.TestAuditor.test_database_auditor ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_save_load test.unit.common.ring.test_builder.TestRingBuilder.test_save_load ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs test.unit.common.ring.test_builder.TestRingBuilder.test_search_devs ... ok test.unit.container.test_backend.TestContainerBroker.test_get_brokers test.unit.container.test_backend.TestContainerBroker.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBroker.test_get_db_state test.unit.container.test_backend.TestContainerBroker.test_get_db_state ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port test.unit.common.ring.test_builder.TestRingBuilder.test_server_per_port ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight test.unit.common.ring.test_builder.TestRingBuilder.test_set_dev_weight ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_decrease ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '450'} test DEBUG: Grant test:tester FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '456'} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the bucket /bucket test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_with_write_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_bucket_acl_PUT_without_permission ... ok profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) Sun Jun 2 04:54:28 2024 /tmp/tmpi893zaaq/unittest.profile3656248 15 function calls in 0.001 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 3 0.001 0.000 0.001 0.000 profile:0(import os;os.getcwd();) 3 0.000 0.000 0.000 0.000 :0(exec) 3 0.000 0.000 0.000 0.000 :1() 3 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 3 0.000 0.000 0.000 0.000 :0(setprofile) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2 new rows) test-replicator ERROR: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (negative max_row_delta: -1) test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Failed to delete db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py ERROR reading HTTP response from {'replication_ip': '127.0.0.1', 'replication_port': 80, 'device': 'sdb1'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 170, in replicate self.request('REPLICATE', self.path, body, File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 353, in other_req raise Exception('blah') Exception: blah test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection test.unit.common.test_db_replicator.TestDBReplicator.test_repl_connection ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_account_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_container_out_of_place_no_node ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_deleted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_because_not_shouldbehere ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_delete_delegated_to_cleanup_post_replicate ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_different_region ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_quarantine ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes test.unit.common.test_db_replicator.TestDBReplicator.test_replicate_object_with_exception_run_out_of_nodes ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table test.unit.common.test_db_replicator.TestDBReplicator.test_replicator_sync_with_broker_replication_missing_table ... ok Loaded db broker for a/c test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs test.unit.common.test_db_replicator.TestDBReplicator.test_roundrobin_datadirs ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_change_after_sync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_call ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_db_rsync_file_failure ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_different_region_and_rsync_compress ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_file_popen_args_whole_file_false ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_db_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_old_does_not_exist ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects test.unit.common.test_db_replicator.TestDBReplicator.test_rsync_then_merge_with_objects ... ok Loaded db broker for a/c Loaded db broker for a/c Loaded db broker for a/c test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_ips ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_no_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_mounted ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_node_is_not_mounted ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase test.unit.common.ring.test_builder.TestRingBuilder.test_set_replicas_increase ... ok Loaded db broker for a/c Loaded db broker for a/c test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather test.unit.common.ring.test_builder.TestRingBuilder.test_shuffled_gather ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info test.unit.container.test_backend.TestContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBroker.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBroker.test_get_info_no_stale_reads ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_fullcontrol_permission ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_server_converge_on_balance ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Expecting an element , got nothing, line 1 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'yes', 'X-Object-Sysmeta-S3Api-Content-Type': 'cat/picture', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: nojunk None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Fri, 01 Apr 2014 12:00:00 GMT', 'X-Amz-Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000'} test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000'} test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000'} test DEBUG: checking permission: bucket None POST {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000'} test DEBUG: bucket None POST fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_01 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 403 (

Forbidden

Access was denied to this reso...) while retrieving /v1/AUTH_test/c/seg_02 fake-swift ERROR: Got status 404 listing container /AUTH_test/c fake-swift ERROR: Too many bytes for /v1/AUTH_test/mancon/manifest; truncating in /v1/AUTH_test/c/seg_03 with 5 bytes left fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, max LO GET time of 86400s exceeded test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_undersize_segment_range ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_auth_overridden ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since test.unit.common.middleware.test_dlo.TestDloGetManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_does_not_match ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_match_matches ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_does_not_match ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches test.unit.common.middleware.test_dlo.TestDloGetManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_etag_fetching_second_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_mismatched_length_fetching_second_segment ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting test.unit.common.middleware.test_dlo.TestDloGetManifest.test_object_prefix_quoting ... ok test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment test.unit.common.middleware.test_dlo.TestDloGetManifest.test_server_error_fetching_first_segment ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadOldManifest.test_zero_byte_manifest ... ok test.unit.common.test_daemon.TestDaemon.test_create test.unit.common.test_daemon.TestDaemon.test_create ... ok test.unit.common.test_daemon.TestDaemon.test_stubs test.unit.common.test_daemon.TestDaemon.test_stubs ... ok test.unit.common.test_db.TestDictFactory.test_normal_case test.unit.common.test_db.TestDictFactory.test_normal_case ... ok test.unit.common.test_swob.TestRequest.test_401_invalid_path test.unit.common.test_swob.TestRequest.test_401_invalid_path ... ok test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path test.unit.common.test_swob.TestRequest.test_401_non_keystone_auth_path ... ok test.unit.common.test_swob.TestRequest.test_401_unauthorized test.unit.common.test_swob.TestRequest.test_401_unauthorized ... ok test.unit.common.test_swob.TestRequest.test_401_valid_account_path test.unit.common.test_swob.TestRequest.test_401_valid_account_path ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists test.unit.common.test_swob.TestRequest.test_401_www_authenticate_exists ... ok test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted test.unit.common.test_swob.TestRequest.test_401_www_authenticate_is_quoted ... ok test.unit.common.test_swob.TestRequest.test_accept_header test.unit.common.test_swob.TestRequest.test_accept_header ... ok test.unit.common.test_swob.TestRequest.test_allow_reserved_names test.unit.common.test_swob.TestRequest.test_allow_reserved_names ... ok test.unit.common.test_swob.TestRequest.test_as_referer test.unit.common.test_swob.TestRequest.test_as_referer ... ok test.unit.common.test_swob.TestRequest.test_bad_path_info_pop test.unit.common.test_swob.TestRequest.test_bad_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_bad_range test.unit.common.test_swob.TestRequest.test_bad_range ... ok test.unit.common.test_swob.TestRequest.test_blank test.unit.common.test_swob.TestRequest.test_blank ... ok test.unit.common.test_swob.TestRequest.test_blank_body_precedence test.unit.common.test_swob.TestRequest.test_blank_body_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_parsing test.unit.common.test_swob.TestRequest.test_blank_parsing ... ok test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence test.unit.common.test_swob.TestRequest.test_blank_path_info_precedence ... ok test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args test.unit.common.test_swob.TestRequest.test_blank_req_environ_property_args ... ok test.unit.common.test_swob.TestRequest.test_copy_get test.unit.common.test_swob.TestRequest.test_copy_get ... ok test.unit.common.test_swob.TestRequest.test_datetime_properties test.unit.common.test_swob.TestRequest.test_datetime_properties ... ok test.unit.common.test_swob.TestRequest.test_get_response test.unit.common.test_swob.TestRequest.test_get_response ... ok test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args test.unit.common.test_swob.TestRequest.test_invalid_req_environ_property_args ... ok test.unit.common.test_swob.TestRequest.test_message_length_just_content_length test.unit.common.test_swob.TestRequest.test_message_length_just_content_length ... ok test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding test.unit.common.test_swob.TestRequest.test_message_length_transfer_encoding ... ok test.unit.common.test_swob.TestRequest.test_not_401 test.unit.common.test_swob.TestRequest.test_not_401 ... ok test.unit.common.test_swob.TestRequest.test_params test.unit.common.test_swob.TestRequest.test_params ... ok test.unit.common.test_swob.TestRequest.test_path test.unit.common.test_swob.TestRequest.test_path ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop test.unit.common.test_swob.TestRequest.test_path_info_pop ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_last test.unit.common.test_swob.TestRequest.test_path_info_pop_last ... ok test.unit.common.test_swob.TestRequest.test_path_info_pop_none test.unit.common.test_swob.TestRequest.test_path_info_pop_none ... ok test.unit.common.test_swob.TestRequest.test_path_qs test.unit.common.test_swob.TestRequest.test_path_qs ... ok test.unit.common.test_swob.TestRequest.test_path_question_mark test.unit.common.test_swob.TestRequest.test_path_question_mark ... ok test.unit.common.test_swob.TestRequest.test_properties test.unit.common.test_swob.TestRequest.test_properties ... ok test.unit.common.test_swob.TestRequest.test_split_path test.unit.common.test_swob.TestRequest.test_split_path ... ok test.unit.common.test_swob.TestRequest.test_swift_entity_path test.unit.common.test_swob.TestRequest.test_swift_entity_path ... ok test.unit.common.test_swob.TestRequest.test_timestamp test.unit.common.test_swob.TestRequest.test_timestamp ... ok test.unit.common.test_swob.TestRequest.test_timestamp_invalid test.unit.common.test_swob.TestRequest.test_timestamp_invalid ... ok test.unit.common.test_swob.TestRequest.test_timestamp_missing test.unit.common.test_swob.TestRequest.test_timestamp_missing ... ok test.unit.common.test_swob.TestRequest.test_unicode_params test.unit.common.test_swob.TestRequest.test_unicode_params ... ok test.unit.common.test_swob.TestRequest.test_unicode_path test.unit.common.test_swob.TestRequest.test_unicode_path ... ok test.unit.common.test_swob.TestRequest.test_unicode_query test.unit.common.test_swob.TestRequest.test_unicode_query ... ok test.unit.common.test_swob.TestRequest.test_url test.unit.common.test_swob.TestRequest.test_url ... ok test.unit.common.test_swob.TestRequest.test_url2 test.unit.common.test_swob.TestRequest.test_url2 ... ok test.unit.common.test_swob.TestRequest.test_wsgify test.unit.common.test_swob.TestRequest.test_wsgify ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance test.unit.common.ring.test_builder.TestRingBuilder.test_undispersable_zone_converge_on_balance ... ok test.unit.common.test_swob.TestRequest.test_wsgify_method test.unit.common.test_swob.TestRequest.test_wsgify_method ... ok test.unit.common.test_swob.TestRequest.test_wsgify_raise test.unit.common.test_swob.TestRequest.test_wsgify_raise ... ok test.unit.common.test_utils.ResellerConfReader.test_defaults test.unit.common.test_utils.ResellerConfReader.test_defaults ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_same_resellers ... ok test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers test.unit.common.test_utils.ResellerConfReader.test_multiple_stray_commas_resellers ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBroker.test_get_info_sharding_states ... ok test.unit.common.test_utils.ResellerConfReader.test_same_as_default test.unit.common.test_utils.ResellerConfReader.test_same_as_default ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank test.unit.common.test_utils.ResellerConfReader.test_several_resellers_first_blank ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_blank_comma ... ok test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf test.unit.common.test_utils.ResellerConfReader.test_several_resellers_with_conf ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller ... ok test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf test.unit.common.test_utils.ResellerConfReader.test_single_blank_reseller_with_conf ... ok test.unit.common.test_utils.ResellerConfReader.test_stray_comma test.unit.common.test_utils.ResellerConfReader.test_stray_comma ... ok test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options test.unit.common.test_utils.ResellerConfReader.test_unprefixed_options ... ok test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set test.unit.common.utils.test_libc.TestModifyPriority.test__NR_ioprio_set ... ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBroker.test_get_info_stale_read_ok ... ok test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority test.unit.common.utils.test_libc.TestModifyPriority.test_modify_priority ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip test.unit.obj.test_expirer.TestObjectExpirer.test_build_task_obj_round_trip ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_container_timestamp_break ... ok Ring Builder file does not exist: /tmp/tmpgmxnywna/tmprr3fbomp Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.2:6200R127.0.0.2:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-127.0.0.1:6200R127.0.0.1:6200/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:6200R[3::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Ring Builder file does not exist: /tmp/tmp1qiee_ci/tmp2dfouot2 Partition power increase in progress. You need to finish the increase first before adding devices. Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sda9_"" with 3.14159265359 weight got id 4 Device 4 already uses 127.0.0.1:6200/sda9. The on-disk ring builder is unchanged. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. /tmp/tmpdp68g_o7/tmp_wrnig2k, build version 4, id 83ec4996bdc642f3901b50ef7dd0fd78 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmpdp68g_o7/tmp_wrnig2k.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_output_id_assigned Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 0.0 Device d4r1z2-127.0.0.5:6004R127.0.0.5:6004/sda5_"" with 100.0 weight got id 4 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d0r2z1-127.0.0.6:6005R127.0.0.6:6005/sdb6_"" with 100.0 weight got id 0 /build/reproducible-path/swift-2.33.0/test/unit/cli/test_default_sorted_output Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. Invalid device name The on-disk ring builder is unchanged. /build/reproducible-path/swift-2.33.0/test/unit/cli/test_ipv6_output Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 3 1 20 1 Partition Matches 1 1 4 1 6 1 15 1 20 1 24 1 25 1 28 1 38 1 40 1 41 1 42 1 48 1 54 1 57 1 61 1 0 1 2 1 5 1 7 1 10 1 16 1 18 1 21 1 23 1 29 1 30 1 33 1 34 1 51 1 60 1 62 1 8 1 9 1 12 1 19 1 26 1 27 1 36 1 37 1 39 1 44 1 45 1 47 1 52 1 53 1 56 1 58 1 Partition Matches 2 1 15 1 16 1 17 1 21 1 22 1 25 1 27 1 29 1 30 1 37 1 38 1 45 1 51 1 61 1 62 1 0 1 1 1 4 1 10 1 20 1 26 1 28 1 43 1 44 1 46 1 52 1 54 1 55 1 56 1 57 1 63 1 3 1 5 1 7 1 11 1 13 1 19 1 24 1 33 1 35 1 36 1 39 1 41 1 42 1 48 1 53 1 60 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 24 1 52 1 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Partition Matches 14 1 19 1 Specified builder file "/tmp/tmp_znwowuu/tmp5lqzhiis" is not rebalanced yet. Please rebalance first. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_list_parts_number_of_arguments ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_delete_at_time_of_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object test.unit.obj.test_expirer.TestObjectExpirer.test_delete_object ... ok test.unit.container.test_backend.TestContainerBroker.test_get_items_since test.unit.container.test_backend.TestContainerBroker.test_get_items_since ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload test.unit.common.ring.test_builder.TestRingBuilder.test_unoverload ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces test.unit.container.test_backend.TestContainerBroker.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_root_container_fill_gap ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate test.unit.common.ring.test_builder.TestRingBuilder.test_validate ... ok test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBroker.test_get_namespaces_state_filtering ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment test.unit.common.ring.test_builder.TestRingBuilder.test_validate_duplicate_part_assignment ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica test.unit.common.ring.test_builder.TestRingBuilder.test_validate_partial_replica ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects test.unit.container.test_backend.TestContainerBroker.test_get_objects ... ok test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go test.unit.common.ring.test_builder.TestRingBuilder.test_wrong_tier_with_no_where_to_go ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_empty_shard_container ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_fullcontrol_permission ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_enable_sharding ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_builders ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_exception ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_insufficient_builders ... ok test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row test.unit.container.test_backend.TestContainerBroker.test_get_objects_since_row ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_missing_builder_id ... ok test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00072 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpxn68ao0o/sda/accounts/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00872 seconds (114.63161/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00070 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00051 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00035 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sda/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', 'ca5f9270-4d5a-4cd2-8815-9d6337ed6482-sda', '1717347264.70751', '1717347264.69999', '0', '') test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp0sgipj8w/sda/accounts/0/cca/7e85f39cfbd92b74b0b21a19f4188cca/7e85f39cfbd92b74b0b21a19f4188cca.db REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', '3a8fd7fb-2887-4a61-994d-a0fecf7d45de-sda', '1717347264.75331', '1717347264.74963', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('3a8fd7fb-2887-4a61-994d-a0fecf7d45de-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (-1, '00000000000000000000000000000000', '3a8fd7fb-2887-4a61-994d-a0fecf7d45de-sda', '1717347264.75331', '1717347264.74963', '0', '') REPLICATE: /sdc/0/7e85f39cfbd92b74b0b21a19f4188cca, complete_rsync, ('3a8fd7fb-2887-4a61-994d-a0fecf7d45de-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10822 seconds (9.24062/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (1, '70151fa197f01cc8afd7ae2f6dd7a770', 'ee4705d8-3180-494d-8671-8b051dbb4037-sda', '1717347264.89773', '1717347264.89168', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, rsync_then_merge, ('ee4705d8-3180-494d-8671-8b051dbb4037-sda', '7e85f39cfbd92b74b0b21a19f4188cca.db') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, sync, (11, 'c3416ae418d9a4ff4c8082c09099af2a', 'b00ecccd-7311-4d47-9b28-643dc6fc1246-sda', '1717347265.02006', '1717347265.01379', '0', '') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_items, ([{'ROWID': 1, 'name': '/a/c_0', 'put_timestamp': '1717347265.05233', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c_1', 'put_timestamp': '1717347265.05336', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c_2', 'put_timestamp': '1717347265.05374', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c_3', 'put_timestamp': '1717347265.05425', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c_4', 'put_timestamp': '1717347265.05477', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c_5', 'put_timestamp': '1717347265.05518', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c_6', 'put_timestamp': '1717347265.05551', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c_7', 'put_timestamp': '1717347265.05585', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c_8', 'put_timestamp': '1717347265.05617', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c_9', 'put_timestamp': '1717347265.05647', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c_missing', 'put_timestamp': '1717347265.05684', 'delete_timestamp': '0', 'object_count': 0, 'bytes_used': 0, 'deleted': 0, 'storage_policy_index': 0}], 'b00ecccd-7311-4d47-9b28-643dc6fc1246-sda') REPLICATE: /sdb/0/7e85f39cfbd92b74b0b21a19f4188cca, merge_syncs, ([],) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: User tenant_id with role(s) rdr1,rdr2 has project reader authorizing test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: None in token vs test_id in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Inconsistent project domain id: unexpected_id in token vs None in account metadata. test WARNING: Error sending UDP message to ('some.host.com', 8125): [Errno 1] test errno 1 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp0evtbzzi/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp0evtbzzi/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp0evtbzzi/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp0evtbzzi/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0211 "-" 3656261 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0147 "-" 3656261 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0180 "-" 3656261 - test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file test.unit.proxy.test_server.TestProxyObjectPerformance.test_GET_debug_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_server.TestProxyServer.test_check_response_503 ... ok test.unit.proxy.test_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_server.TestProxyServer.test_denied_host_header ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'Content-Md5': '0psj556+oeNaquEdDeMxig==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd29b23e79ebea1e35aaae11d0de3318a', 'Content-Length': '226', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_empty_body ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_older_builder ... ok test DEBUG: Successfully deleted db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347264.64981.db test DEBUG: Successfully deleted db /tmp/tmpqw256rhv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, 'b79c84428ee232694bf709e6eb63a222', '590112ac-f315-48df-9be6-bec83ec01cdf-sda', '1717347264.84894', '1717347264.00000', '0', '', '1717347264.00000', 51, 0) test DEBUG: /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o0', 'created_at': '1717347265.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o1', 'created_at': '1717347266.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o2', 'created_at': '1717347267.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o3', 'created_at': '1717347268.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o4', 'created_at': '1717347269.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o5', 'created_at': '1717347270.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o6', 'created_at': '1717347271.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o7', 'created_at': '1717347272.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o8', 'created_at': '1717347273.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o9', 'created_at': '1717347274.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '590112ac-f315-48df-9be6-bec83ec01cdf-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 11, 'name': 'o10', 'created_at': '1717347275.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 12, 'name': 'o11', 'created_at': '1717347276.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 13, 'name': 'o12', 'created_at': '1717347277.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 14, 'name': 'o13', 'created_at': '1717347278.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 15, 'name': 'o14', 'created_at': '1717347279.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 16, 'name': 'o15', 'created_at': '1717347280.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 17, 'name': 'o16', 'created_at': '1717347281.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 18, 'name': 'o17', 'created_at': '1717347282.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 19, 'name': 'o18', 'created_at': '1717347283.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 20, 'name': 'o19', 'created_at': '1717347284.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '590112ac-f315-48df-9be6-bec83ec01cdf-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': 'o20', 'created_at': '1717347285.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 22, 'name': 'o21', 'created_at': '1717347286.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 23, 'name': 'o22', 'created_at': '1717347287.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 24, 'name': 'o23', 'created_at': '1717347288.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 25, 'name': 'o24', 'created_at': '1717347289.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 26, 'name': 'o25', 'created_at': '1717347290.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 27, 'name': 'o26', 'created_at': '1717347291.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 28, 'name': 'o27', 'created_at': '1717347292.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 29, 'name': 'o28', 'created_at': '1717347293.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 30, 'name': 'o29', 'created_at': '1717347294.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '590112ac-f315-48df-9be6-bec83ec01cdf-sda') test DEBUG: /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 30 (0.0556531s) test DEBUG: Synchronization for /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db has fallen more than 30 rows behind; moving on and will try again next pass. REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, 'b79c84428ee232694bf709e6eb63a222', '590112ac-f315-48df-9be6-bec83ec01cdf-sda', '1717347264.84894', '1717347264.00000', '0', '', '1717347264.00000', 51, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('590112ac-f315-48df-9be6-bec83ec01cdf-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.23543 seconds (4.24758/s) test INFO: Removed 0 dbs test INFO: 1 successes, 1 failures test INFO: diff:1 diff_capped:1 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, 'b79c84428ee232694bf709e6eb63a222', '590112ac-f315-48df-9be6-bec83ec01cdf-sda', '1717347264.84894', '1717347264.00000', '0', '', '1717347264.00000', 51, 0) test DEBUG: /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row 30 test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync test.unit.container.test_replicator.TestReplicatorSync.test_diff_capped_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part test.unit.container.test_replicator.TestReplicatorSync.test_find_local_handoff_for_part ... ok test.unit.container.test_replicator.TestReplicatorSync.test_local_ids test.unit.container.test_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_replica_count_changed ... ok test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0185 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0015 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0185 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0202 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0170 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0206 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0174 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0223 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0286 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0133 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0029 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0037 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0030 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0034 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0034 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0032 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0031 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0181 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0315 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0003 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0314 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0130 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0132 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0105 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0138 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0166 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0195 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0113 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0126 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0109 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0120 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0132 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0108 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0030 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0209 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0140 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0103 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0139 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0127 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0141 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0412 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0119 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0163 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0210 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0163 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0173 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0153 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0023 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0224 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0071 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0109 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0143 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0205 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0192 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0128 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0206 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0138 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0118 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0170 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0196 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0156 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0176 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0043 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0073 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0202 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656221 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_json ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_plain ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_containers_xml ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_account_name ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_empty_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_with_containers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range test.unit.container.test_backend.TestContainerBroker.test_get_own_shard_range ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked test.unit.common.test_db.TestGreenDBConnection.test_execute_when_locked ... ok test.unit.common.test_exceptions.TestExceptions.test_client_exception test.unit.common.test_exceptions.TestExceptions.test_client_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_exception test.unit.common.test_exceptions.TestExceptions.test_replication_exception ... ok test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout test.unit.common.test_exceptions.TestExceptions.test_replication_lock_timeout ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after test.unit.common.test_utils.TestRateLimitedIterator.test_limit_after ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting ... ok test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes test.unit.common.test_utils.TestRateLimitedIterator.test_rate_limiting_sometimes ... ok test.unit.common.test_utils.TestStringAlong.test_happy test.unit.common.test_utils.TestStringAlong.test_happy ... ok test.unit.common.test_utils.TestStringAlong.test_unhappy test.unit.common.test_utils.TestStringAlong.test_unhappy ... ok test.unit.container.test_backend.TestContainerBroker.test_get_policy_info test.unit.container.test_backend.TestContainerBroker.test_get_policy_info ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending test.unit.container.test_backend.TestCommonContainerBroker.test_commit_pending ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db test.unit.container.test_backend.TestCommonContainerBroker.test_delete_db ... ok test.unit.container.test_backend.TestContainerBroker.test_get_replication_info test.unit.container.test_backend.TestContainerBroker.test_get_replication_info ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_info test.unit.container.test_backend.TestCommonContainerBroker.test_get_info ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row test.unit.container.test_backend.TestCommonContainerBroker.test_get_max_row ... ok test.unit.container.test_backend.TestContainerBroker.test_get_report_info test.unit.container.test_backend.TestContainerBroker.test_get_report_info ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata test.unit.container.test_backend.TestCommonContainerBroker.test_get_raw_metadata ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBroker.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs test.unit.container.test_backend.TestCommonContainerBroker.test_get_syncs ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get test.unit.container.test_backend.TestCommonContainerBroker.test_maybe_get ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_delete_with_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_recreate_with_objects ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_delete ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBroker.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_simple_recreate ... ok test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage test.unit.container.test_backend.TestContainerBroker.test_get_shard_usage ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_delete_no_status_change ... ok test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies test.unit.container.test_backend.TestContainerBroker.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change test.unit.container.test_backend.TestCommonContainerBroker.test_merge_timestamps_update_put_no_status_change ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp test.unit.container.test_backend.TestCommonContainerBroker.test_put_timestamp ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': 'Je0by0I7C3IA9IX8X/ccjg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '25ed1bcb423b0b7200f485fc5ff71c8e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_handle_acl_header_with_s3acl ... ok test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET test.unit.common.middleware.s3api.test_acl.TestS3ApiAclCommonSetup.test_object_acl_GET ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_custom ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_default ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_get_only ... ok test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass test.unit.common.middleware.test_crossdomain.TestCrossDomain.test_crossdomain_pass ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_cache_middleware ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_filter_factory ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_config_with_options ... ok test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config test.unit.common.middleware.test_memcache.TestCacheMiddleware.test_real_memcache_config ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account test.unit.common.middleware.test_staticweb.TestStaticWeb.test_account ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_app_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set test.unit.common.middleware.test_staticweb.TestStaticWeb.test_conf_set ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container10unicodesubdirlisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkeraltdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerinvaliddirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkermatchdirtype ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container11subdirmarkerobjectindex ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container12unredirectedrequest ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container13empty ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container1_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_off ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2json_web_mode_explicitly_on ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container2onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3 ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3bindexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirx ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdiry ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subdirz ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3subsubdircontents ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container3unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4indexhtmlauthed ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4onetxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4tempurl ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4twotxt ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container4unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5indexhtml ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container5unknown ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6blisting ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container6subdir ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container7listing ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container8subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9listingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container9subdirlistingcss ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css test.unit.common.middleware.test_staticweb.TestStaticWeb.test_container_404_has_no_css ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware test.unit.common.middleware.test_staticweb.TestStaticWeb.test_no_auth_middleware ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root test.unit.common.middleware.test_staticweb.TestStaticWeb.test_root ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_not_override_auth ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible test.unit.common.middleware.test_staticweb.TestStaticWeb.test_subrequest_once_if_possible ... ok test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version test.unit.common.middleware.test_staticweb.TestStaticWeb.test_version ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller test.unit.common.middleware.test_tempauth.PrefixAccount.test_blank_reseller ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_default ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers test.unit.common.middleware.test_tempauth.PrefixAccount.test_multiple_resellers ... ok test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default test.unit.common.middleware.test_tempauth.PrefixAccount.test_same_as_default ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_delete_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_account_put_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_anon_badpath ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_fail ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_account_access ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_group_access ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referer_after_user_groups ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_authorize_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_deny_cross_reseller ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_detect_reseller_request ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_missing_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_memcache_error ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_get_user_group ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_object_name_containing_slash ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_overdate ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_old_token_with_old_data ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_options_call ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_override_default_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_regular_is_not_owner ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_request_method_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_reseller_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_statsd_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_default ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_storage_url_overridden_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_key ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_top_level_deny ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_old_token_from_memcached ... ok test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuthWithMultiplePrefixes.test_use_token_success ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict test.unit.common.middleware.test_xprofile.TestStats2.test_func_to_dict ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv test.unit.common.middleware.test_xprofile.TestStats2.test_to_csv ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_json test.unit.common.middleware.test_xprofile.TestStats2.test_to_json ... ok test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods test.unit.common.middleware.test_xprofile.TestStats2.test_to_ods ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_adding_second_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers test.unit.common.ring.test_builder.TestGetRequiredOverload.test_balanced_zones_unbalanced_servers ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_big_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_enormous_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights test.unit.common.ring.test_builder.TestGetRequiredOverload.test_equal_replica_and_devices_count_ignore_weights ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count test.unit.common.ring.test_builder.TestGetRequiredOverload.test_gradual_replica_count ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multi_zone_with_failed_device ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_each ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_replicas_in_zone_with_single_device ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones test.unit.common.ring.test_builder.TestGetRequiredOverload.test_multiple_small_zones ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed test.unit.common.ring.test_builder.TestGetRequiredOverload.test_none_needed ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_big_guy_does_not_spoil_his_buddy ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy test.unit.common.ring.test_builder.TestGetRequiredOverload.test_one_small_guy_does_not_spoil_his_buddy ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_extra_server_in_zone_with_multiple_replicas ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone test.unit.common.ring.test_builder.TestGetRequiredOverload.test_small_zone ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_big_two_small ... ok test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica test.unit.common.ring.test_builder.TestGetRequiredOverload.test_two_servers_with_more_than_one_replica ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_abs_paths_persisted ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_modified_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_ok ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_override_component_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_compose_with_builder_files ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_load_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_errors ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_rebalance_with_unrebalanced_builders ... ok test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors test.unit.common.ring.test_composite_builder.TestCompositeRingBuilder.test_save_errors ... ok test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting test.unit.common.test_db.TestDatabaseBroker.test_DB_PREALLOCATION_setting ... ok test.unit.common.test_db.TestDatabaseBroker.test_commit_puts test.unit.common.test_db.TestDatabaseBroker.test_commit_puts ... ok test.unit.common.test_db.TestDatabaseBroker.test_delete_db test.unit.common.test_db.TestDatabaseBroker.test_delete_db ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init test.unit.common.test_db.TestDatabaseBroker.test_disk_db_init ... ok test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate test.unit.common.test_db.TestDatabaseBroker.test_disk_preallocate ... ok test.unit.common.test_db.TestDatabaseBroker.test_initialize test.unit.common.test_db.TestDatabaseBroker.test_initialize ... ok test.unit.common.test_db.TestDatabaseBroker.test_lock test.unit.common.test_db.TestDatabaseBroker.test_lock ... ok test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init test.unit.common.test_db.TestDatabaseBroker.test_memory_db_init ... ok test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs test.unit.common.test_db.TestDatabaseBroker.test_merge_syncs ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata test.unit.common.test_db.TestDatabaseBroker.test_metadata ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_on_non_utf8 ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_count ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_raises_exception_over_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_count ... ok test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size test.unit.common.test_db.TestDatabaseBroker.test_metadata_with_max_overall_size ... ok test.unit.common.test_db.TestDatabaseBroker.test_newid test.unit.common.test_db.TestDatabaseBroker.test_newid ... ok test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at test.unit.container.test_backend.TestCommonContainerBroker.test_status_changed_at ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/PREFIX_uuid/c/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.prefix_uuid.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'example.com', 'SERVER_PORT': '80', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007251630>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'example.com:8080', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007331750>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage1.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_uuid/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage2.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/test', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'auth-uuid.storage3.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7fc007252cb0>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } Sun Jun 2 04:54:23 2024 /tmp/log/swift/profile/default.profile456 Sun Jun 2 04:54:23 2024 /tmp/log/swift/profile/default.profile123 Sun Jun 2 04:54:23 2024 /tmp/log/swift/profile/default.profile3656228 45 function calls in 0.003 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 9 0.001 0.000 0.003 0.000 profile:0(import os;os.getcwd();) 9 0.001 0.000 0.001 0.000 :0(exec) 9 0.000 0.000 0.000 0.000 :1() 9 0.000 0.000 0.000 0.000 :0(getcwd) 0 0.000 0.000 profile:0(profiler) 9 0.000 0.000 0.000 0.000 :0(setprofile) test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmp4adp929c/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmp4adp929c/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmp9v1x6cyu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test INFO: Since Wed Dec 31 12:00:00 1969: container audits: 0 passed audit, 1 failed audit test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass test.unit.common.test_db_auditor.TestAuditor.test_one_audit_pass ... ok test INFO: - - - [02/Jun/2024:16:54:25 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0230 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~TX~~A" 201 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~TX~~B" 201 - "-" "-" "-" 0.0184 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~~A" 201 - "-" "-" "-" 0.0162 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~~B" 201 - "-" "-" "-" 0.0153 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0173 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:25 +0000] "PUT /sda1/p/a/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0158 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/US~~UT~~A" 201 - "-" "-" "-" 0.0264 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/US~~UT~~~B" 201 - "-" "-" "-" 0.0154 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0278 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0035 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0038 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0033 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0029 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 119 "-" "-" "-" 0.0034 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0037 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 189 "-" "-" "-" 0.0035 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0033 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0151 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0309 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0020 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0008 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 412 22 "-" "-" "-" 0.0003 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0144 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0023 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0151 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0141 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0153 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0135 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0126 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0161 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0127 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0130 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0109 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0191 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0308 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0144 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 336 "-" "-" "-" 0.0037 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 276 "-" "-" "-" 0.0035 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0278 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0054 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0136 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0157 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0157 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0178 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0247 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0135 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0200 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0182 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0183 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0205 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0185 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 5 "-" "-" "-" 0.0216 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 39 "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 24 "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0154 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0" 201 - "-" "-" "-" 0.0068 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.0" 201 - "-" "-" "-" 0.0148 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.1" 201 - "-" "-" "-" 0.0119 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.0.2" 201 - "-" "-" "-" 0.0148 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1" 201 - "-" "-" "-" 0.0186 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.0" 201 - "-" "-" "-" 0.0106 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.1" 201 - "-" "-" "-" 0.0160 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.1.2" 201 - "-" "-" "-" 0.0243 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2" 201 - "-" "-" "-" 0.0160 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.0" 201 - "-" "-" "-" 0.0133 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.1" 201 - "-" "-" "-" 0.0270 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/sub.2.2" 201 - "-" "-" "-" 0.0114 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 91 "-" "-" "-" 0.0251 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 536 "-" "-" "-" 0.0044 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 467 "-" "-" "-" 0.0094 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0222 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0016 "-" 3656197 - test.unit.account.test_server.TestAccountController.test_GET_with_containers_json test.unit.account.test_server.TestAccountController.test_GET_with_containers_json ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain test.unit.account.test_server.TestAccountController.test_GET_with_containers_plain ... ok test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml test.unit.account.test_server.TestAccountController.test_GET_with_containers_xml ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_account_name ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name ... ok test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir test.unit.account.test_server.TestAccountController.test_GET_xml_escapes_container_name_as_subdir ... ok test.unit.account.test_server.TestAccountController.test_HEAD_empty_account test.unit.account.test_server.TestAccountController.test_HEAD_empty_account ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept test.unit.account.test_server.TestAccountController.test_HEAD_invalid_accept ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type test.unit.account.test_server.TestAccountController.test_HEAD_invalid_content_type ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format test.unit.account.test_server.TestAccountController.test_HEAD_invalid_format ... ok test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition test.unit.account.test_server.TestAccountController.test_HEAD_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_HEAD_not_found test.unit.account.test_server.TestAccountController.test_HEAD_not_found ... ok test.unit.account.test_server.TestAccountController.test_HEAD_with_containers test.unit.account.test_server.TestAccountController.test_HEAD_with_containers ... ok test.unit.account.test_server.TestAccountController.test_OPTIONS test.unit.account.test_server.TestAccountController.test_OPTIONS ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_metadata ... ok test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata test.unit.account.test_server.TestAccountController.test_POST_HEAD_sys_metadata ... ok test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found test.unit.account.test_server.TestAccountController.test_POST_after_DELETE_not_found ... ok test.unit.account.test_server.TestAccountController.test_POST_insufficient_space test.unit.account.test_server.TestAccountController.test_POST_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_POST_invalid_partition test.unit.account.test_server.TestAccountController.test_POST_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float test.unit.account.test_server.TestAccountController.test_POST_timestamp_not_float ... ok test.unit.account.test_server.TestAccountController.test_PUT test.unit.account.test_server.TestAccountController.test_PUT ... ok test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Error on keystone reply: 201 b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID"}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'' test DEBUG: Error on keystone reply: 201 b'' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{}' test DEBUG: Error on keystone reply: 201 b'{}' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'[]' test DEBUG: Error on keystone reply: 201 b'[]' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'null' test DEBUG: Error on keystone reply: 201 b'null' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'"foo"' test DEBUG: Error on keystone reply: 201 b'"foo"' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'1' test DEBUG: Error on keystone reply: 201 b'1' - rejecting request test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'true' test DEBUG: Error on keystone reply: 201 b'true' - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Received error, rejecting request with error: 400 Bad Request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone reply error: status=403 reason=None test DEBUG: Received error, rejecting request with error: 401 Unauthorized test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_denied_responses ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_detect_reseller_request ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_identified_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_invalid_request_non_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_override_default_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_not_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_added_underscore ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuth.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_bytes_quota_copy_from_no_src ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_PUT_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_cross_account_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_from ... ok test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb test.unit.common.middleware.test_quotas.ContainerQuotaCopyingTestCases.test_not_exceed_counts_quota_copy_verb ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit test.unit.common.middleware.test_ratelimit.TestRateLimit.test_account_ratelimit ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled test.unit.common.middleware.test_ratelimit.TestRateLimit.test_already_handled ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_invalid_path ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path test.unit.common.middleware.test_ratelimit.TestRateLimit.test_call_non_swift_api_path ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_maxrate ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples test.unit.common.middleware.test_ratelimit.TestRateLimit.test_get_ratelimitable_key_tuples ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict test.unit.common.middleware.test_ratelimit.TestRateLimit.test_memcached_container_info_dict ... ok test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: Found /path/to/file for /a%20c%20t/c%20o%20n when it should be on partition 0; will replicate out and remove. test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/0 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (0/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 1, 'weight': 10.0, 'zone': 1, 'ip': '1.1.1.1', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.1', 'replication_port': 6200, 'region': 1}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 675, in _replicate_object success = self._repl_to_node(node, broker, partition, info, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 481, in _repl_to_node return self._handle_sync_response(node, response, info, broker, http, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 486, in _handle_sync_response if response.status == HTTP_NOT_FOUND: # completely missing, rsync ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py", line 713, in status raise self._status Exception: ugh test-replicator ERROR: ERROR syncing /path/to/file with node {'id': 2, 'weight': 10.0, 'zone': 2, 'ip': '1.1.1.2', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.2', 'replication_port': 6200, 'region': 2}: Timeout (Nones) test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (1/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (2/3 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator ERROR: ERROR Remote drive not mounted {'id': 3, 'weight': 10.0, 'zone': 3, 'ip': '1.1.1.3', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.3', 'replication_port': 6200, 'region': 1} test-replicator ERROR: ERROR Remote drive not mounted {'id': 4, 'weight': 10.0, 'zone': 4, 'ip': '1.1.1.4', 'port': 6200, 'device': 'sdb', 'meta': '', 'replication_ip': '1.1.1.4', 'replication_port': 6200, 'region': 2} test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.5:6200/sdb, nothing to do test-replicator DEBUG: Not deleting db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py (3/5 success) test-replicator DEBUG: Replicating db /path/to/file test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.1:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.2:6200/sdb, nothing to do test-replicator DEBUG: /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py in sync with 1.1.1.3:6200/sdb, nothing to do test-replicator DEBUG: Successfully deleted db /build/reproducible-path/swift-2.33.0/test/unit/common/test_db_replicator.py test-replicator ERROR: Unable to decode remote metadata 'metadata' test-replicator ERROR: Quarantining DB test-replicator ERROR: ERROR Failed to get my own IPs? test-replicator ERROR: Can't find itself 127.0.0.1 with port 1000 in ring file, not replicating test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00034 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00093 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: Boom! test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00075 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 1 failures test-replicator ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator WARNING: Skipping: /srv/node/sdb is not mounted test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring test.unit.common.test_db_replicator.TestDBReplicator.test_run_once_with_local_device_in_ring ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_stats test.unit.common.test_db_replicator.TestDBReplicator.test_stats ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync test.unit.common.test_db_replicator.TestDBReplicator.test_usync ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_above_300 ... ok test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 test.unit.common.test_db_replicator.TestDBReplicator.test_usync_http_error_below_200 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestDirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestDirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestDirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_get_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestDirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestDirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_deleted ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestDirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestDirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container test.unit.common.test_direct_client.TestDirectClient.test_direct_post_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_chunked ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestDirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_args ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_header_content_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestDirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestDirectClient.test_gen_headers test.unit.common.test_direct_client.TestDirectClient.test_gen_headers ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry test.unit.common.test_direct_client.TestDirectClient.test_retry ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_client_exception ... ok test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestDirectClient.test_retry_http_exception ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_init test.unit.common.test_internal_client.TestCompressingfileReader.test_init ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_read test.unit.common.test_internal_client.TestCompressingfileReader.test_read ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek test.unit.common.test_internal_client.TestCompressingfileReader.test_seek ... ok test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception test.unit.common.test_internal_client.TestCompressingfileReader.test_seek_not_implemented_exception ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_404_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry test.unit.common.test_internal_client.TestSimpleClient.test_delete_object_with_409_no_retry ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get test.unit.common.test_internal_client.TestSimpleClient.test_get ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_all_failed ... ok test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param test.unit.common.test_internal_client.TestSimpleClient.test_get_with_retries_param ... ok test.unit.common.test_internal_client.TestSimpleClient.test_head test.unit.common.test_internal_client.TestSimpleClient.test_head ... ok test.unit.common.test_internal_client.TestSimpleClient.test_proxy test.unit.common.test_internal_client.TestSimpleClient.test_proxy ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_container_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_object_with_retries_with_HTTPError ... ok test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError test.unit.common.test_internal_client.TestSimpleClient.test_request_with_retries_with_HTTPError ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names test.unit.common.test_storage_policy.TestStoragePolicies.test_add_remove_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache test.unit.common.test_storage_policy.TestStoragePolicies.test_bind_ports_cache ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_defaults ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecate_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default test.unit.common.test_storage_policy.TestStoragePolicies.test_deprecated_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached test.unit.common.test_storage_policy.TestStoragePolicies.test_ec_fragment_size_cached ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_custom_diskfile ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config test.unit.common.test_storage_policy.TestStoragePolicies.test_get_diskfile_manager_invalid_policy_config ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_get_object_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_get_policy_string ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_abort_multipart_upload_acl_without_permission ... ok test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! ERROR: swift.cache could not be found in env! test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_no_memcache ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_blacklist ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_double_container_listing ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_max_rate_multiple_acc ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_memcache_format ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_old_white_black_list ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_set_incr ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta test.unit.common.middleware.test_ratelimit.TestRateLimit.test_ratelimit_whitelist_sysmeta ... ok test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache test.unit.common.middleware.test_ratelimit.TestRateLimit.test_restarting_memcache ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrors.test_download_takes_too_long ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_segment ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrors.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrors.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrors.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrors.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrors.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrors.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrors.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrors.test_sub_slo_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_download_takes_too_long ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_first_segment_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_segment ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_error_fetching_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_available ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_first_segment_not_exists ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_generator_closure_iter_app_resp ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_bogus_manifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_get_invalid_sysmeta_passthrough ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_head_does_not_validate_first_segment_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_invalid_json_submanifest ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_checksum ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_etag ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_length ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_mismatched_size ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_recursion_limit ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_slo_sysmeta_on_error ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion ... ok test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit test.unit.common.middleware.test_slo.TestSloErrorsOldManifests.test_sub_slo_recursion_limit ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_delete_latest_version_no_marker_success ... ok test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success test.unit.common.middleware.test_symlink.SymlinkVersioningTestCase.test_new_symlink_version_success ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_account_with_no_user ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_base64_encoding ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_key_with_no_value ... ok test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation test.unit.common.middleware.test_tempauth.TestParseUserCreation.test_parse_user_creation ... ok test.unit.common.test_swob.TestEtag.test_normalize_bytes test.unit.common.test_swob.TestEtag.test_normalize_bytes ... ok test.unit.common.test_swob.TestEtag.test_normalize_etag test.unit.common.test_swob.TestEtag.test_normalize_etag ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_closed_part_iterator ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_multiple_parts ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_no_parts ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part ... ok test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges test.unit.common.test_utils.TestDocumentItersToHTTPResponseBody.test_single_part_unexpected_ranges ... ok test.unit.common.test_utils.TestNamespace.test_bounds_serialization test.unit.common.test_utils.TestNamespace.test_bounds_serialization ... ok test.unit.common.test_utils.TestNamespace.test_comparisons test.unit.common.test_utils.TestNamespace.test_comparisons ... ok test.unit.common.test_utils.TestNamespace.test_contains test.unit.common.test_utils.TestNamespace.test_contains ... ok test.unit.common.test_utils.TestNamespace.test_end_marker test.unit.common.test_utils.TestNamespace.test_end_marker ... ok test.unit.common.test_utils.TestNamespace.test_entire_namespace test.unit.common.test_utils.TestNamespace.test_entire_namespace ... ok test.unit.common.test_utils.TestNamespace.test_expand test.unit.common.test_utils.TestNamespace.test_expand ... ok test.unit.common.test_utils.TestNamespace.test_includes test.unit.common.test_utils.TestNamespace.test_includes ... ok test.unit.common.test_utils.TestNamespace.test_lower_setter test.unit.common.test_utils.TestNamespace.test_lower_setter ... ok test.unit.common.test_utils.TestNamespace.test_name test.unit.common.test_utils.TestNamespace.test_name ... ok test.unit.common.test_utils.TestNamespace.test_name_unexpected_format test.unit.common.test_utils.TestNamespace.test_name_unexpected_format ... ok test.unit.common.test_utils.TestNamespace.test_total_ordering test.unit.common.test_utils.TestNamespace.test_total_ordering ... ok test.unit.common.test_utils.TestNamespace.test_unicode_name test.unit.common.test_utils.TestNamespace.test_unicode_name ... ok test.unit.common.test_utils.TestNamespace.test_upper_setter test.unit.common.test_utils.TestNamespace.test_upper_setter ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_default_sample_rate ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_metric_prefix ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate test.unit.common.test_utils.TestStatsdLoggingDelegation.test_delegate_methods_with_no_default_sample_rate ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_bad_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_f_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_fsync_no_fullsync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled test.unit.common.test_utils.TestStatsdLoggingDelegation.test_methods_are_no_ops_when_not_enabled ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_no_fdatasync ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals test.unit.common.test_utils.TestStatsdLoggingDelegation.test_thread_locals ... ok test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync test.unit.common.test_utils.TestStatsdLoggingDelegation.test_yes_fdatasync ... ok test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_catch_errors_gatekeeper_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start test.unit.common.test_wsgi.TestPipelineModification.test_gatekeeper_insertion_catch_errors_configured_not_at_start ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app test.unit.common.test_wsgi.TestPipelineModification.test_load_app_request_logging_app ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_proxy ... ok test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage test.unit.common.test_wsgi.TestPipelineModification.test_loadapp_storage ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_inserts_versioned_writes ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_ordering ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines test.unit.common.test_wsgi.TestPipelineModification.test_proxy_modify_wsgi_pipeline_recommended_pipelines ... ok test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline test.unit.common.test_wsgi.TestPipelineModification.test_proxy_unmodified_wsgi_pipeline ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil test.unit.common.utils.test_timestamp.TestTimestamp.test_ceil ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none test.unit.common.utils.test_timestamp.TestTimestamp.test_cmp_with_none ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_delta test.unit.common.utils.test_timestamp.TestTimestamp.test_delta ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_false test.unit.common.utils.test_timestamp.TestTimestamp.test_false ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_float test.unit.common.utils.test_timestamp.TestTimestamp.test_float ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal test.unit.common.utils.test_timestamp.TestTimestamp.test_force_internal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_from_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_greater_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable test.unit.common.utils.test_timestamp.TestTimestamp.test_hashable ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_int test.unit.common.utils.test_timestamp.TestTimestamp.test_int ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_internal_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_input ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion test.unit.common.utils.test_timestamp.TestTimestamp.test_invalid_string_conversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion test.unit.common.utils.test_timestamp.TestTimestamp.test_inversion ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat test.unit.common.utils.test_timestamp.TestTimestamp.test_isoformat ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_no_force_internal_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_normal_format_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal test.unit.common.utils.test_timestamp.TestTimestamp.test_not_equal ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit test.unit.common.utils.test_timestamp.TestTimestamp.test_offset_limit ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering test.unit.common.utils.test_timestamp.TestTimestamp.test_ordering ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons test.unit.common.utils.test_timestamp.TestTimestamp.test_out_of_range_comparisons ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_raw test.unit.common.utils.test_timestamp.TestTimestamp.test_raw ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_short_format_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_no_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset test.unit.common.utils.test_timestamp.TestTimestamp.test_smaller_with_offset ... ok test.unit.common.utils.test_timestamp.TestTimestamp.test_true test.unit.common.utils.test_timestamp.TestTimestamp.test_true ... ok test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing test.unit.container.test_backend.TestUpdateNewItemFromExisting.test_update_new_item_from_existing ... ok test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring test.unit.obj.test_replicator.TestObjectReplicator.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_failure_report_with_auditor_stats_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_diff_ports_normal ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_multi_disk_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir test.unit.obj.test_replicator.TestObjectReplicator.test_collect_jobs_with_override_parts_and_unexpected_part_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_objs_ssync_only_when_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_default_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_data_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ignores_os_no_entity_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_override_params_os_not_empty_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_single_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_cleanup_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_ssync_with_sync_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_fail_in_other_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures test.unit.obj.test_replicator.TestObjectReplicator.test_delete_partition_with_handoff_delete_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params test.unit.obj.test_replicator.TestObjectReplicator.test_delete_policy_override_params ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir test.unit.obj.test_replicator.TestObjectReplicator.test_do_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings test.unit.obj.test_replicator.TestObjectReplicator.test_handoff_replication_setting_warnings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_abort_if_handoffs_remaining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs test.unit.obj.test_replicator.TestObjectReplicator.test_handoffs_first_mode_will_process_all_jobs_after_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log test.unit.obj.test_replicator.TestObjectReplicator.test_limit_rsync_log ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_rsync_timeout_wedged ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase test.unit.obj.test_replicator.TestObjectReplicator.test_replicate_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files test.unit.obj.test_replicator.TestObjectReplicator.test_replicator_skips_rsync_temp_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_revert_partition_lock_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip test.unit.obj.test_replicator.TestObjectReplicator.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_compress_different_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_failure_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer test.unit.obj.test_replicator.TestObjectReplicator.test_rsync_success_logging_no_transfer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run test.unit.obj.test_replicator.TestObjectReplicator.test_run ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once test.unit.obj.test_replicator.TestObjectReplicator.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_no_local_device_in_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout test.unit.obj.test_replicator.TestObjectReplicator.test_run_once_recover_from_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog test.unit.obj.test_replicator.TestObjectReplicator.test_run_withlog ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method test.unit.obj.test_replicator.TestObjectReplicator.test_sync_just_calls_sync_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 test.unit.proxy.controllers.test_account.TestGetAccountInfo.test_get_deleted_account_410 ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_close test.unit.proxy.controllers.test_base.TestGetterSource.test_close ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_init test.unit.proxy.controllers.test_base.TestGetterSource.test_init ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_sort test.unit.proxy.controllers.test_base.TestGetterSource.test_sort ... ok test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp test.unit.proxy.controllers.test_base.TestGetterSource.test_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk test.unit.proxy.controllers.test_obj.TestECFunctions.test_chunk_transformer_non_aligned_last_chunk ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_client_range_to_segment_range ... ok test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range test.unit.proxy.controllers.test_obj.TestECFunctions.test_segment_range_to_fragment_range ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'}test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_bucket ... ok test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_has_other_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '448'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_authenticated_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_full_control ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0146 "-" 3656261 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx649a75d8f2004473a5fb9-00665ca3cb" "proxy-server 3656261" 0.0023 "-" 3656261 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:53035/sda1/0/a/c" "tx649a75d8f2004473a5fb9-00665ca3cb" "container-server 3656261" 0.0031 "-" 3656261 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:57305/sdb1/0/a/c" "tx649a75d8f2004473a5fb9-00665ca3cb" "container-server 3656261" 0.0016 "-" 3656261 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx649a75d8f2004473a5fb9-00665ca3cb" "proxy-server 3656261" 0.0301 "-" 3656261 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx649a75d8f2004473a5fb9-00665ca3cb" "proxy-server 3656261" 0.0614 "-" 3656261 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/35 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx649a75d8f2004473a5fb9-00665ca3cb - 0.0785 - - 1717347275.234376907 1717347275.312922955 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txabbfaedf6f5247d590bf5-00665ca3cb" "proxy-server 3656261" 0.0019 "-" 3656261 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:57305/sdb1/2/a1/c1" "txabbfaedf6f5247d590bf5-00665ca3cb" "container-server 3656261" 0.0025 "-" 3656261 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:53035/sda1/2/a1/c1" "txabbfaedf6f5247d590bf5-00665ca3cb" "container-server 3656261" 0.0028 "-" 3656261 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txabbfaedf6f5247d590bf5-00665ca3cb" "proxy-server 3656261" 0.0455 "-" 3656261 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txabbfaedf6f5247d590bf5-00665ca3cb" "proxy-server 3656261" 0.0310 "-" 3656261 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/35 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txabbfaedf6f5247d590bf5-00665ca3cb - 0.0601 - - 1717347275.315209389 1717347275.375327587 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe94613e8a1c94f239a898-00665ca3cb" "proxy-server 3656261" 0.0164 "-" 3656261 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:53035/sda1/0/a/c1" "txe94613e8a1c94f239a898-00665ca3cb" "container-server 3656261" 0.0186 "-" 3656261 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:57305/sdb1/0/a/c1" "txe94613e8a1c94f239a898-00665ca3cb" "container-server 3656261" 0.0031 "-" 3656261 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe94613e8a1c94f239a898-00665ca3cb" "proxy-server 3656261" 0.0710 "-" 3656261 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe94613e8a1c94f239a898-00665ca3cb" "proxy-server 3656261" 0.0499 "-" 3656261 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/35 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txe94613e8a1c94f239a898-00665ca3cb - 0.1023 - - 1717347275.378315687 1717347275.480621338 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3034b8308ee34690a6696-00665ca3cb" "proxy-server 3656261" 0.0131 "-" 3656261 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:53035/sda1/2/a/c2" "tx3034b8308ee34690a6696-00665ca3cb" "container-server 3656261" 0.0125 "-" 3656261 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:57305/sdb1/2/a/c2" "tx3034b8308ee34690a6696-00665ca3cb" "container-server 3656261" 0.0017 "-" 3656261 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3034b8308ee34690a6696-00665ca3cb" "proxy-server 3656261" 0.0589 "-" 3656261 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:35 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3034b8308ee34690a6696-00665ca3cb" "proxy-server 3656261" 0.0395 "-" 3656261 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/35 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx3034b8308ee34690a6696-00665ca3cb - 0.0850 - - 1717347275.483278990 1717347275.568284273 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_server.TestProxyServer.test_error_occurred test.unit.proxy.test_server.TestProxyServer.test_error_occurred ... ok test-replicator INFO: Beginning replication run test-replicator INFO: Replication run OVER test-replicator INFO: Attempted to replicate 0 dbs in 0.00062 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 6 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 0 dbs in 0.00000 seconds (0.00000/s) test-replicator INFO: Removed 0 dbs test-replicator INFO: 0 successes, 0 failures test-replicator INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test-replicator INFO: Attempted to replicate 30 dbs in 246.81358 seconds (0.12155/s) test-replicator INFO: Removed 9 dbs test-replicator INFO: 25 successes, 1 failures test-replicator INFO: diff:5 diff_capped:4 empty:7 hashmatch:8 no_change:6 remote_merge:2 rsync:3 ts_repl:10 direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! test-ic DEBUG: -> 2014-05-27T20:54:11 GET http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - test-ic DEBUG: -> 2014-05-27T20:54:11 HEAD http://127.0.0.1%3Fformat%3Djson 200 123 345 1401224050.98 1401224051.98 1.0 - Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. Storage policy bad-policy uses an EC configuration known to harm data durability. Any data in this policy should be migrated. See https://bugs.launchpad.net/swift/+bug/1639691 for more information. test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config test.unit.common.test_storage_policy.TestStoragePolicies.test_known_bad_ec_config ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_names_EC ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero test.unit.common.test_storage_policy.TestStoragePolicies.test_multiple_policies_with_no_policy_index_zero ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized test.unit.common.test_storage_policy.TestStoragePolicies.test_names_are_normalized ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default test.unit.common.test_storage_policy.TestStoragePolicies.test_no_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_parse_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute test.unit.common.test_storage_policy.TestStoragePolicies.test_policies_type_attribute ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate test.unit.common.test_storage_policy.TestStoragePolicies.test_policy_baseclass_instantiate ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_erasure_coding ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication test.unit.common.test_storage_policy.TestStoragePolicies.test_quorum_size_replication ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies test.unit.common.test_storage_policy.TestStoragePolicies.test_reload_invalid_storage_policies ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough test.unit.common.test_storage_policy.TestStoragePolicies.test_singleton_passthrough ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string test.unit.common.test_storage_policy.TestStoragePolicies.test_split_policy_string ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policies_as_dict_keys ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_get_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_ordering ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr test.unit.common.test_storage_policy.TestStoragePolicies.test_storage_policy_repr ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info test.unit.common.test_storage_policy.TestStoragePolicies.test_swift_info ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_defaults ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_indexes ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_names ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_default ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policies_type_invalid ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_policy_params ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring test.unit.common.test_storage_policy.TestStoragePolicies.test_validate_ring ... ok test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names test.unit.common.test_storage_policy.TestStoragePolicies.test_wacky_int_names ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_closes ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_owner_permission ... ok test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates test.unit.common.test_utils.TestCloseableChain.test_closeable_chain_iterates ... ok test.unit.common.test_utils.TestFileLikeIter.test_close test.unit.common.test_utils.TestFileLikeIter.test_close ... ok test.unit.common.test_utils.TestFileLikeIter.test_get_hub test.unit.common.test_utils.TestFileLikeIter.test_get_hub ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter test.unit.common.test_utils.TestFileLikeIter.test_iter_file_iter ... ok test.unit.common.test_utils.TestFileLikeIter.test_next test.unit.common.test_utils.TestFileLikeIter.test_next ... ok test.unit.common.test_utils.TestFileLikeIter.test_read test.unit.common.test_utils.TestFileLikeIter.test_read ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size test.unit.common.test_utils.TestFileLikeIter.test_read_with_size ... ok test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero test.unit.common.test_utils.TestFileLikeIter.test_read_with_size_zero ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline test.unit.common.test_utils.TestFileLikeIter.test_readline ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline2 test.unit.common.test_utils.TestFileLikeIter.test_readline2 ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline3 test.unit.common.test_utils.TestFileLikeIter.test_readline3 ... ok test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size test.unit.common.test_utils.TestFileLikeIter.test_readline_with_size ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines test.unit.common.test_utils.TestFileLikeIter.test_readlines ... ok test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size test.unit.common.test_utils.TestFileLikeIter.test_readlines_with_size ... ok test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container test.unit.common.test_utils.TestGetExpirerContainer.test_get_expirer_container ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_bad_start ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_basic2 ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_big_reads ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_broken_mid_stream ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_empty ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_leading_crlfs ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_readline_with_tiny_chunks ... ok test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads test.unit.common.test_utils.TestIterMultipartMimeDocuments.test_tiny_reads ... ok test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers test.unit.common.test_utils.TestParseMimeHeaders.test_parse_mime_headers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_old_style_shard_container ... ok fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Expected another 1 bytes for /v1/AUTH_test/mancon/manifest; closing connection fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had a21075a36eeddd084e17611a238c7101, but object MD5 was actually 3902ee7e149eb8313a34757e89e21af6 fake-swift ERROR: Bad response length for /v1/AUTH_test/c/seg_02 as part of /v1/AUTH_test/mancon/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: While processing manifest /v1/AUTH_test/mancon/manifest, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/c/seg_01 test DEBUG: set nice priority to 1 test DEBUG: set ionice class 1 priority 0 WARNING: Unable to modify I/O scheduling class and priority of process. Keeping unchanged! Check logs for more info. test ERROR: Unable to modify ionice priority: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/libc.py", line 326, in _ioprio_set io_class = IO_CLASS_ENUM[io_class] ~~~~~~~~~~~~~^^^^^^^^^^ KeyError: 'class_foo' test DEBUG: set ionice class 2 priority 4 test DEBUG: set nice priority to -15 test DEBUG: set ionice class 3 priority 6 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Unexpected response while deleting object account container obj: 404 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 412 test-expirer DEBUG: b'' test-expirer ERROR: Unexpected response while deleting object account container obj: 500 test-expirer DEBUG: b'' test-expirer ERROR: Exception while deleting object account container obj : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer ERROR: Exception while deleting object .expiring_objects 1717260867 1717260867-a0/c0/o0 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260867 1717260867-a1/c1/o1 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260867 1717260867-a2/c2/o2 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260867 1717260867-a3/c3/o3 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717260867 1717260867-a4/c4/o4 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347266 1717347266-a5/c5/o5 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347266 1717347266-a6/c6/o6 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347266 1717347266-a7/c7/o7 TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer ERROR: Exception while deleting object .expiring_objects 1717347266 1717347266-a8/c8/o8♡ TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry test.unit.obj.test_expirer.TestObjectExpirer.test_failed_delete_keeps_entry ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_config ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_from_kwargs ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_process ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_negative_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_equal_to_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes test.unit.obj.test_expirer.TestObjectExpirer.test_get_process_values_process_greater_than_processes ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod test.unit.obj.test_expirer.TestObjectExpirer.test_hash_mod ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init test.unit.obj.test_expirer.TestObjectExpirer.test_init ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small test.unit.obj.test_expirer.TestObjectExpirer.test_init_concurrency_too_small ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name test.unit.obj.test_expirer.TestObjectExpirer.test_init_internal_client_log_name ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_accounts_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire test.unit.obj.test_expirer.TestObjectExpirer.test_iter_task_to_expire ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break test.unit.obj.test_expirer.TestObjectExpirer.test_object_timestamp_break ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj test.unit.obj.test_expirer.TestObjectExpirer.test_parse_task_obj ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue test.unit.obj.test_expirer.TestObjectExpirer.test_pop_queue ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency test.unit.obj.test_expirer.TestObjectExpirer.test_process_based_concurrency ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_report test.unit.obj.test_expirer.TestObjectExpirer.test_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order test.unit.obj.test_expirer.TestObjectExpirer.test_round_robin_order ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_catches_usual_exceptions ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random test.unit.obj.test_expirer.TestObjectExpirer.test_run_forever_initial_sleep_random ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_calls_report ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_nothing_to_do ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_rate_limited ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem test.unit.obj.test_expirer.TestObjectExpirer.test_run_once_unicode_problem ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container test.unit.obj.test_expirer.TestObjectExpirer.test_skip_task_account_without_task_container ... ok test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted test.unit.obj.test_expirer.TestObjectExpirer.test_success_gets_counted ... ok test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_duplicate_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveECDuplicationFactor.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_GET test.unit.obj.test_server.TestZeroCopy.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_GET_big test.unit.obj.test_server.TestZeroCopy.test_GET_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_quarantine test.unit.obj.test_server.TestZeroCopy.test_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file test.unit.obj.test_server.TestZeroCopy.test_quarantine_on_well_formed_zero_byte_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing test.unit.obj.test_ssync_receiver.TestModuleMethods.test_decode_missing ... ok test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted test.unit.obj.test_ssync_receiver.TestModuleMethods.test_encode_wanted ... ok test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_device_not_available ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_invalid_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_SSYNC_read_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index test.unit.obj.test_ssync_receiver.TestSsyncRxServer.test_bad_request_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_acl_headers_translated_for_GET_HEAD ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_account_info_in_response_env ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_add_acls_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_cache_key_impossible_cases ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_get_deleted_account ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_long_acct_names ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_DELETE ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_response_code_for_PUT ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_stripping_swift_admin_headers ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_swift_owner ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_account.TestAccountController4Replicas.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication test.unit.proxy.controllers.test_base.TestNodeIter.test_annotate_node_with_use_replication ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_default_fake_ring ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_does_not_mutate_supplied_nodes ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs test.unit.proxy.controllers.test_base.TestNodeIter.test_iter_with_handoffs ... ok test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration test.unit.proxy.controllers.test_base.TestNodeIter.test_multi_iteration ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 31, 'name': 'o30', 'created_at': '1717347295.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 32, 'name': 'o31', 'created_at': '1717347296.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 33, 'name': 'o32', 'created_at': '1717347297.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 34, 'name': 'o33', 'created_at': '1717347298.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 35, 'name': 'o34', 'created_at': '1717347299.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 36, 'name': 'o35', 'created_at': '1717347300.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 37, 'name': 'o36', 'created_at': '1717347301.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 38, 'name': 'o37', 'created_at': '1717347302.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 39, 'name': 'o38', 'created_at': '1717347303.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 40, 'name': 'o39', 'created_at': '1717347304.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '590112ac-f315-48df-9be6-bec83ec01cdf-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 41, 'name': 'o40', 'created_at': '1717347305.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 42, 'name': 'o41', 'created_at': '1717347306.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 43, 'name': 'o42', 'created_at': '1717347307.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 44, 'name': 'o43', 'created_at': '1717347308.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 45, 'name': 'o44', 'created_at': '1717347309.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 46, 'name': 'o45', 'created_at': '1717347310.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 47, 'name': 'o46', 'created_at': '1717347311.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 48, 'name': 'o47', 'created_at': '1717347312.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 49, 'name': 'o48', 'created_at': '1717347313.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 50, 'name': 'o49', 'created_at': '1717347314.00000', 'size': 0, 'content_type': 'content-type-old', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '590112ac-f315-48df-9be6-bec83ec01cdf-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 51, 'name': 'o101', 'created_at': '1717347415.00000', 'size': 0, 'content_type': 'content-type-new', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '590112ac-f315-48df-9be6-bec83ec01cdf-sda') test DEBUG: /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 51 (0.0662723s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (51, 'b79c84428ee232694bf709e6eb63a222', '590112ac-f315-48df-9be6-bec83ec01cdf-sda', '1717347264.84894', '1717347264.00000', '0', '', '1717347264.00000', 51, 0) test DEBUG: /tmp/tmpllfao470/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09555 seconds (10.46521/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:0 ts_repl:0 test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdc test WARNING: Could not find a non-zero weight device for handoff partition 0, falling back device sdb test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00048 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00148 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00037 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp0dt9jd5m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'fef7b383-3a34-43d2-ae2c-f297134ddbd5-sda', '1717347265.31382', '1717344000.00000', '0', '', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmp0dt9jd5m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'fef7b383-3a34-43d2-ae2c-f297134ddbd5-sda') test DEBUG: /tmp/tmp0dt9jd5m/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0263224s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'fef7b383-3a34-43d2-ae2c-f297134ddbd5-sda', '1717347265.31382', '1717344000.00000', '0', '', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fef7b383-3a34-43d2-ae2c-f297134ddbd5-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmp0dt9jd5m/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.18451 seconds (5.41984/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp0dt9jd5m/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_old_style_shard ... ok test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard test.unit.container.test_replicator.TestReplicatorSync.test_misplaced_rows_replicate_and_enqueue_from_shard ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBroker.test_initialize_container_broker_in_default ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'Content-Md5': 'i4CS9WiTvEfCFGqAHP8j0A==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8b8092f56893bc47c2146a801cff23d0', 'Content-Length': '1030946'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_no_key ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_quiet ... ok test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0108 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0139 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0117 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0123 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0174 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0228 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0247 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0029 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0177 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0192 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0031 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0140 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0947 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0129 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0177 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0207 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0211 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0218 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0181 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0141 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0174 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0009 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0005 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0218 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0240 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0022 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0247 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0106 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0194 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0135 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0134 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0190 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0182 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0180 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0037 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0119 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0033 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0126 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0191 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0110 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0107 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0072 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0104 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0206 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0091 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0005 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0273 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0105 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0493 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0040 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0135 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0032 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0265 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0031 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0042 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0039 "-" 3656221 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_auto_create ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_not_found ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_container_in_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_except_for_used_policies ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ Quarantined /tmp/tmp11y9386n/dev/dbs/par/pre/db to /tmp/tmp11y9386n/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmp11y9386n/dev/dbs/par/pre/db to /tmp/tmp11y9386n/dev/quarantined/tests/db due to malformed database Quarantined /tmp/tmp11y9386n/dev/dbs/par/pre/db to /tmp/tmp11y9386n/dev/quarantined/tests/db due to corrupted database Quarantined /tmp/tmp11y9386n/dev/dbs/par/pre/db to /tmp/tmp11y9386n/dev/quarantined/tests/db due to disk error while accessing database test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors test.unit.common.test_db.TestDatabaseBroker.test_possibly_quarantine_db_errors ... ok test.unit.common.test_db.TestDatabaseBroker.test_put_record test.unit.common.test_db.TestDatabaseBroker.test_put_record ... ok Quarantined /tmp/tmpn2q4ad9g/dev/dbs/par/pre/db to /tmp/tmpn2q4ad9g/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_reclaim_missing_container_info ... ok test.unit.common.test_db.TestDatabaseBroker.test_skip_commits test.unit.common.test_db.TestDatabaseBroker.test_skip_commits ... ok Quarantined /tmp/tmpd3gnexnv/dev/dbs/par/pre/db to /tmp/tmpd3gnexnv/dev/quarantined/containers/db due to missing row in container_stat table test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info test.unit.common.test_db.TestDatabaseBroker.test_update_metadata_missing_container_info ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_called_from_update_metadata ... ok test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata test.unit.common.test_db.TestDatabaseBroker.test_validate_metadata_is_not_called_from_update_metadata ... ok test.unit.common.test_db.TestGetDBConnection.test_invalid_path test.unit.common.test_db.TestGetDBConnection.test_invalid_path ... ok test.unit.common.test_db.TestGetDBConnection.test_locked_db test.unit.common.test_db.TestGetDBConnection.test_locked_db ... ok test.unit.common.test_db.TestGetDBConnection.test_normal_case test.unit.common.test_db.TestGetDBConnection.test_normal_case ... ok test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided test.unit.common.test_utils.TestDistributeEvenly.test_evenly_divided ... ok test.unit.common.test_utils.TestDistributeEvenly.test_just_one test.unit.common.test_utils.TestDistributeEvenly.test_just_one ... ok test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items test.unit.common.test_utils.TestDistributeEvenly.test_more_buckets_than_items ... ok test.unit.common.test_utils.TestDistributeEvenly.test_uneven test.unit.common.test_utils.TestDistributeEvenly.test_uneven ... ok test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 test.unit.common.utils.test_ipaddrs.TestExpandIPv6.test_expand_ipv6 ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config test.unit.obj.test_diskfile.TestDiskFileRouter.test_invalid_policy_config ... ok test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy test.unit.obj.test_diskfile.TestDiskFileRouter.test_policy ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_audit_location_class ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_auditor_status ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_finding_of_hashdirs ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_only_catch_expected_errors ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_files ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_skipping_unmounted_devices ... ok test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle test.unit.obj.test_diskfile.TestObjectAuditLocationGenerator.test_update_auditor_status_throttle ... ok test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructor.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_command_line_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution test.unit.obj.test_replicator.TestMultiProcessReplicator.test_device_distribution ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks test.unit.obj.test_replicator.TestMultiProcessReplicator.test_more_workers_than_disks ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing test.unit.obj.test_replicator.TestMultiProcessReplicator.test_no_multiprocessing ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies test.unit.obj.test_replicator.TestMultiProcessReplicator.test_override_policies ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_forever ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_run_once ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides test.unit.obj.test_replicator.TestMultiProcessReplicator.test_recon_skipped_with_overrides ... ok test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging test.unit.obj.test_replicator.TestMultiProcessReplicator.test_worker_logging ... ok test.unit.obj.test_updater.TestSweepStats.test_copy test.unit.obj.test_updater.TestSweepStats.test_copy ... ok test.unit.obj.test_updater.TestSweepStats.test_reset test.unit.obj.test_updater.TestSweepStats.test_reset ... ok test.unit.obj.test_updater.TestSweepStats.test_since test.unit.obj.test_updater.TestSweepStats.test_since ... ok test.unit.obj.test_updater.TestSweepStats.test_str test.unit.obj.test_updater.TestSweepStats.test_str ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate ... ok test INFO: - - - [02/Jun/2024:16:54:26 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0135 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0149 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0023 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0939 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 174 "-" "-" "-" 0.0159 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0212 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0015 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0190 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0220 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0124 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0154 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 6 "-" "-" "-" 0.0028 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0125 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0127 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0184 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0031 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0196 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 324 "-" "-" "-" 0.0187 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/%22%27" 201 - "-" "-" "-" 0.0174 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/%22%27" 200 74 "-" "-" "-" 0.0030 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0310 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/%22%3Cword" 201 - "-" "-" "-" 0.0022 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 203 "-" "-" "-" 0.0250 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0173 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/%22%3Cword-test" 201 - "-" "-" "-" 0.0022 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 200 102 "-" "-" "-" 0.0136 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0178 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 400 21 "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 406 - "-" "-" "-" 0.0011 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 400 34 "-" "-" "-" 0.0005 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/./a" 400 20 "-" "-" "-" 0.0005 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0007 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0224 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0071 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0194 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0179 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0017 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0121 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0122 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0139 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0126 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0155 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0119 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0082 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0250 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0037 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0012 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0136 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0205 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0128 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0066 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0102 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0094 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0058 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0128 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0390 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0170 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 404 - "-" "-" "-" 0.0018 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 400 26 "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0128 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0088 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0490 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0106 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0035 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0247 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0038 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3656197 - test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata test.unit.account.test_server.TestAccountController.test_PUT_GET_sys_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE test.unit.account.test_server.TestAccountController.test_PUT_after_DELETE ... ok test.unit.account.test_server.TestAccountController.test_PUT_auto_create test.unit.account.test_server.TestAccountController.test_PUT_auto_create ... ok test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space test.unit.account.test_server.TestAccountController.test_PUT_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition test.unit.account.test_server.TestAccountController.test_PUT_invalid_partition ... ok test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata test.unit.account.test_server.TestAccountController.test_PUT_non_utf8_metadata ... ok test.unit.account.test_server.TestAccountController.test_PUT_not_found test.unit.account.test_server.TestAccountController.test_PUT_not_found ... ok test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race test.unit.account.test_server.TestAccountController.test_PUT_simulated_create_race ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works test.unit.account.test_server.TestAccountController.test_REPLICATE_complete_rsync_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space test.unit.account.test_server.TestAccountController.test_REPLICATE_insufficient_space ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works test.unit.account.test_server.TestAccountController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync test.unit.account.test_server.TestAccountController.test_REPLICATE_unknown_sync ... ok test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works test.unit.account.test_server.TestAccountController.test_REPLICATE_value_error_works ... ok test.unit.account.test_server.TestAccountController.test__call__raise_timeout test.unit.account.test_server.TestAccountController.test__call__raise_timeout ... ok test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD test.unit.account.test_server.TestAccountController.test_content_type_on_HEAD ... ok test.unit.account.test_server.TestAccountController.test_correct_allowed_method test.unit.account.test_server.TestAccountController.test_correct_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_invalid_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account test.unit.account.test_server.TestAccountController.test_create_reserved_container_in_account ... ok test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account test.unit.account.test_server.TestAccountController.test_create_reserved_namespace_account ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_no_public ... ok test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public test.unit.account.test_server.TestAccountController.test_delimiter_with_reserved_and_public ... ok test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies test.unit.account.test_server.TestAccountController.test_empty_except_for_used_policies ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET ... ok test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b test ERROR: Returning 497 because of blacklisting: b fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, max LO GET time of 86400s exceeded fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-manifest-a, GET of submanifest /v1/AUTH_test/gettest/manifest-a failed with status 403 (

Forbidden

Access was denied to this reso...) fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-abcd, got 401 (

Unauthorized

This server could not verif...) while retrieving /v1/AUTH_test/gettest/c_15 fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, GET of submanifest /v1/AUTH_test/gettest/manifest-bc failed with status 401 (

Unauthorized

This server could not verif...) fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != wrong! or 5 != 5. fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/a_5 etag: 594f803b380a41396ed63dca39503542 != 594f803b380a41396ed63dca39503542 or 5 != 999999. fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-avail, got 503 (

Service Unavailable

The server is curren...) while retrieving /v1/AUTH_test/gettest/not_avail_obj fake-swift ERROR: While processing manifest /v1/AUTH_test/gettest/manifest-not-exists, got 404 (

Not Found

The resource could not be foun...) while retrieving /v1/AUTH_test/gettest/not_exists_obj fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 2 (char 1) fake-swift ERROR: while fetching /v1/AUTH_test/gettest/manifest-abcd, JSON-decoding of submanifest /v1/AUTH_test/gettest/manifest-bc failed with 500 Internal Error fake-swift ERROR: Bad MD5 checksum for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 594f803b380a41396ed63dca39503542, but object MD5 was actually fb0e22c79ac75679e9881e6ba183b354 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != wrong! or 10 != 10. fake-swift ERROR: Bad response length for /v1/AUTH_test/gettest/a_5 as part of /v1/AUTH_test/gettest/manifest: headers had 5, but response length was actually 4 fake-swift ERROR: Object segment no longer valid: /v1/AUTH_test/gettest/b_10 etag: 82136b4240d6ce4ea7d03e51469a393b != 82136b4240d6ce4ea7d03e51469a393b or 10 != 999999. fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded fake-swift ERROR: Unable to load SLO manifest: Expecting value: line 1 column 1 (char 0) fake-swift ERROR: While processing manifest '/v1/AUTH_test/gettest/man1', max recursion depth was exceeded test WARNING: More than one part in a single-part response? proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp9jehqxpe/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp9jehqxpe/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp9jehqxpe/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp9jehqxpe/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:47 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.6843 "-" 3656212 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:47 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.2098 "-" 3656212 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.6351 "-" 3656212 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0244 "-" 3656212 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx27b0ff85b37e48bc9e043-00665ca3d8" "proxy-server 3656212" 0.0024 "-" 3656212 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:35979/sda1/1/a/c" "tx27b0ff85b37e48bc9e043-00665ca3d8" "container-server 3656212" 0.0023 "-" 3656212 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:45671/sdb1/1/a/c" "tx27b0ff85b37e48bc9e043-00665ca3d8" "container-server 3656212" 0.0022 "-" 3656212 0 test_digest_utils WARNING: The following digest algorithms are allowed by default but deprecated: sha1. Support will be disabled by default in a future release, and later removed entirely. test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 test_digest_utils WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test_digest_utils WARNING: The following digest algorithms are configured but not supported: md5 proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET https://cluster.domain/bucket/key HTTP/1.0" 200 106 0.000098 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET http://cluster.domain/v1/acct/cont/obj HTTP/1.0" 200 112 0.000072 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET ftp://cluster.domain/bucket/key HTTP/1.0" 200 126 0.000081 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET https://cluster.domain HTTP/1.0" 200 117 0.000116 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET http:omg//wtf/bbq HTTP/1.0" 200 112 0.000093 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: tx95d5c2c475714e11aea25-00665ca3c0) proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) test ERROR: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD') (txn: test-trans-id) proxy INFO: wsgi exiting (txn: test-trans-id) proxy INFO: (3656266) wsgi exited, is_accepting=True (txn: test-trans-id) proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Bad request syntax ('ONLY-METHOD'), (txn: test-trans-id) proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "PUT /path HTTP/1.0" 200 105 0.000084 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET ///some-leading-slashes HTTP/1.0" 200 118 0.000072 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET /ohÿboy%what$now%E2%80%bd HTTP/1.0" 200 120 0.000084 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET /some%fFpath%D8%AA HTTP/1.0" 200 113 0.000070 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET / HTTP/1.1" 200 120 0.000129 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 44\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET /%FF HTTP/1.1" 200 123 0.000123 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 47\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "/%FF", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET /ÿ HTTP/1.1" 200 126 0.000107 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 50\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "/\\u00ff", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "PUT /Here%20Is%20A%20SnowMan:☃ HTTP/1.0" 200 181 0.000086 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 86\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\nConnection: close\r\n\r\n{"RAW_PATH_INFO": "/Here%20Is%20A%20SnowMan:\\u00e2\\u0098\\u0083", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "POST /?and%20it=does+nothing+to+params&PALMTREE=ð%9fŒ%b4 HTTP/1.1" 200 178 0.000110 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 101\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "/", "QUERY_STRING": "and%20it=does+nothing+to+params&PALMTREE=\\u00f0%9f\\u008c%b4"}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET // HTTP/1.1" 200 121 0.000105 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 45\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "//", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET //bar HTTP/1.1" 200 124 0.000102 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 48\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "//bar", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET //////baz HTTP/1.1" 200 128 0.000103 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True b'HTTP/1.1 200 OK\r\nContent-Length: 52\r\nDate: Sun, 02 Jun 2024 16:54:24 GMT\r\n\r\n{"RAW_PATH_INFO": "//////baz", "QUERY_STRING": null}' proxy INFO: (3656266) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656266) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:54:24] "GET /someurl HTTP/1.0" 200 102 0.000082 proxy INFO: wsgi exiting proxy INFO: (3656266) wsgi exited, is_accepting=True test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_errors ... ok test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1!test.unit.proxy.test_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_server.TestProxyServer.test_exception_occurred ... ok test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_info_defaults test.unit.proxy.test_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_server.TestProxyServer.test_node_timing test.unit.proxy.test_server.TestProxyServer.test_node_timing ... ok test.unit.proxy.test_server.TestProxyServer.test_private_method_request test.unit.proxy.test_server.TestProxyServer.test_private_method_request ... ok test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid test.unit.proxy.test_mem_server.TestContainerController.test_CORS_valid ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE test.unit.proxy.test_mem_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_mem_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket0 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket1 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket2 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket3 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket4 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket5 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket6 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket test.unit.common.middleware.s3api.test_service.TestS3ApiServiceAcl.test_service_GET_without_owner_bucket ... ok test.unit.common.middleware.test_bulk.TestConfig.test_defaults test.unit.common.middleware.test_bulk.TestConfig.test_defaults ... ok test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency test.unit.common.middleware.test_bulk.TestConfig.test_delete_concurrency ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_invalid_ranges ... ok test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range test.unit.common.middleware.test_copy.TestServerSideCopyMiddlewareWithEC.test_COPY_with_single_range ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_existing_swift_owner_ignored ... ok test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix test.unit.common.middleware.test_keystoneauth.ServiceTokenFunctionality.test_unknown_prefix ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_path_single ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_primary_path_multiple ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_authed_for_second_path_with_operator_role_and_service ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_denied_for_second_path ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_delete_of_second_path_by_reseller_admin ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_path_single ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_for_service_user ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_operator_role ... ok test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service test.unit.common.middleware.test_tempauth.ServiceTokenFunctionality.test_denied_for_second_path_with_only_service ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_float_replica_count_builder_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_compose_rings_rebalance_needed ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_only_one_ring_in_the_args_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_device_in_the_different_rings_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_composite_same_region_in_the_different_rings_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_part_power_error ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_different_replica_count_works ... ok test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap test.unit.common.ring.test_composite_builder.TestCompositeBuilder.test_ring_swap ... ok test.unit.common.test_manager.TestManager.test_create test.unit.common.test_manager.TestManager.test_create ... ok test.unit.common.test_manager.TestManager.test_default_strict test.unit.common.test_manager.TestManager.test_default_strict ... ok test.unit.common.test_manager.TestManager.test_force_reload test.unit.common.test_manager.TestManager.test_force_reload ... ok test.unit.common.test_manager.TestManager.test_get_command test.unit.common.test_manager.TestManager.test_get_command ... ok test.unit.common.test_manager.TestManager.test_iter test.unit.common.test_manager.TestManager.test_iter ... ok test.unit.common.test_manager.TestManager.test_list_commands test.unit.common.test_manager.TestManager.test_list_commands ... ok test.unit.common.test_manager.TestManager.test_no_daemon test.unit.common.test_manager.TestManager.test_no_daemon ... ok test.unit.common.test_manager.TestManager.test_no_wait test.unit.common.test_manager.TestManager.test_no_wait ... ok test.unit.common.test_manager.TestManager.test_once test.unit.common.test_manager.TestManager.test_once ... ok test.unit.common.test_manager.TestManager.test_reload test.unit.common.test_manager.TestManager.test_reload ... ok test.unit.common.test_manager.TestManager.test_restart test.unit.common.test_manager.TestManager.test_restart ... ok test.unit.common.test_manager.TestManager.test_run_command test.unit.common.test_manager.TestManager.test_run_command ... ok test.unit.common.test_manager.TestManager.test_shutdown test.unit.common.test_manager.TestManager.test_shutdown ... ok test.unit.common.test_manager.TestManager.test_start test.unit.common.test_manager.TestManager.test_start ... ok test.unit.common.test_manager.TestManager.test_status test.unit.common.test_manager.TestManager.test_status ... ok test.unit.common.test_manager.TestManager.test_stop test.unit.common.test_manager.TestManager.test_stop ... ok test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout test.unit.common.test_manager.TestManager.test_stop_kill_after_timeout ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same test.unit.common.test_swob.TestConditionalIfUnmodifiedSince.test_same ... ok test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type test.unit.common.test_utils.TestParseContentDisposition.test_basic_content_type ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space test.unit.common.test_utils.TestParseContentDisposition.test_content_disposition_without_white_space ... ok test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset test.unit.common.test_utils.TestParseContentDisposition.test_content_type_with_charset ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking test.unit.common.test_utils.TestPipeMutex.test_blocking ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership test.unit.common.test_utils.TestPipeMutex.test_blocking_preserves_ownership ... ok test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool test.unit.common.test_utils.TestPipeMutex.test_blocking_tpool ... ok test.unit.common.test_utils.TestPipeMutex.test_nonblocking test.unit.common.test_utils.TestPipeMutex.test_nonblocking ... ok test.unit.common.test_utils.TestPipeMutex.test_recursive test.unit.common.test_utils.TestPipeMutex.test_recursive ... ok test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire test.unit.common.test_utils.TestPipeMutex.test_release_without_acquire ... ok test.unit.common.test_utils.TestPipeMutex.test_too_many_releases test.unit.common.test_utils.TestPipeMutex.test_too_many_releases ... ok test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser test.unit.common.test_utils.TestPipeMutex.test_wrong_releaser ... ok test.unit.common.test_utils.TestUTC.test_tzname test.unit.common.test_utils.TestUTC.test_tzname ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_empty_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed test.unit.common.test_utils.TestUnlinkOlder.test_unlink_mixed ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_exists_paths ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough test.unit.common.test_utils.TestUnlinkOlder.test_unlink_not_old_enough ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_file ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_now ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists test.unit.common.test_utils.TestUnlinkOlder.test_unlink_older_than_path_not_exists ... ok test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths test.unit.common.test_utils.TestUnlinkOlder.test_unlink_paths ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_DELETE_without_permission ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_PUT_recreate_replication_race ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments test.unit.cli.test_ringbuilder.TestCommands.test_main_no_arguments ... ok REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'ea4b6417-0333-4596-867a-71552a949a1d-sda', '1717347265.52769', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('ea4b6417-0333-4596-867a-71552a949a1d-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', 'ea4b6417-0333-4596-867a-71552a949a1d-sda', '1717347265.52769', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('ea4b6417-0333-4596-867a-71552a949a1d-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmplnoqalh5/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'fa4c8591-87e7-4d3e-81b4-5352da531db9-sda', '1717347265.67438', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717347265.68569"]}', '1717344000.00000', 0, 0) test DEBUG: /tmp/tmplnoqalh5/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'fa4c8591-87e7-4d3e-81b4-5352da531db9-sda') test DEBUG: /tmp/tmplnoqalh5/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.0222867s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', 'fa4c8591-87e7-4d3e-81b4-5352da531db9-sda', '1717347265.67438', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Root": ["a/c", "1717347265.68569"]}', '1717344000.00000', 0, 0) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('fa4c8591-87e7-4d3e-81b4-5352da531db9-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmplnoqalh5/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.22102 seconds (4.52454/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmplnoqalh5/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', '9a1028c1-a8f2-4f72-9113-e21cdbfddf8e-sda', '1717347265.92675', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9a1028c1-a8f2-4f72-9113-e21cdbfddf8e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '35ca71e51a826434bcb62f936521e4b9', '9a1028c1-a8f2-4f72-9113-e21cdbfddf8e-sda', '1717347265.92675', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('9a1028c1-a8f2-4f72-9113-e21cdbfddf8e-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpz175380g/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '8c601449-25fe-4c9d-85ba-8a3b1f66200a-sda', '1717347266.10076', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717347266.11517"]}', '1717344000.00000', 0, 1) test DEBUG: /tmp/tmpz175380g/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_items, ([{'ROWID': 1, 'name': 'o', 'created_at': '1717344002.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o2', 'created_at': '1717344003.00000+186a0+186a0', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '8c601449-25fe-4c9d-85ba-8a3b1f66200a-sda') test DEBUG: /tmp/tmpz175380g/sda/containers/0/df1/7d69a3daa789abb831fde54674123df1/7d69a3daa789abb831fde54674123df1.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.01969s) REPLICATE: /sdb/0/7d69a3daa789abb831fde54674123df1, merge_syncs, ([],) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, sync, (2, 'd3e341cf02c266c3ca2b59b10642b22d', '8c601449-25fe-4c9d-85ba-8a3b1f66200a-sda', '1717347266.10076', '1717344000.00000', '0', '{"X-Container-Sysmeta-Shard-Quoted-Root": ["a/c", "1717347266.11517"]}', '1717344000.00000', 0, 1) REPLICATE: /sdc/0/7d69a3daa789abb831fde54674123df1, complete_rsync, ('8c601449-25fe-4c9d-85ba-8a3b1f66200a-sda', '7d69a3daa789abb831fde54674123df1.db') test DEBUG: Adding 2 objects to the reconciler at /tmp/tmpz175380g/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.22246 seconds (4.49521/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpz175380g/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', 'a0f79db7-af5b-4fd5-b251-bd61e776eedd-sda', '1717347266.34849', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('a0f79db7-af5b-4fd5-b251-bd61e776eedd-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, '6787ede482dace3718e3c023ac01a5b0', 'a0f79db7-af5b-4fd5-b251-bd61e776eedd-sda', '1717347266.34849', '1717344000', '0', '', '1717344000', 2, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('a0f79db7-af5b-4fd5-b251-bd61e776eedd-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpskyhyu_f/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'da3ed2b1aa16d584db7b344f718316fa', 'f6f958af-69f7-4c0f-a583-172fc6cc84c0-sda', '1717347266.55633', '1717347266.00000', '0', '', '1717347266.00000', 1, 1) test DEBUG: /tmp/tmpskyhyu_f/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-unu', 'created_at': '1717347268.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o-nulo', 'created_at': '1717347269.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'f6f958af-69f7-4c0f-a583-172fc6cc84c0-sda') test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize test.unit.container.test_replicator.TestReplicatorSync.test_multiple_out_sync_reconciler_enqueue_normalize ... ok test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook test.unit.container.test_replicator.TestReplicatorSync.test_post_replicate_hook ... ok test-expirer ERROR: Exception while deleting object .expiring_objects 1717347266 1717347266-a9/c9/o9ø TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 421, in delete_object self.delete_actual_object(target_path, delete_timestamp, TypeError: TestObjectExpirer.test_failed_delete_keeps_entry..deliberately_blow_up() takes 2 positional arguments but 3 were given test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer ERROR: Unexcepted error handling task '1717260869-invalid0': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid0 test-expirer ERROR: Unexcepted error handling task '1717260869-invalid1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 265, in iter_task_to_expire target_object = parse_task_obj(task_object) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 65, in parse_task_obj split_path('/' + target_path, 3, 3, True) File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1165, in split_path raise ValueError('Invalid path: %s' % quote(path)) ValueError: Invalid path: /invalid1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test-expirer INFO: Pass so far 0s; 0 objects expired test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0000', 'task_object': '0000-invalid0', 'delete_timestamp': 0000000000.00000_0000000000000000, 'target_path': 'invalid0', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0001', 'task_object': '0001-invalid1', 'delete_timestamp': 0000000001.00000_0000000000000000, 'target_path': 'invalid1', 'is_async_delete': False} test-expirer ERROR: Unexcepted error handling task {'task_account': '.expiring_objects', 'task_container': '0002', 'task_object': '0002-invalid2', 'delete_timestamp': 0000000002.00000_0000000000000000, 'target_path': 'invalid2', 'is_async_delete': False} test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 380, in run_forever self.run_once(*args, **kwargs) File "/build/reproducible-path/swift-2.33.0/test/unit/obj/test_expirer.py", line 907, in raise_exceptions raise Exception('exception %d' % raises[0]) Exception: exception 1 test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer ERROR: Unhandled exception: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/expirer.py", line 330, in run_once self.swift.get_account_info(task_account) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'get_account_info' test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test-expirer DEBUG: Run begin test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 0 objects expired test-expirer DEBUG: Run begin test-expirer INFO: Pass beginning for task account .expiring_objects; 4 possible containers; 12 possible objects test-expirer DEBUG: Run end test-expirer INFO: Pass completed in 0s; 10 objects expired test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test WARNING: Handoff requested (9) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShard.test_get_update_shard_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_private ... ok test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0448 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0042 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0239 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0046 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0099 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:27 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0292 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0031 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0028 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0193 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0037 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0204 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.0109 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0023 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0008 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.3556 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0005 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.5212 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0030 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.6579 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.1134 "-" 3656221 - test DEBUG: - - - [02/Jun/2024:16:54:29 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0008 "-" 3656221 - test DEBUG: - - - [02/Jun/2024:16:54:29 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0008 "-" 3656221 - test DEBUG: - - - [02/Jun/2024:16:54:29 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0008 "-" 3656221 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:54:30 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0061 "-" 3656221 - test DEBUG: - - - [02/Jun/2024:16:54:30 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0007 "-" 3656221 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:54:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.8783 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:30 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0031 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:30 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:30 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:30 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:30 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.3151 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0002 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5064 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.1105 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.0188 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.0193 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0188 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0029 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0144 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0028 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0029 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0218 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0023 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0131 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0195 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0023 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0022 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0032 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.2533 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0248 "-" 3656221 2 test INFO: - - - [02/Jun/2024:16:54:32 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.2685 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.4263 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:33 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3656221 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_empty_policy_stats ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_init ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_null_markers ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_non_default ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBroker.test_is_deleted test.unit.container.test_backend.TestContainerBroker.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBroker.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable test.unit.container.test_backend.TestContainerBroker.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_last_modified_time test.unit.container.test_backend.TestContainerBroker.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_DELETE_without_permission ... ok test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files test.unit.container.test_backend.TestContainerBroker.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_is_deleted ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test ERROR: ERROR Could not get container info /tmp/tmp9v1x6cyu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test DEBUG: Audit passed for test DEBUG: Audit passed for test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.02s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test DEBUG: Audit passed for test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test.unit.common.test_db_auditor.TestAuditor.test_run_forever test.unit.common.test_db_auditor.TestAuditor.test_run_forever ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'Content-Md5': '/JzPJjlXCqpRBmlyhGXNDg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'fc9ccf2639570aaa510669728465cd0e', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'Content-Md5': 'PggRsy85Tu07EsbATYB/cA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '3e0811b32f394eed3b12c6c04d807f70', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'Content-Md5': 'zDBBHq17IpuHzskOm+GHEQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'cc30411ead7b229b87cec90e9be18711', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'Content-Md5': 'hTnOh9sxGnzPiScnD+t0Mg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '8539ce87db311a7ccf8927270feb7432', 'Content-Length': '139'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'Content-Md5': 'mbEtd8ufKpkpCIfMjGD7Tw==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '99b12d77cb9f2a99290887cc8c60fb4f', 'Content-Length': '90'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_to_object ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_too_many_keys ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_exception ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBroker.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBroker.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items test.unit.container.test_backend.TestContainerBroker.test_merge_items ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green test.unit.container.test_backend.TestContainerBroker.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBroker.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBroker.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBroker.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_find_shard_ranges_with_misplaced_objects ... ok test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txdb2dbfe7aa264cab86269-00665ca3cb) test INFO: Client disconnected on read of '/a/c' (txn: txdb2dbfe7aa264cab86269-00665ca3cb) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: txa9260cd4f08640b4a7b38-00665ca3cb) test INFO: Client disconnected on read of '/a/c' (txn: txa9260cd4f08640b4a7b38-00665ca3cb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: txf08a30a180b0440381ae8-00665ca3cc) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: txf08a30a180b0440381ae8-00665ca3cc) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: txf08a30a180b0440381ae8-00665ca3cc) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: txf08a30a180b0440381ae8-00665ca3cc) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test DEBUG: Skipping the update of recently changed /tmp/tmpd8il4zkx/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmpd8il4zkx/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmpd8il4zkx/sdp/objects/auditor_status_ALL.json test DEBUG: Skipping the update of recently changed /tmp/tmpd8il4zkx/sdq/objects/auditor_status_ALL.json test DEBUG: Skipping: /tmp/tmpcb7ki_r4/garbage is not a directory test DEBUG: Skipping: /tmp/tmpcb7ki_r4/garbage is not mounted test DEBUG: Skipping: /tmp/tmpfv60zjmh/sdq is not mounted test-replicator INFO: [worker 1/2 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 1/2 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/2 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 2/2 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 2/3 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 2/3 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 3/3 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 3/3 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=3656208] Running object replicator in script mode. test-replicator INFO: [worker 1/3 pid=3656208] Object replication complete (once). (2.00 minutes) test-replicator INFO: [worker 1/3 pid=8804] Running object replicator in script mode. test-replicator DEBUG: [worker 1/3 pid=8804] debug message test-replicator INFO: [worker 1/3 pid=8804] info message test-replicator WARNING: [worker 1/3 pid=8804] warning message test-replicator ERROR: [worker 1/3 pid=8804] error message test-replicator INFO: [worker 1/3 pid=8804] Object replication complete (once). (0.00 minutes) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx7a1ddf88ad844e199a2ac-00665ca3da) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sdb (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sdu (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sdw (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sdq (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sdy (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_all_primaries_error_limited ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_brokers ... ok test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0186 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0269 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0028 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0102 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0198 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0058 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:28 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0033 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 1.4015 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:29 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0030 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:30 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.5278 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "DELETE /sda1/p/a" 204 - "-" "-" "-" 0.8257 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 403 16 "-" "-" "-" 0.0017 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/.a/c" 201 - "-" "-" "-" 0.5073 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a/.c" 404 - "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/./a" 400 20 "-" "-" "-" 0.0003 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.1295 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 400 28 "-" "-" "-" 0.0018 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:31 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0241 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0207 "-" 3656197 - test DEBUG: - - - [02/Jun/2024:16:54:32 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3656197 - test DEBUG: - - - [02/Jun/2024:16:54:32 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3656197 - test DEBUG: - - - [02/Jun/2024:16:54:32 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0005 "-" 3656197 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 279, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:54:32 +0000] "REPLICATE /sda1/p/a/" 500 833 "-" "-" "-" 0.0041 "-" 3656197 - test DEBUG: - - - [02/Jun/2024:16:54:32 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0004 "-" 3656197 - test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/server.py", line 313, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_server.py", line 2606, in mock_put_method raise Exception() Exception test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0150 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0016 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.1771 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a/c%00stuff" 400 36 "-" "-" "-" 0.0001 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/foo%00bar" 400 34 "-" "-" "-" 0.0003 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.2899 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:32 +0000] "PUT /sda1/p/a/%00c%00stuff" 201 - "-" "-" "-" 0.0601 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:33 +0000] "PUT /sda1/p/%00a" 201 - "-" "-" "-" 0.3510 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:33 +0000] "PUT /sda1/p/%00foo%00bar" 201 - "-" "-" "-" 0.4191 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:33 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.4186 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.1261 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.6127 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0025 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0028 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0063 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0293 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0033 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0133 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0208 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 20 "-" "-" "-" 0.0052 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0045 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 26 "-" "-" "-" 0.0042 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 94 "-" "-" "-" 0.0032 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 116 "-" "-" "-" 0.0036 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0240 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0014 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0026 "-" 3656197 1 test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0157 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3656197 - test.unit.account.test_server.TestAccountController.test_empty_policy_stats test.unit.account.test_server.TestAccountController.test_empty_policy_stats ... ok test.unit.account.test_server.TestAccountController.test_init test.unit.account.test_server.TestAccountController.test_init ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_false ... ok test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true test.unit.account.test_server.TestAccountController.test_insufficient_storage_mount_check_true ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist test.unit.account.test_server.TestAccountController.test_invalid_method_doesnt_exist ... ok test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public test.unit.account.test_server.TestAccountController.test_invalid_method_is_not_public ... ok test.unit.account.test_server.TestAccountController.test_list_allowed_methods test.unit.account.test_server.TestAccountController.test_list_allowed_methods ... ok test.unit.account.test_server.TestAccountController.test_log_line_format test.unit.account.test_server.TestAccountController.test_log_line_format ... ok test.unit.account.test_server.TestAccountController.test_markers_with_reserved test.unit.account.test_server.TestAccountController.test_markers_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use test.unit.account.test_server.TestAccountController.test_multiple_policies_in_use ... ok test.unit.account.test_server.TestAccountController.test_not_allowed_method test.unit.account.test_server.TestAccountController.test_not_allowed_method ... ok test.unit.account.test_server.TestAccountController.test_null_markers test.unit.account.test_server.TestAccountController.test_null_markers ... ok test.unit.account.test_server.TestAccountController.test_params_format test.unit.account.test_server.TestAccountController.test_params_format ... ok test.unit.account.test_server.TestAccountController.test_params_utf8 test.unit.account.test_server.TestAccountController.test_params_utf8 ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_non_default test.unit.account.test_server.TestAccountController.test_policy_stats_non_default ... ok test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy test.unit.account.test_server.TestAccountController.test_policy_stats_with_legacy ... ok test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_and_delim_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_prefix_with_reserved test.unit.account.test_server.TestAccountController.test_prefix_with_reserved ... ok test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods test.unit.account.test_server.TestAccountController.test_replicaiton_server_call_all_methods ... ok test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved test.unit.account.test_server.TestAccountController.test_reserved_markers_with_non_reserved ... ok test.unit.account.test_server.TestAccountController.test_serv_reserv test.unit.account.test_server.TestAccountController.test_serv_reserv ... ok test.unit.account.test_server.TestAccountController.test_through_call test.unit.account.test_server.TestAccountController.test_through_call ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path test.unit.account.test_server.TestAccountController.test_through_call_invalid_path ... ok test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestAccountController.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_is_deleted ... ok proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx89d0b4101ae540328e85e-00665ca3d4) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx062f6acf9daf41bc864a9-00665ca3d4) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpheammquw/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpheammquw/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpheammquw/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpheammquw/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0162 "-" 3656269 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0658 "-" 3656269 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0198 "-" 3656269 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0195 "-" 3656269 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx7b7933d6dbb84330a25bf-00665ca3d4" "proxy-server 3656269" 0.0027 "-" 3656269 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:39789/sda1/1/a/c" "tx7b7933d6dbb84330a25bf-00665ca3d4" "container-server 3656269" 0.0025 "-" 3656269 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:55867/sdb1/1/a/c" "tx7b7933d6dbb84330a25bf-00665ca3d4" "container-server 3656269" 0.0029 "-" 3656269 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx7b7933d6dbb84330a25bf-00665ca3d4" "proxy-server 3656269" 0.0338 "-" 3656269 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx7b7933d6dbb84330a25bf-00665ca3d4" "proxy-server 3656269" 0.0547 "-" 3656269 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/44 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx7b7933d6dbb84330a25bf-00665ca3d4 - 0.0723 - - 1717347284.531186104 1717347284.603486061 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx1e9d406e32d04d29bc16e-00665ca3d4" "proxy-server 3656269" 0.0026 "-" 3656269 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:55867/sdb1/1/a1/c1" "tx1e9d406e32d04d29bc16e-00665ca3d4" "container-server 3656269" 0.0017 "-" 3656269 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:39789/sda1/1/a1/c1" "tx1e9d406e32d04d29bc16e-00665ca3d4" "container-server 3656269" 0.0015 "-" 3656269 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1e9d406e32d04d29bc16e-00665ca3d4" "proxy-server 3656269" 0.1385 "-" 3656269 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1e9d406e32d04d29bc16e-00665ca3d4" "proxy-server 3656269" 0.1115 "-" 3656269 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/44 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx1e9d406e32d04d29bc16e-00665ca3d4 - 0.1555 - - 1717347284.606867790 1717347284.762335062 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx644f8174615d4685a6d08-00665ca3d4" "proxy-server 3656269" 0.0124 "-" 3656269 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:39789/sda1/2/a/c1" "tx644f8174615d4685a6d08-00665ca3d4" "container-server 3656269" 0.0134 "-" 3656269 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:55867/sdb1/2/a/c1" "tx644f8174615d4685a6d08-00665ca3d4" "container-server 3656269" 0.0023 "-" 3656269 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx644f8174615d4685a6d08-00665ca3d4" "proxy-server 3656269" 0.0397 "-" 3656269 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx644f8174615d4685a6d08-00665ca3d4" "proxy-server 3656269" 0.0619 "-" 3656269 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/44 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx644f8174615d4685a6d08-00665ca3d4 - 0.0865 - - 1717347284.764940023 1717347284.851437569 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx2538eac50fa246edbc748-00665ca3d4" "proxy-server 3656269" 0.0140 "-" 3656269 - test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit test.unit.proxy.test_mem_server.TestProxyServer.test_adds_transaction_id_length_limit ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_allow ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny test.unit.proxy.test_mem_server.TestProxyServer.test_calls_authorize_deny ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_200 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_503 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 test.unit.proxy.test_mem_server.TestProxyServer.test_check_response_507 ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header test.unit.proxy.test_mem_server.TestProxyServer.test_denied_host_header ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods test.unit.proxy.test_mem_server.TestProxyServer.test_error_limit_methods ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_error_occurred ... ok test DEBUG: /tmp/tmpskyhyu_f/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 2 (0.016608s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, 'da3ed2b1aa16d584db7b344f718316fa', 'f6f958af-69f7-4c0f-a583-172fc6cc84c0-sda', '1717347266.55633', '1717347266.00000', '0', '', '1717347266.00000', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f6f958af-69f7-4c0f-a583-172fc6cc84c0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpskyhyu_f/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15990 seconds (6.25394/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpskyhyu_f/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57da64f7508604df7c024d35e0f2df33', 'd2932c7f-dd42-4583-afa3-85994c271858-sda', '1717347266.75938', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d2932c7f-dd42-4583-afa3-85994c271858-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57da64f7508604df7c024d35e0f2df33', 'd2932c7f-dd42-4583-afa3-85994c271858-sda', '1717347266.75938', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d2932c7f-dd42-4583-afa3-85994c271858-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpskyhyu_f/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '83b5dc33eae0f7cc7e88a51377b0655c', 'd2d6d217-2c28-4e90-a1f9-7c322cdcf265-sdb', '1717347266.55633', '1717347267.00000', '0', '', '1717347267.00000', 1, 1) test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row -1 REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-unu', 'created_at': '1717347270.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o-nulo', 'created_at': '1717347271.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'd2d6d217-2c28-4e90-a1f9-7c322cdcf265-sdb') test DEBUG: Replicating db /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57da64f7508604df7c024d35e0f2df33', 'c2c7da03-18cf-434e-849d-46c450215bd6-', '1717347266.75938', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.2:1002/sdc, nothing to do REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '57da64f7508604df7c024d35e0f2df33', 'c2c7da03-18cf-434e-849d-46c450215bd6-', '1717347266.75938', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db in sync with 10.0.0.0:1000/sda, nothing to do test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.0499501s) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': 'f6f958af-69f7-4c0f-a583-172fc6cc84c0-sda', 'sync_point': 2}],) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (2, '83b5dc33eae0f7cc7e88a51377b0655c', 'd2d6d217-2c28-4e90-a1f9-7c322cdcf265-sdb', '1717347266.55633', '1717347267.00000', '0', '', '1717347267.00000', 1, 1) test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, starting at row -1 REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o-unu', 'created_at': '1717347270.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}, {'ROWID': 2, 'name': 'o-nulo', 'created_at': '1717347271.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'd2d6d217-2c28-4e90-a1f9-7c322cdcf265-sdb') test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.033973s) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([{'remote_id': 'f6f958af-69f7-4c0f-a583-172fc6cc84c0-sda', 'sync_point': 2}],) test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.24607 seconds (8.12769/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:2 diff_capped:0 empty:0 hashmatch:0 no_change:2 remote_merge:0 rsync:0 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, 'f7f8bab4b20d02d55ea2bcbb5a167d1c', 'c2c7da03-18cf-434e-849d-46c450215bd6-', '1717347266.75938', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, starting at row 1 REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '0:/a/c/o-nulo', 'created_at': '1717347271.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717347271.00000', 'deleted': 0, 'storage_policy_index': 0}], 'c2c7da03-18cf-434e-849d-46c450215bd6-') test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.2:1002/sdc, finished at row 2 (0.023231s) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': 'd2932c7f-dd42-4583-afa3-85994c271858-sda', 'sync_point': 1}],) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (2, 'f7f8bab4b20d02d55ea2bcbb5a167d1c', 'c2c7da03-18cf-434e-849d-46c450215bd6-', '1717347266.75938', '1717344000', '0', '', '1717344000', 1, 0) test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, starting at row 1 REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_items, ([{'ROWID': 2, 'name': '0:/a/c/o-nulo', 'created_at': '1717347271.00000', 'size': 0, 'content_type': 'application/x-put', 'etag': '1717347271.00000', 'deleted': 0, 'storage_policy_index': 0}], 'c2c7da03-18cf-434e-849d-46c450215bd6-') test DEBUG: /tmp/tmpskyhyu_f/sdb/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db usyncing chunks to 10.0.0.0:1000/sda, finished at row 2 (0.0186479s) REPLICATE: /sda/0/8f3afd7cc0a55bbf641992e992c0b16f, merge_syncs, ([{'remote_id': 'd2932c7f-dd42-4583-afa3-85994c271858-sda', 'sync_point': 1}],) test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharded_large_diff ... ok test DEBUG: checking permission: bucket7 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket8 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: checking permission: bucket9 None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:othertest:other test DEBUG: permission denined: 403 Forbidden test:tester OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) user quit No .MockServer object at 0x7fca36a156d0> running .MockServerFactory.MockServer object at 0x7fca36a15210> (1) appears to have stopped No .MockServerFactory.MockServer object at 0x7fca368cecd0> running No .MockServerFactory.MockServer object at 0x7fca368ce9d0> running Waited 15 seconds for .MockServerFactory.MockServer object at 0x7fca368cf950> to die; giving up Waited 15 seconds for .MockServerFactory.MockServer object at 0x7fca368ce9d0> to die; killing Signal .MockServerFactory.MockServer object at 0x7fca368ce9d0> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7fca368cf290> to die; killing Signal .MockServerFactory.MockServer object at 0x7fca368cf290> pid: None signal: 9 Waited 15 seconds for .MockServerFactory.MockServer object at 0x7fca368cfa50> to die; killing Signal .MockServerFactory.MockServer object at 0x7fca368cfa50> pid: None signal: 9 test1 STDOUT: test2 STDERR: test4 test5 STDERR: test6 test8 STDOUT: a#012b#012c STDOUT: d test.unit.common.test_utils.TestUtils.test_LoggerFileObject test.unit.common.test_utils.TestUtils.test_LoggerFileObject ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range ... ok test INFO: - - - [02/Jun/2024:16:54:33 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0015 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0015 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0001 "-" 3656221 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.4548 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0045 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0170 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0168 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0169 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0147 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0126 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0279 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0035 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0211 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/c_zero" 201 - "-" "-" "-" 0.0061 "-" 3656221 0 test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0159 "-" 3656221 1 test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0126 "-" 3656221 2 test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0185 "-" 3656221 3 test INFO: - - - [02/Jun/2024:16:54:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0160 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0166 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0154 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:34 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0142 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0216 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0037 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0032 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0254 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0026 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0004 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0001 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0002 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0189 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0025 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0063 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0026 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0234 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0026 "-" 3656221 0 test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0154 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0029 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0214 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0188 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0020 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0188 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0081 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0188 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0170 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0142 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0156 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0033 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0164 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0021 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0130 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0140 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0149 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0138 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0026 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0190 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0104 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0248 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0194 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0028 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0028 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0027 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3656221 - test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata test.unit.account.test_server.TestNonLegacyDefaultStoragePolicy.test_utf8_metadata ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_analyze_stdin_with_overlaps ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_all_donors_shrink_to_root ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_bad_args ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_donors_but_no_suitable_acceptor ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_expansion_limit_less_than_shrink_threshold ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_four_donors_two_acceptors ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_expanding ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_max_shrinking_default ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_no_gaps ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_root ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_not_sharded ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_nothing_to_do ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_overlapping_shard_ranges ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shard_ranges_in_found_state ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_shrink_threshold_with_tombstones ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_single_donor_shrink_to_root ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_compact_user_input ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_deprecated_options ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_does_not_exist ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_invalid_deprecated_options ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_conf_file_options ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_dry_run_and_yes_is_invalid ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_enable ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_replace_enable ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_find_shard_ranges_with_minimum_size ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_info ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_fills_gap ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_gap ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_merge_warns_of_overlap ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_children_overlaps_with_parent_donor ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_sharding_state ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_exits_if_shrinking_state ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_complete_sequence ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_multiple_missing ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_not_root ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_gaps_with_overlap ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_no_shard_ranges ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_needed ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_not_root ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_older_overlapping_donor_and_acceptor_shards ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_complete_sequences_one_incomplete ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_one_incomplete_sequence ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_donor_and_acceptor_shards_default ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_overlapping_incomplete_sequences ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_parent_overlaps_with_children_donors ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_two_complete_sequences_one_incomplete ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_acceptor_with_overlapping_donor_shards ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_repair_younger_overlapping_donor_shards ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_replace ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_show_and_analyze ... ok test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required test.unit.cli.test_manage_shard_ranges.TestManageShardRanges.test_subcommand_required ... ok test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit test.unit.cli.test_recon.TestHelpers.test_seconds2timeunit ... ok test.unit.cli.test_recon.TestHelpers.test_size_suffix test.unit.cli.test_recon.TestHelpers.test_size_suffix ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_container_key_for_default_required_list ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_bad_object_key_for_default_required_list ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_callback_exception ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_callback ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_container_key_for_specified_required_list ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_key_for_default_required_list ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_missing_object_key_for_specified_required_list ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_not_a_dict ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_keys_with_crypto_meta ... ok test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys test.unit.common.middleware.crypto.test_crypto_utils.TestCryptoWsgiContext.test_get_multiple_keys ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_GET ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_bucket_logging_PUT ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_GET_error ... ok test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error test.unit.common.middleware.s3api.test_logging.TestS3ApiLogging.test_object_logging_PUT_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_empty_body ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_lots_of_keys ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_no_key ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_quiet ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_to_object ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_too_many_keys ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_unhandled_exception ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_versioned_suspended ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_with_system_entity ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_canned_acl_public_read_write ... ok cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx27b0ff85b37e48bc9e043-00665ca3d8" "proxy-server 3656212" 0.0543 "-" 3656212 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:51 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx27b0ff85b37e48bc9e043-00665ca3d8" "proxy-server 3656212" 2.4574 "-" 3656212 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/51 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx27b0ff85b37e48bc9e043-00665ca3d8 - 2.4933 - - 1717347288.526509285 1717347291.019843340 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:51 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx2532a8136f7f42b9bbfe4-00665ca3db" "proxy-server 3656212" 0.0023 "-" 3656212 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:52 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:35979/sda1/1/a1/c1" "tx2532a8136f7f42b9bbfe4-00665ca3db" "container-server 3656212" 0.1328 "-" 3656212 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:52 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:45671/sdb1/1/a1/c1" "tx2532a8136f7f42b9bbfe4-00665ca3db" "container-server 3656212" 0.0022 "-" 3656212 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:52 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2532a8136f7f42b9bbfe4-00665ca3db" "proxy-server 3656212" 1.6426 "-" 3656212 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:52 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2532a8136f7f42b9bbfe4-00665ca3db" "proxy-server 3656212" 1.0183 "-" 3656212 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/52 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx2532a8136f7f42b9bbfe4-00665ca3db - 1.6571 - - 1717347291.022012234 1717347292.679122448 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:54 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd32d16736ea841128fd02-00665ca3dc" "proxy-server 3656212" 1.3766 "-" 3656212 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:55 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:45671/sdb1/2/a/c1" "txd32d16736ea841128fd02-00665ca3dc" "container-server 3656212" 0.0017 "-" 3656212 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:56 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:35979/sda1/2/a/c1" "txd32d16736ea841128fd02-00665ca3dc" "container-server 3656212" 0.9571 "-" 3656212 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:56 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd32d16736ea841128fd02-00665ca3dc" "proxy-server 3656212" 1.8185 "-" 3656212 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:56 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txd32d16736ea841128fd02-00665ca3dc" "proxy-server 3656212" 2.5648 "-" 3656212 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/56 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txd32d16736ea841128fd02-00665ca3dc - 3.9527 - - 1717347292.681165218 1717347296.633829594 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:57 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf2b9b0170f95471bbaf56-00665ca3e0" "proxy-server 3656212" 0.8657 "-" 3656212 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:59 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:45671/sdb1/1/a/c2" "txf2b9b0170f95471bbaf56-00665ca3e0" "container-server 3656212" 0.0020 "-" 3656212 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:59 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:35979/sda1/1/a/c2" "txf2b9b0170f95471bbaf56-00665ca3e0" "container-server 3656212" 0.5917 "-" 3656212 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:59 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf2b9b0170f95471bbaf56-00665ca3e0" "proxy-server 3656212" 2.4712 "-" 3656212 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:59 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf2b9b0170f95471bbaf56-00665ca3e0" "proxy-server 3656212" 1.4177 "-" 3656212 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/59 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txf2b9b0170f95471bbaf56-00665ca3e0 - 3.3502 - - 1717347296.639126062 1717347299.989358664 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf6bb5245815a4528ba417-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx2588619a14ea4528a2737-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: txd4134ef926b14f3b9c1b1-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: txd4134ef926b14f3b9c1b1-00665ca3e4) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txd4134ef926b14f3b9c1b1-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txc0f2472934c14a73bd48f-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txc0f2472934c14a73bd48f-00665ca3e4) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txc0f2472934c14a73bd48f-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx720a850f6e514ce9b2585-00665ca3e4) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx720a850f6e514ce9b2585-00665ca3e4) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx68d975d190f746d5a7949-00665ca3e4) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx68d975d190f746d5a7949-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content test.unit.proxy.test_mem_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_mem_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST test.unit.proxy.test_mem_server.TestContainerController.test_POST ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_POST_calls_clean_acl ... ok test ERROR: ERROR Could not get container info /tmp/tmpj8jaz_eu/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit pass completed: 1.08s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit pass. test ERROR: ERROR auditing: Timeout (Nones) test INFO: Container audit pass completed: 0.00s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container audit "once" mode test DEBUG: Audit passed for test.unit.common.test_db_auditor.TestAuditor.test_run_once test.unit.common.test_db_auditor.TestAuditor.test_run_once ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers test.unit.common.test_db_replicator.TestReplicatorSync.test_clean_up_after_deleted_brokers ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids test.unit.common.test_db_replicator.TestReplicatorSync.test_local_ids ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync test.unit.common.test_db_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.common.test_db_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.common.test_utils.TestContextPool.test_close test.unit.common.test_utils.TestContextPool.test_close ... ok test.unit.common.test_utils.TestContextPool.test_context_manager test.unit.common.test_utils.TestContextPool.test_context_manager ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_at_end_of_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile test.unit.common.test_utils.TestGreenAsyncPile.test_exceptions_in_streaming_pile ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous test.unit.common.test_utils.TestGreenAsyncPile.test_is_asynchronous ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty test.unit.common.test_utils.TestGreenAsyncPile.test_next_when_empty ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes test.unit.common.test_utils.TestGreenAsyncPile.test_no_blocking_last_next_explodes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_pending test.unit.common.test_utils.TestGreenAsyncPile.test_pending ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything test.unit.common.test_utils.TestGreenAsyncPile.test_runs_everything ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn test.unit.common.test_utils.TestGreenAsyncPile.test_wait_with_firstn ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_completes ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout test.unit.common.test_utils.TestGreenAsyncPile.test_waitall_timeout_timesout ... ok test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first test.unit.common.test_utils.TestGreenAsyncPile.test_waitfirst_only_returns_first ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit test.unit.common.test_wsgi.TestWorkersStrategy.test_log_sock_exit ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout test.unit.common.test_wsgi.TestWorkersStrategy.test_loop_timeout ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestWorkersStrategy.test_new_worker_socks ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestWorkersStrategy.test_no_fork_sock ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestWorkersStrategy.test_post_fork_hook ... ok test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestWorkersStrategy.test_shutdown_sockets ... ok test.unit.container.test_sharder.TestCleavingContext.test_cursor test.unit.container.test_sharder.TestCleavingContext.test_cursor ... ok test.unit.container.test_sharder.TestCleavingContext.test_delete test.unit.container.test_sharder.TestCleavingContext.test_delete ... ok test.unit.container.test_sharder.TestCleavingContext.test_done test.unit.container.test_sharder.TestCleavingContext.test_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_init test.unit.container.test_sharder.TestCleavingContext.test_init ... ok test.unit.container.test_sharder.TestCleavingContext.test_iter test.unit.container.test_sharder.TestCleavingContext.test_iter ... ok test.unit.container.test_sharder.TestCleavingContext.test_load test.unit.container.test_sharder.TestCleavingContext.test_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_all test.unit.container.test_sharder.TestCleavingContext.test_load_all ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_load_modify_store_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_range_done test.unit.container.test_sharder.TestCleavingContext.test_range_done ... ok test.unit.container.test_sharder.TestCleavingContext.test_reset test.unit.container.test_sharder.TestCleavingContext.test_reset ... ok test.unit.container.test_sharder.TestCleavingContext.test_start test.unit.container.test_sharder.TestCleavingContext.test_start ... ok test.unit.container.test_sharder.TestCleavingContext.test_store test.unit.container.test_sharder.TestCleavingContext.test_store ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_add_row_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_modify_db_id_load_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_old_style ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load ... ok test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style test.unit.container.test_sharder.TestCleavingContext.test_store_reclaim_load_old_style ... ok test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync test.unit.obj.test_ssync.TestSsyncReplication.test_content_type_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object test.unit.obj.test_ssync.TestSsyncReplication.test_expired_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_not_synced_to_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync test.unit.obj.test_ssync.TestSsyncReplication.test_meta_file_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta test.unit.obj.test_ssync.TestSsyncReplication.test_no_longer_expired_object_syncs_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync test.unit.obj.test_ssync.TestSsyncReplication.test_nothing_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncReplication.test_sync test.unit.obj.test_ssync.TestSsyncReplication.test_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_empty_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_exact ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_have_one_newer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_durable_older_than_remote_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_local_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_durable_but_commit_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_missing_meta_expired_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_MISSING_CHECK_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_bad_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_default_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_invalid_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_matched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_mismatched_indexes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_only_node_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header test.unit.obj.test_ssync_receiver.TestReceiver.test_Receiver_with_storage_policy_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_Exception_Exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_calls_replication_lock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_initial_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_mount_check_isdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_replication_lock_per_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked test.unit.obj.test_ssync_receiver.TestReceiver.test_SSYNC_semaphore_locked ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_BONK ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_PUT_with_storage_policy_and_node_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_bad_subrequest_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_content_length_with_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_early_termination ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_failures ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_multiple ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_content_length_with_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_problems_no_hard_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_no_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_other_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_subreq_does_not_read_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy test.unit.obj.test_ssync_receiver.TestReceiver.test_UPDATES_with_storage_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordTypeLegacy.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': '9Z+cE+YYsVYXLtR89GLUSA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'f59f9c13e618b156172ed47cf462d448', 'Content-Length': '88'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_unhandled_exception ... ok I like ham STDERR: --- Logging error --- STDERR: Traceback (most recent call last): STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 845, in emit raise socket.error(errno.ENOTCONN, "This is an ex-syslog") STDERR: OSError: [Errno 107] This is an ex-syslog STDERR: Call stack: STDERR: File "", line 198, in _run_module_as_main STDERR: File "", line 88, in _run_code STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 88, in main() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 82, in main program.TestProgram( STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 197, in __init__ self.runTests() STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/program.py", line 247, in runTests self.result = testRunner.run(self.test) STDERR: File "/usr/lib/python3/dist-packages/stestr/subunit_runner/run.py", line 51, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/suite.py", line 122, in run test(result) STDERR: File "/usr/lib/python3.11/unittest/case.py", line 678, in __call__ return self.run(*args, **kwds) STDERR: File "/usr/lib/python3.11/unittest/case.py", line 623, in run self._callTestMethod(testMethod) STDERR: File "/usr/lib/python3.11/unittest/case.py", line 579, in _callTestMethod if method() is not None: STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 877, in test_LoggerFileObject_recursion logger.info("I like ham") STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1489, in info self._log(INFO, msg, args, **kwargs) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1634, in _log self.handle(record) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1644, in handle self.callHandlers(record) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 1706, in callHandlers hdlr.handle(record) STDERR: File "/usr/lib/python3.11/logging/__init__.py", line 978, in handle self.emit(record) STDERR: File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_utils.py", line 847, in emit self.handleError(record) STDERR: Message: 'I like ham' Arguments: () test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion test.unit.common.test_utils.TestUtils.test_LoggerFileObject_recursion ... ok test.unit.common.test_utils.TestUtils.test_NullLogger test.unit.common.test_utils.TestUtils.test_NullLogger ... ok test.unit.common.test_utils.TestUtils.test_TRUE_VALUES test.unit.common.test_utils.TestUtils.test_TRUE_VALUES ... ok test.unit.common.test_utils.TestUtils.test_backwards test.unit.common.test_utils.TestUtils.test_backwards ... ok test.unit.common.test_utils.TestUtils.test_cache_from_env test.unit.common.test_utils.TestUtils.test_cache_from_env ... ok test.unit.common.test_utils.TestUtils.test_cap_length test.unit.common.test_utils.TestUtils.test_cap_length ... ok test.unit.common.test_utils.TestUtils.test_capture_stdio test.unit.common.test_utils.TestUtils.test_capture_stdio ... ok test.unit.common.test_utils.TestUtils.test_clean_content_type test.unit.common.test_utils.TestUtils.test_clean_content_type ... ok test.unit.common.test_utils.TestUtils.test_clean_logger_exception test.unit.common.test_utils.TestUtils.test_clean_logger_exception ... ok test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene test.unit.common.test_utils.TestUtils.test_clean_up_daemon_hygene ... ok test.unit.common.test_utils.TestUtils.test_config_auto_int_value test.unit.common.test_utils.TestUtils.test_config_auto_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_fallocate_value test.unit.common.test_utils.TestUtils.test_config_fallocate_value ... ok test.unit.common.test_utils.TestUtils.test_config_float_value test.unit.common.test_utils.TestUtils.test_config_float_value ... ok test.unit.common.test_utils.TestUtils.test_config_percent_value test.unit.common.test_utils.TestUtils.test_config_percent_value ... ok test.unit.common.test_utils.TestUtils.test_config_positive_int_value test.unit.common.test_utils.TestUtils.test_config_positive_int_value ... ok test.unit.common.test_utils.TestUtils.test_config_request_node_count_value test.unit.common.test_utils.TestUtils.test_config_request_node_count_value ... ok test.unit.common.test_utils.TestUtils.test_config_true_value test.unit.common.test_utils.TestUtils.test_config_true_value ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close test.unit.common.test_utils.TestUtils.test_drain_and_close ... ok test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit test.unit.common.test_utils.TestUtils.test_drain_and_close_with_limit ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges test.unit.common.test_utils.TestUtils.test_drop_privileges ... ok test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups test.unit.common.test_utils.TestUtils.test_drop_privileges_no_setgroups ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache test.unit.common.test_utils.TestUtils.test_dump_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied test.unit.common.test_utils.TestUtils.test_dump_recon_cache_permission_denied ... ok test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner test.unit.common.test_utils.TestUtils.test_dump_recon_cache_set_owner ... ok test.unit.common.test_utils.TestUtils.test_extract_swift_bytes test.unit.common.test_utils.TestUtils.test_extract_swift_bytes ... ok test.unit.common.test_utils.TestUtils.test_find_namespace test.unit.common.test_utils.TestUtils.test_find_namespace ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body test.unit.common.test_utils.TestUtils.test_friendly_close_exploding_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_large_body test.unit.common.test_utils.TestUtils.test_friendly_close_large_body ... ok test.unit.common.test_utils.TestUtils.test_friendly_close_small_body test.unit.common.test_utils.TestUtils.test_friendly_close_small_body ... ok test.unit.common.test_utils.TestUtils.test_fsync_dir test.unit.common.test_utils.TestUtils.test_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_generate_trans_id test.unit.common.test_utils.TestUtils.test_generate_trans_id ... ok test.unit.common.test_utils.TestUtils.test_get_db_files test.unit.common.test_utils.TestUtils.test_get_db_files ... ok test.unit.common.test_utils.TestUtils.test_get_log_line test.unit.common.test_utils.TestUtils.test_get_log_line ... ok test.unit.common.test_utils.TestUtils.test_get_logger test.unit.common.test_utils.TestUtils.test_get_logger ... ok test.unit.common.test_utils.TestUtils.test_get_logger_console test.unit.common.test_utils.TestUtils.test_get_logger_console ... ok test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route test.unit.common.test_utils.TestUtils.test_get_logger_name_and_route ... ok test.unit.common.test_utils.TestUtils.test_get_partition_for_hash test.unit.common.test_utils.TestUtils.test_get_partition_for_hash ... ok test.unit.common.test_utils.TestUtils.test_get_partition_from_path test.unit.common.test_utils.TestUtils.test_get_partition_from_path ... ok test.unit.common.test_utils.TestUtils.test_get_policy_index test.unit.common.test_utils.TestUtils.test_get_policy_index ... ok test.unit.common.test_utils.TestUtils.test_get_redirect_data test.unit.common.test_utils.TestUtils.test_get_redirect_data ... ok test.unit.common.test_utils.TestUtils.test_get_trans_id_time test.unit.common.test_utils.TestUtils.test_get_trans_id_time ... ok test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str test.unit.common.test_utils.TestUtils.test_get_valid_utf8_str ... ok test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string test.unit.common.test_utils.TestUtils.test_get_zero_indexed_base_string ... ok test.unit.common.test_utils.TestUtils.test_hash_path test.unit.common.test_utils.TestUtils.test_hash_path ... ok test.unit.common.test_utils.TestUtils.test_human_readable test.unit.common.test_utils.TestUtils.test_human_readable ... ok test.unit.common.test_utils.TestUtils.test_is_file_older test.unit.common.test_utils.TestUtils.test_is_file_older ... ok test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist test.unit.common.test_utils.TestUtils.test_ismount_path_does_not_exist ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_error test.unit.common.test_utils.TestUtils.test_ismount_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_root test.unit.common.test_utils.TestUtils.test_ismount_path_is_root ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink test.unit.common.test_utils.TestUtils.test_ismount_path_is_symlink ... ok test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount test.unit.common.test_utils.TestUtils.test_ismount_path_not_mount ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_parent_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error test.unit.common.test_utils.TestUtils.test_ismount_raw_path_error ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_dev test.unit.common.test_utils.TestUtils.test_ismount_successes_dev ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_ino test.unit.common.test_utils.TestUtils.test_ismount_successes_ino ... ok test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile test.unit.common.test_utils.TestUtils.test_ismount_successes_stubfile ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp ... ok test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC test.unit.common.test_utils.TestUtils.test_last_modified_date_to_timestamp_when_system_not_UTC ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT test.unit.common.test_utils.TestUtils.test_link_fd_to_path_errno_not_EEXIST_or_ENOENT ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success test.unit.common.test_utils.TestUtils.test_link_fd_to_path_linkat_success ... ok test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists test.unit.common.test_utils.TestUtils.test_link_fd_to_path_target_exists ... ok test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists test.unit.common.test_utils.TestUtils.test_linkat_race_dir_not_exists ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_no_stale_reads ... ok test ERROR: ERROR: swift.cache could not be found in env! test ERROR: ERROR: swift.cache could not be found in env! test WARNING: Unable to perform fsync() on directory /tmp/tmpdphq808w: Bad file descriptor Unable to locate some_not_real_function in libc. Leaving as a no-op. test.unit.common.test_utils.TestUtils.test_load_libc_function test.unit.common.test_utils.TestUtils.test_load_libc_function ... ok test.unit.common.test_utils.TestUtils.test_load_pkg_resource test.unit.common.test_utils.TestUtils.test_load_pkg_resource ... skipped 'pkg_resources loading is only available on python 3.7 and earlier' test.unit.common.test_utils.TestUtils.test_load_recon_cache test.unit.common.test_utils.TestUtils.test_load_recon_cache ... ok test.unit.common.test_utils.TestUtils.test_lock_file test.unit.common.test_utils.TestUtils.test_lock_file ... ok test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink test.unit.common.test_utils.TestUtils.test_lock_file_held_on_unlink ... ok test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail test.unit.common.test_utils.TestUtils.test_lock_file_no_unlink_if_fail ... ok test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open test.unit.common.test_utils.TestUtils.test_lock_file_unlinked_after_open ... ok test.unit.common.test_utils.TestUtils.test_lock_path test.unit.common.test_utils.TestUtils.test_lock_path ... ok test.unit.common.test_utils.TestUtils.test_lock_path_class test.unit.common.test_utils.TestUtils.test_lock_path_class ... ok test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit test.unit.common.test_utils.TestUtils.test_lock_path_invalid_limit ... ok test.unit.common.test_utils.TestUtils.test_lock_path_name test.unit.common.test_utils.TestUtils.test_lock_path_name ... ok test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps test.unit.common.test_utils.TestUtils.test_lock_path_num_sleeps ... ok test.unit.common.test_utils.TestUtils.test_log_string_formatter test.unit.common.test_utils.TestUtils.test_log_string_formatter ... ok test.unit.common.test_utils.TestUtils.test_majority_size test.unit.common.test_utils.TestUtils.test_majority_size ... ok test.unit.common.test_utils.TestUtils.test_make_db_file_path test.unit.common.test_utils.TestUtils.test_make_db_file_path ... ok test.unit.common.test_utils.TestUtils.test_makedirs_count test.unit.common.test_utils.TestUtils.test_makedirs_count ... ok test.unit.common.test_utils.TestUtils.test_md5_with_data test.unit.common.test_utils.TestUtils.test_md5_with_data ... ok test.unit.common.test_utils.TestUtils.test_md5_without_data test.unit.common.test_utils.TestUtils.test_md5_without_data ... ok test.unit.common.test_utils.TestUtils.test_mkdirs test.unit.common.test_utils.TestUtils.test_mkdirs ... ok test.unit.common.test_utils.TestUtils.test_monkey_patch test.unit.common.test_utils.TestUtils.test_monkey_patch ... ok test.unit.common.test_utils.TestUtils.test_nested_prefixlogger test.unit.common.test_utils.TestUtils.test_nested_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_node_to_string test.unit.common.test_utils.TestUtils.test_node_to_string ... ok test.unit.common.test_utils.TestUtils.test_non_negative_float test.unit.common.test_utils.TestUtils.test_non_negative_float ... ok test.unit.common.test_utils.TestUtils.test_non_negative_int test.unit.common.test_utils.TestUtils.test_non_negative_int ... ok test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error test.unit.common.test_utils.TestUtils.test_none_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp ... ok test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision test.unit.common.test_utils.TestUtils.test_normalize_delete_at_timestamp_high_precision ... ok test.unit.common.test_utils.TestUtils.test_normalize_timestamp test.unit.common.test_utils.TestUtils.test_normalize_timestamp ... ok test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type test.unit.common.test_utils.TestUtils.test_override_bytes_from_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_content_type test.unit.common.test_utils.TestUtils.test_parse_content_type ... ok test.unit.common.test_utils.TestUtils.test_parse_db_filename test.unit.common.test_utils.TestUtils.test_parse_db_filename ... ok test.unit.common.test_utils.TestUtils.test_parse_options test.unit.common.test_utils.TestUtils.test_parse_options ... ok test.unit.common.test_utils.TestUtils.test_parse_options_errors test.unit.common.test_utils.TestUtils.test_parse_options_errors ... ok test.unit.common.test_utils.TestUtils.test_parse_override_options test.unit.common.test_utils.TestUtils.test_parse_override_options ... ok test.unit.common.test_utils.TestUtils.test_prefixlogger test.unit.common.test_utils.TestUtils.test_prefixlogger ... ok test.unit.common.test_utils.TestUtils.test_quorum_size test.unit.common.test_utils.TestUtils.test_quorum_size ... ok test.unit.common.test_utils.TestUtils.test_quote_bytes test.unit.common.test_utils.TestUtils.test_quote_bytes ... ok test.unit.common.test_utils.TestUtils.test_quote_unicode test.unit.common.test_utils.TestUtils.test_quote_unicode ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_incr ... ok test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep test.unit.common.test_utils.TestUtils.test_ratelimit_sleep_with_sleep ... ok test.unit.common.test_utils.TestUtils.test_readconf test.unit.common.test_utils.TestUtils.test_readconf ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir test.unit.common.test_utils.TestUtils.test_readconf_dir ... ok test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files test.unit.common.test_utils.TestUtils.test_readconf_dir_ignores_hidden_and_nondotconf_files ... ok test.unit.common.test_utils.TestUtils.test_readconf_raw test.unit.common.test_utils.TestUtils.test_readconf_raw ... ok test.unit.common.test_utils.TestUtils.test_remove_directory test.unit.common.test_utils.TestUtils.test_remove_directory ... ok test.unit.common.test_utils.TestUtils.test_remove_file test.unit.common.test_utils.TestUtils.test_remove_file ... ok test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false test.unit.common.test_utils.TestUtils.test_renamer_when_fsync_is_false ... ok test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir test.unit.common.test_utils.TestUtils.test_renamer_with_fsync_dir ... ok test.unit.common.test_utils.TestUtils.test_replace_partition_in_path test.unit.common.test_utils.TestUtils.test_replace_partition_in_path ... ok test.unit.common.test_utils.TestUtils.test_round_robin_iter test.unit.common.test_utils.TestUtils.test_round_robin_iter ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv4_localhost ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_ipv4_compatible ... ok test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip test.unit.common.test_utils.TestUtils.test_rsync_ip_ipv6_random_ip ... ok test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation test.unit.common.test_utils.TestUtils.test_rsync_module_interpolation ... ok test.unit.common.test_utils.TestUtils.test_safe_json_loads test.unit.common.test_utils.TestUtils.test_safe_json_loads ... ok test.unit.common.test_utils.TestUtils.test_search_tree test.unit.common.test_utils.TestUtils.test_search_tree ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_pid_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match test.unit.common.test_utils.TestUtils.test_search_tree_conf_dir_with_named_conf_match ... ok test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match test.unit.common.test_utils.TestUtils.test_search_tree_with_directory_ext_match ... ok test.unit.common.test_utils.TestUtils.test_select_node_ip test.unit.common.test_utils.TestUtils.test_select_node_ip ... ok test.unit.common.test_utils.TestUtils.test_split_path test.unit.common.test_utils.TestUtils.test_split_path ... ok test.unit.common.test_utils.TestUtils.test_storage_directory test.unit.common.test_utils.TestUtils.test_storage_directory ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer test.unit.common.test_utils.TestUtils.test_str_anonymizer ... ok test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness test.unit.common.test_utils.TestUtils.test_str_anonymizer_python_maddness ... ok test.unit.common.test_utils.TestUtils.test_str_format_time test.unit.common.test_utils.TestUtils.test_str_format_time ... ok test.unit.common.test_utils.TestUtils.test_streq_const_time test.unit.common.test_utils.TestUtils.test_streq_const_time ... ok test.unit.common.test_utils.TestUtils.test_strict_b64decode test.unit.common.test_utils.TestUtils.test_strict_b64decode ... ok test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error test.unit.common.test_utils.TestUtils.test_string_data_raises_type_error ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter test.unit.common.test_utils.TestUtils.test_swift_log_formatter ... ok test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length test.unit.common.test_utils.TestUtils.test_swift_log_formatter_max_line_length ... ok test.unit.common.test_utils.TestUtils.test_systemd_notify test.unit.common.test_utils.TestUtils.test_systemd_notify ... ok test.unit.common.test_utils.TestUtils.test_validate_device_partition test.unit.common.test_utils.TestUtils.test_validate_device_partition ... ok test.unit.common.test_utils.TestUtils.test_validate_hash_conf test.unit.common.test_utils.TestUtils.test_validate_hash_conf ... ok test.unit.common.test_utils.TestUtils.test_validate_sync_to test.unit.common.test_utils.TestUtils.test_validate_sync_to ... ok test.unit.common.test_utils.TestUtils.test_write_file test.unit.common.test_utils.TestUtils.test_write_file ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout test.unit.common.test_wsgi.TestServersPerPortStrategy.test_loop_timeout ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks test.unit.common.test_wsgi.TestServersPerPortStrategy.test_new_worker_socks ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock test.unit.common.test_wsgi.TestServersPerPortStrategy.test_no_fork_sock ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook test.unit.common.test_wsgi.TestServersPerPortStrategy.test_post_fork_hook ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container test.unit.common.test_wsgi.TestServersPerPortStrategy.test_servers_per_port_in_container ... ok test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets test.unit.common.test_wsgi.TestServersPerPortStrategy.test_shutdown_sockets ... ok test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined test.unit.obj.test_auditor.TestAuditor.test_audit_location_gets_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_auditor_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_expired_object_is_ignored_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_failsafe_object_audit_will_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling test.unit.obj.test_auditor.TestAuditor.test_generic_exception_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up test.unit.obj.test_auditor.TestAuditor.test_inprogress_rsync_tempfiles_get_cleaned_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_non_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums test.unit.obj.test_auditor.TestAuditor.test_object_audit_adds_metadata_checksums ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments test.unit.obj.test_auditor.TestAuditor.test_object_audit_checks_EC_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_diff_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data test.unit.obj.test_auditor.TestAuditor.test_object_audit_extra_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta test.unit.obj.test_auditor.TestAuditor.test_object_audit_no_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests test.unit.obj.test_auditor.TestAuditor.test_object_audit_will_not_swallow_errors_in_tests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_non_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed test.unit.obj.test_auditor.TestAuditor.test_object_run_fast_track_zero_check_closed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_logging test.unit.obj.test_auditor.TestAuditor.test_object_run_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices test.unit.obj.test_auditor.TestAuditor.test_object_run_once_multi_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda test.unit.obj.test_auditor.TestAuditor.test_object_run_once_no_sda ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass test.unit.obj.test_auditor.TestAuditor.test_object_run_once_pass ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache test.unit.obj.test_auditor.TestAuditor.test_object_run_recon_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_custom_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps test.unit.obj.test_auditor.TestAuditor.test_reclaimable_tombstone_with_zero_byte_fps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option test.unit.obj.test_auditor.TestAuditor.test_rsync_tempfile_timeout_auto_option ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps test.unit.obj.test_auditor.TestAuditor.test_run_audit_once_zbfps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork test.unit.obj.test_auditor.TestAuditor.test_run_parallel_audit_once_failed_fork ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_sleeper test.unit.obj.test_auditor.TestAuditor.test_sleeper ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location test.unit.obj.test_auditor.TestAuditor.test_with_inaccessible_object_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone test.unit.obj.test_auditor.TestAuditor.test_with_only_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data test.unit.obj.test_auditor.TestAuditor.test_with_tombstone_and_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms test.unit.obj.test_auditor.TestAuditor.test_worker_conf_parms ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_replication_info ... ok test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_duplicate_frags_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructorECDuplicationFactor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchiveUTF8.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_exception_during_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_rebuilt_content ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_no_responses ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_partial_rebuild ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_quarantines_lonely_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok test.unit.obj.test_ssync.TestSsyncECReconstructorSyncJob.test_sync_reconstructor_rebuild_ok ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_bad_response_body ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix test.unit.proxy.test_server.TestProxyServer.test_statsd_prefix ... ok test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_server.TestProxyServer.test_valid_api_version ... ok test DEBUG: Audit passed for test ERROR: ERROR Could not get container info /tmp/tmp9ii2lypj/tmp_test_database_auditor/fail2.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test ERROR: ERROR Could not get container info /tmp/tmp9ii2lypj/tmp_test_database_auditor/fail1.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/db_auditor.py", line 140, in audit info = broker.get_info() ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_db_auditor.py", line 40, in get_info raise ValueError ValueError test DEBUG: Audit passed for test INFO: Container audit "once" mode completed: 7.48s test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00044 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpj4k8ixga/sda/fake/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.00477 seconds (209.53270/s) test INFO: Removed 1 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00146 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00067 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test INFO: Beginning replication run test INFO: Replication run OVER test INFO: Attempted to replicate 0 dbs in 0.00075 seconds (0.00000/s) test INFO: Removed 0 dbs test INFO: 0 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 test NOTICE: Child 3656228 exiting normally test NOTICE: Started child 88 from parent 3656228 test NOTICE: Started child 89 from parent 3656228 test ERROR: Removing dead child 88 from parent 3656228 test NOTICE: Started child 90 from parent 3656228 test NOTICE: Started child 88 from parent 3656228 test NOTICE: Started child 89 from parent 3656228 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sdr (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sdt (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sdx (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sds (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sdo (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sdv (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sdz (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sdp (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server ERROR: Object returning 503 for [] (txn: tx4ea20189a45c49e7b5d6b-00665ca3da) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from primary (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplicate_but_sufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_duplication_factor_almost_duplicate_dispersion ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '9d7355a13f14e34c263530e91e46a52b', '06c33a69-efad-46a6-83f0-6e0f7384d020-', '1717347267.32348', '1717347267.31307', '0', '', '1717347267.31307', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('06c33a69-efad-46a6-83f0-6e0f7384d020-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.34454.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '9d7355a13f14e34c263530e91e46a52b', '06c33a69-efad-46a6-83f0-6e0f7384d020-', '1717347267.32348', '1717347267.31307', '0', '', '1717347267.31307', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347267.34436', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347267.34436', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347267.34436', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347267.34476', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347267.34476', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347267.34454', 'epoch': '1717347267.34454', 'reported': 0, 'tombstones': -1}], '06c33a69-efad-46a6-83f0-6e0f7384d020-') test DEBUG: /tmp/tmpm_3yuy50/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.34454.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpm_3yuy50/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.34454.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'd64e054f416e31edfebf4570c4f3e73f', 'c80d9e44-ce7d-4e2d-bfe2-1190ba7f70ac-', '1717347267.54060', '1717347267.53484', '0', '', '1717347267.53484', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347267.66808', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347267.66808', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347267.66808', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347267.67996', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347267.67996', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347267.55665', 'epoch': '1717347267.55665', 'reported': 0, 'tombstones': -1}], 'c80d9e44-ce7d-4e2d-bfe2-1190ba7f70ac-') test DEBUG: /tmp/tmp7w1q311q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.55665.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp7w1q311q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.55665.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'd64e054f416e31edfebf4570c4f3e73f', 'c80d9e44-ce7d-4e2d-bfe2-1190ba7f70ac-', '1717347267.54060', '1717347267.64965', '0', '', '1717347267.53484', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347267.66808', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347267.66808', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347267.66808', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347267.67996', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347267.67996', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347267.55665', 'epoch': '1717347267.55665', 'reported': 0, 'tombstones': -1}], 'c80d9e44-ce7d-4e2d-bfe2-1190ba7f70ac-') test DEBUG: /tmp/tmp7w1q311q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.55665.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp7w1q311q/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.55665.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'd64e054f416e31edfebf4570c4f3e73f', '5c629c21-46ed-405a-8ec2-408a1346007a-', '1717347267.93884', '1717347267.93269', '0', '', '1717347267.93269', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347268.07712', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347268.07712', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347268.07712', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347268.08800', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347268.08800', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347267.94802', 'epoch': '1717347267.94802', 'reported': 0, 'tombstones': -1}], '5c629c21-46ed-405a-8ec2-408a1346007a-') test DEBUG: /tmp/tmpu_sq2qw4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.94802.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpu_sq2qw4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.94802.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, 'd64e054f416e31edfebf4570c4f3e73f', '5c629c21-46ed-405a-8ec2-408a1346007a-', '1717347267.93884', '1717347268.05968', '0', '', '1717347267.93269', 101, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347268.07712', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347268.07712', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347268.07712', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347268.08800', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347268.08800', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347267.94802', 'epoch': '1717347267.94802', 'reported': 0, 'tombstones': -1}], '5c629c21-46ed-405a-8ec2-408a1346007a-') test DEBUG: /tmp/tmpu_sq2qw4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.94802.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpu_sq2qw4/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347267.94802.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '278f2c439ef4bb62fc37db9420dfc37e', '34e0bbe4-2193-4d29-8273-99a86ccbd29a-', '1717347268.28154', '1717347268.27684', '0', '', '1717347268.27684', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_sharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test INFO: - - - [02/Jun/2024:16:54:34 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "method_doesnt_exist /sda1/p/a" 405 - "-" "-" "-" 0.0048 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "__init__ /sda1/p/a" 405 - "-" "-" "-" 0.0002 "-" 3656197 - test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a" 404 - "-" "-" "-" 2.0000 "-" 1234 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0290 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0024 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0172 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0164 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0027 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 109 "-" "-" "-" 0.0030 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0175 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0020 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0139 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test03" 201 - "-" "-" "-" 0.0163 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 219 "-" "-" "-" 0.0167 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 110 "-" "-" "-" 0.0028 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0209 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0025 "-" 3656197 0 test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0102 "-" 3656197 1 test INFO: - - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0201 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0237 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00" 201 - "-" "-" "-" 0.0015 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0143 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/null" 201 - "-" "-" "-" 0.0130 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 92 "-" "-" "-" 0.0120 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 304 "-" "-" "-" 0.0031 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0067 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 201 "-" "-" "-" 0.0030 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0156 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 68 "-" "-" "-" 0.0027 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0022 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 37 "-" "-" "-" 0.0004 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 33 "-" "-" "-" 0.0001 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 38 "-" "-" "-" 0.0001 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 400 34 "-" "-" "-" 0.0001 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0164 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0019 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0017 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0018 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 204 - "-" "-" "-" 0.0066 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0177 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0018 "-" 3656197 1 test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0146 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0023 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0202 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/c1" 201 - "-" "-" "-" 0.0020 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 3 "-" "-" "-" 0.0131 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "HEAD /sda1/p/a" 204 - "-" "-" "-" 0.0022 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0169 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0040 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0099 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0108 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0109 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0134 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 133 "-" "-" "-" 0.0062 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0160 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0021 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0146 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00foo" 201 - "-" "-" "-" 0.0156 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00nullish" 201 - "-" "-" "-" 0.0114 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 2 "-" "-" "-" 0.0114 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 218 "-" "-" "-" 0.0023 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0152 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test01" 201 - "-" "-" "-" 0.0017 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/%00null%00test02" 201 - "-" "-" "-" 0.0129 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a/nullish" 201 - "-" "-" "-" 0.0206 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0147 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 95 "-" "-" "-" 0.0030 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 313 "-" "-" "-" 0.0028 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 200 204 "-" "-" "-" 0.0026 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a" 404 - "-" "-" "-" 0.0006 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0001 "-" 3656197 - test.unit.account.test_server.TestAccountController.test_utf8_metadata test.unit.account.test_server.TestAccountController.test_utf8_metadata ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument test.unit.cli.test_ringbuilder.TestCommands.test_main_single_argument ... ok test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe test.unit.cli.test_ringbuilder.TestCommands.test_main_with_safe ... ok test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced test.unit.cli.test_ringbuilder.TestCommands.test_no_warn_when_balanced ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_add_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_list_parts_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_remove_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format test.unit.cli.test_ringbuilder.TestCommands.test_parse_search_values_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_info_values_number_of_arguments ... ok Finished. Marked 0 objects for deletion. test WARNING: boom test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:54:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.9257 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.5621 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:52 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.9763 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:54:52 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0028 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:52 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:52 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:54:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.4962 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:55 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.9720 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:54:55 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:57 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 2.0088 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:57 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:57 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:54:58 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.9285 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:59 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 1.0110 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:54:59 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:54:59 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.5382 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:01 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.6039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:02 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 1.1452 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.3971 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:02 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0249 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:02 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0375 "-" 3656251 2 test ERROR: ERROR Account update failed with 127.0.0.1:51005/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:55:02 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0218 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0229 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0185 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0004 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0014 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0157 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0159 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0529 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:03 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0039 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0214 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0143 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0136 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0157 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0161 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0141 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0124 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0109 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0043 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0019 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:03 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.1991 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0187 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0231 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0417 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0121 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0157 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0142 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0153 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0068 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0111 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0223 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0019 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0219 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0112 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0028 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0022 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmpfy8z7su6/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0141 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0152 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:04 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0020 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0020 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0125 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0019 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_auto_record_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_non_ascii ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_invalid_accept ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_Response ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Grant AllUsers WRITE permission on the object /bucket/object test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_all_users_with_uppercase_type ... ok test INFO: - - - [02/Jun/2024:16:54:35 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0003 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:39 +0000] "GET /sda1/p/a/c%D8%3E%2520" 412 12 "-" "-" "-" 0.0001 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:40 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 1.2999 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:40 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0158 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:40 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0146 "-" 3656221 - test INFO: - - - [02/Jun/2024:16:54:40 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0130 "-" 3656221 - test ERROR: Bad key for 'container': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: Bad key for 'object': Key must be length 32 bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 181, in get_keys self.crypto.check_key(key) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 144, in check_key raise ValueError("Key must be length %s bytes" % self.key_length) ValueError: Key must be length 32 bytes test ERROR: get_keys(): from callback: boom: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 169, in get_keys keys = fetch_crypto_keys(key_id=key_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/crypto/test_crypto_utils.py", line 95, in callback raise Exception('boom') Exception: boom test ERROR: ERROR get_keys() missing callback: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 161, in get_keys fetch_crypto_keys = env[CRYPTO_KEY_CALLBACK] ~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'swift.callback.fetch_crypto_keys' test ERROR: Missing key for 'container': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'container' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Missing key for 'object': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ KeyError: 'object' test ERROR: Did not get a keys dict: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/crypto/crypto_utils.py", line 180, in get_keys key = keys[name] ~~~~^^^^^^ TypeError: list indices must be integers or slices, not str test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: A key is specified for bucket API. test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteNoAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_with_write_permission ... ok test ERROR: Invalid swift_bytes: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4161, in override_bytes_from_content_type listing_dict['bytes'] = int(swift_bytes) ^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: 'hey' test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test NOTICE: Closing unnecessary sock for port 6006 (child pid 88) test NOTICE: Closing unnecessary sock for port 6006 (child pid 89) test NOTICE: Closing unnecessary sock for port 6006 (child pid 90) test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test CRITICAL: Unable to bind to port 6009: ack test NOTICE: Started child 0 (PID 94) for port 6009 test NOTICE: Started child 1 (PID 95) for port 6009 test NOTICE: Started child 2 (PID 96) for port 6009 test NOTICE: Started child 1 (PID 97) for port 6009 test NOTICE: Child 2 (PID 3656256, port 6007) exiting normally test NOTICE: Started child 0 (PID 88) for port 6006 test NOTICE: Started child 1 (PID 89) for port 6006 test NOTICE: Started child 2 (PID 90) for port 6006 test NOTICE: Started child 0 (PID 91) for port 6007 test NOTICE: Started child 1 (PID 92) for port 6007 test NOTICE: Started child 2 (PID 93) for port 6007 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx0ee319bb87474da1b099d-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx0ac48b350d2b44e599455-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx492e4440718b45d5a7863-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb83e39be81fa43deb91b5-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx804f003b7aba4d06964f4-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1a0806c37f0a4416a849b-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf6838ffb98b74403bd856-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx55e198eeea3c4f7188de6-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx7ad294ef3af74e5181d7b-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd2ef63ca9758400ea897e-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txee1b1f7280ad45d2b323a-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx6fbb5bca253b43fdb64c2-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx211fe770a6fb4d5580675-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3419f1dd532b4bbfb12db-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx96faee717292405aadf6e-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txbad3f06286ce46ed8fa1a-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx9e099dc46ae3487d96e8b-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txe2d881ab4b7e44fbafa8a-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb0457db6ae6a48e8b9341-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd506703cb1ed497f9e135-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx3b38d5091239493897ced-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx50a28f2cc2e046b9b5a0e-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx941b3e8b556648d4b636f-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txeaacb86d21d641998ddbb-00665ca3f2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: tx367d16265846405e80178-00665ca3f2) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: tx63d0d894fea6419bb162c-00665ca3f2) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txeffddfe768fb43e4b18ef-00665ca3f2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_no_memcache_available ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCaching.test_get_from_shards_add_root_spi ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_record_type_shard_with_listing_state ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx5e33eeebb2904154be7c8-00665ca3e4) test INFO: Client disconnected on read of '/a/c' (txn: tx5e33eeebb2904154be7c8-00665ca3e4) test INFO: Client disconnected on read of '/a/c' (txn: txfd21176f9086447380db4-00665ca3e4) test INFO: Client disconnected on read of '/a/c' (txn: txfd21176f9086447380db4-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx44d78e65cfc447a3a3053-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txec57e25e4cdf4835ab88d-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx8f756bf66a064c48a2d26-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx8f756bf66a064c48a2d26-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx8f756bf66a064c48a2d26-00665ca3e4) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx8f756bf66a064c48a2d26-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx5fdb9fccdb2943efaae84-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx5fdb9fccdb2943efaae84-00665ca3e4) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx5fdb9fccdb2943efaae84-00665ca3e4) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx5fdb9fccdb2943efaae84-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: txf2d164d5425d40009cbd6-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: txf2d164d5425d40009cbd6-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: txf2d164d5425d40009cbd6-00665ca3e4) test ERROR: Account HEAD returning 503 for [] (txn: txf2d164d5425d40009cbd6-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx06c7ab37e8844466827b2-00665ca3e4) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx89aa795643e441dca675e-00665ca3e4) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx89aa795643e441dca675e-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5a2aa5f833ad478d9db12-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx39bda7f8e1be481b86a9f-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx955a3ab6039b4a44b539c-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx71fcb23146c847daa5f65-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata test.unit.proxy.test_mem_server.TestContainerController.test_POST_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT test.unit.proxy.test_mem_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_mem_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_mem_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test ERROR: Unexpected Error handling DELETE of 'bucket' 'Key1': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 128, in do_delete resp = req.get_response(self.app, method='DELETE', query=query, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1358, in _get_response sw_resp = sw_req.get_response(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 182, in __call__ return self.app(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/s3api/__init__.py", line 85, in __call__ return self.swift(env, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 197, in _select_response is_success = resp_class().is_success ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: kaboom test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': 'RKGh+lI7CsSmdqPX7Qqb7g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '44a1a1fa523b0ac4a676a3d7ed0a9bee', 'Content-Length': '289'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_enabled ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_versioned_suspended ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347272.00804', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347272.00804', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347272.00804', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347272.02537', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347272.02537', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347268.71900', 'epoch': '1717347268.71900', 'reported': 0, 'tombstones': -1}], '34e0bbe4-2193-4d29-8273-99a86ccbd29a-') test DEBUG: /tmp/tmp3_3ihv7o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347268.71900.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp3_3ihv7o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347268.71900.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '278f2c439ef4bb62fc37db9420dfc37e', '34e0bbe4-2193-4d29-8273-99a86ccbd29a-', '1717347268.28154', '1717347271.97826', '0', '', '1717347268.27684', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347272.00804', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347272.00804', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347272.00804', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347272.02537', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347272.02537', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347268.71900', 'epoch': '1717347268.71900', 'reported': 0, 'tombstones': -1}], '34e0bbe4-2193-4d29-8273-99a86ccbd29a-') test DEBUG: /tmp/tmp3_3ihv7o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347268.71900.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp3_3ihv7o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347268.71900.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '5124ef95b3d7cdb2483b6b62285bcd5d', '2416af4c-8575-4b7b-b0fa-be99855590be-', '1717347273.77492', '1717347273.77006', '0', '', '1717347273.77006', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347274.82170', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347274.82170', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347274.82170', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347274.84324', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347274.84324', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347274.20917', 'epoch': '1717347274.20917', 'reported': 0, 'tombstones': -1}], '2416af4c-8575-4b7b-b0fa-be99855590be-') test DEBUG: /tmp/tmpgiq2oxii/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347274.20917.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpgiq2oxii/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347274.20917.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (5, '5124ef95b3d7cdb2483b6b62285bcd5d', '2416af4c-8575-4b7b-b0fa-be99855590be-', '1717347273.77492', '1717347274.79158', '0', '', '1717347273.77006', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347274.82170', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347274.82170', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347274.82170', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347274.84324', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347274.84324', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347274.20917', 'epoch': '1717347274.20917', 'reported': 0, 'tombstones': -1}], '2416af4c-8575-4b7b-b0fa-be99855590be-') test DEBUG: /tmp/tmpgiq2oxii/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347274.20917.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpgiq2oxii/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347274.20917.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '3b7ba7f265e748ed0a1b689c674b5ccd', 'd0f62ee2-99d1-44e0-85f4-d17d0bdf3270-', '1717347275.08875', '1717347275.08135', '0', '', '1717347275.08135', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347275.10144', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347275.10144', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347275.10144', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347275.10208', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347275.10208', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347275.10161', 'epoch': '1717347275.10161', 'reported': 0, 'tombstones': -1}], 'd0f62ee2-99d1-44e0-85f4-d17d0bdf3270-') test DEBUG: /tmp/tmp_yay31mf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.10161.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp_yay31mf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.10161.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '3b7ba7f265e748ed0a1b689c674b5ccd', 'd0f62ee2-99d1-44e0-85f4-d17d0bdf3270-', '1717347275.08875', '1717347275.23397', '0', '', '1717347275.08135', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347275.10144', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347275.10144', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347275.10144', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347275.10208', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347275.10208', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347275.10161', 'epoch': '1717347275.10161', 'reported': 0, 'tombstones': -1}], 'd0f62ee2-99d1-44e0-85f4-d17d0bdf3270-') test DEBUG: /tmp/tmp_yay31mf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.10161.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp_yay31mf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.10161.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharded_remote_unsharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_missing_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_sharding_overlaps ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_Response ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_get_shard_usage ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:44 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:55867/sdb1/1/a/c2" "tx2538eac50fa246edbc748-00665ca3d4" "container-server 3656269" 0.0146 "-" 3656269 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:39789/sda1/1/a/c2" "tx2538eac50fa246edbc748-00665ca3d4" "container-server 3656269" 0.0026 "-" 3656269 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx2538eac50fa246edbc748-00665ca3d4" "proxy-server 3656269" 3.7043 "-" 3656269 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:54:48 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx2538eac50fa246edbc748-00665ca3d4" "proxy-server 3656269" 3.7261 "-" 3656269 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/54/48 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx2538eac50fa246edbc748-00665ca3d4 - 3.7526 - - 1717347284.853950500 1717347288.606599331 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /v1/a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 full Trying to GET /v1/a/c/o From Object Server 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage test DEBUG: Node is error limited: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: test msg 10.0.0.0:1000/sda test ERROR: ERROR with test1 server 10.0.0.0:1000/sda re: test1 msg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1307, in test_error_limit_methods raise Exception('kaboom1!') Exception: kaboom1! test WARNING: ERROR with test2 server 10.0.0.0:1000/sda re: test2 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1321, in test_error_limit_methods raise Exception('kaboom2!') Exception: kaboom2! test WARNING: ERROR with test3 server 10.0.0.0:1000/sda re: test3 msg Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1336, in test_error_limit_methods raise Exception('kaboom3!') Exception: kaboom3! test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: succès 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test ERROR: success 10.0.0.0:1000/sda test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: succès: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_multiple_policies ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging test.unit.proxy.test_mem_server.TestProxyServer.test_exception_occurred_replication_ip_port_logging ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller test.unit.proxy.test_mem_server.TestProxyServer.test_get_info_controller ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring test.unit.proxy.test_mem_server.TestProxyServer.test_get_object_ring ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_inexistent_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults test.unit.proxy.test_mem_server.TestProxyServer.test_info_defaults ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request test.unit.proxy.test_mem_server.TestProxyServer.test_internal_reserved_name_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length test.unit.proxy.test_mem_server.TestProxyServer.test_negative_content_length ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency test.unit.proxy.test_mem_server.TestProxyServer.test_node_concurrency ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing test.unit.proxy.test_mem_server.TestProxyServer.test_node_timing ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request test.unit.proxy.test_mem_server.TestProxyServer.test_private_method_request ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_inherits ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_policy_info ... ok test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 201 - "-" "-" "-" 0.0267 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "PUT /sda1/p/a" 202 - "-" "-" "-" 0.0116 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0143 "-" 3656197 - test INFO: - - - [02/Jun/2024:16:54:35 +0000] "POST /sda1/p/a" 204 - "-" "-" "-" 0.0096 "-" 3656197 - test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 408, in DELETE self._delete_segments_bucket(req) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 56, in _delete_segments_bucket resp = req.get_response(self.app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 199, in HEAD return self._handle_acl(app, 'DELETE') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 141, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_while_segment_bucket_delete ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_report_info ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBroker.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBroker.test_newid test.unit.container.test_backend.TestContainerBroker.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBroker.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_old_style_root_account_container_path ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error /tmp/tmp1upvth1n/tmpsnqj_qw8, build version 4, id 1021f8f174c74bc0a4b883b87e6418c8 64 partitions, 3.000000 replicas, 4 regions, 4 zones, 4 devices, 100.00 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 1 (0:00:00 remaining) The overload factor is 0.00% (0.000000) Ring file /tmp/tmp1upvth1n/tmpsnqj_qw8.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 0 0 0 127.0.0.1:6200 127.0.0.1:6200 sda1 100.00 0 -100.00 some meta data 1 1 1 127.0.0.2:6201 127.0.0.2:6201 sda2 100.00 0 -100.00 2 2 2 127.0.0.3:6202 127.0.0.3:6202 sdc3 100.00 0 -100.00 3 3 3 127.0.0.4:6203 127.0.0.4:6203 sdd4 100.00 0 -100.00 Reassigned 192 (300.00%) partitions. Balance is now 5.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_region_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_weight_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_parse_set_zone_values_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed test.unit.cli.test_ringbuilder.TestCommands.test_pretend_min_part_hours_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance test.unit.cli.test_ringbuilder.TestCommands.test_rebalance ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_failure_does_not_reset_last_moves_epoch ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_initialize_container_broker_in_default ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_min_part_hours_not_passed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_device_change ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_remove_zero_weighted_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_removed_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_resets_time_remaining ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_legacy_pending_files ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_saves_dispersion_improvement ... ok test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed test.unit.cli.test_ringbuilder.TestCommands.test_rebalance_with_seed ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_remove_all_devices ... ok test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBroker.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBroker.test_path test.unit.container.test_backend.TestContainerBroker.test_path ... ok test.unit.container.test_backend.TestContainerBroker.test_pending_file_name test.unit.container.test_backend.TestContainerBroker.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBroker.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBroker.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object test.unit.container.test_backend.TestContainerBroker.test_put_object ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBroker.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim test.unit.container.test_backend.TestContainerBroker.test_reclaim ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx87527efc42e84575bb287-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx462c5996baa24b918c725-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: tx756733b988fa49ddb7e93-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx41b1c05fa7e34d1785270-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx1ac7acc9dbed4614bda9f-00665ca3f2) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: tx1446e9ddf388499d982f5-00665ca3f2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb6ffd2036da14335a15f9-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txcc394b7baf5a4cc6bf5db-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx2ea89fe6d41d4b4d869f7-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx84474d9c0f864c409aeba-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txe3068bfa0b094cb89172a-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx14eff5ad683b466bb992d-00665ca3f2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: txda5da80a1b964965b7065-00665ca3f2) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txda5da80a1b964965b7065-00665ca3f2) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf14555fc03114b419182f-00665ca3f5) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_empty_shard_no_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_order_and_reverse ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server ERROR: Object returning 503 for [] (txn: tx724f4bd927d24cb6bafe0-00665ca3da) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server ERROR: Object returning 503 for [] (txn: tx4a29f592e2b74fa4a5d7e-00665ca3db) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_legacy_pending_files ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_error ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txeab13bd6356a4a3e987c8-00665ca3e4) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa4ff386117414e1aa1d76-00665ca3e4) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txa4ff386117414e1aa1d76-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test WARNING: Could not autocreate account '/account' (txn: txab11cb08b3b14aa98ec05-00665ca3e4) test INFO: autocreate account '/account' (txn: tx730b8a43d99e4c23b93a8-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txaf93bdf21d76437386859-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txaf93bdf21d76437386859-00665ca3e4) test ERROR: Account HEAD returning 503 for [503, 503] (txn: txaf93bdf21d76437386859-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: txaf93bdf21d76437386859-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: txaf93bdf21d76437386859-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: txaf93bdf21d76437386859-00665ca3e4) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txaf93bdf21d76437386859-00665ca3e4) test WARNING: Could not autocreate account '/account' (txn: txaf93bdf21d76437386859-00665ca3e4) test INFO: autocreate account '/account' (txn: txc14b0b3a97e34565a4ceb-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf9046b93221d4af0b5381-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txf9046b93221d4af0b5381-00665ca3e4) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txf9046b93221d4af0b5381-00665ca3e4) test INFO: autocreate account '/account' (txn: tx84f5855c225b4b76a2dc3-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txae855f906dbd497699e16-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: txae855f906dbd497699e16-00665ca3e4) test INFO: autocreate account '/account' (txn: txae855f906dbd497699e16-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: txd063d2fd4e324ac5b04b8-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: txd063d2fd4e324ac5b04b8-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: txc23fabf9645a411cab291-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: tx5f6de4c33ad14a5ea41a0-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa0afd4f437d344ac834a8-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx741c93847c1342a3a00e9-00665ca3e4) test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_mem_server.TestContainerController.test_PUT_calls_clean_acl ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_mem_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_mem_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_mem_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_overwrite_unicode ... ok test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0206 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0020 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0013 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0331 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0006 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0225 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0006 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0315 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0313 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0213 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0219 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0108 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0101 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0099 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0056 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0051 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0061 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0064 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0056 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0061 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0064 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0064 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmp1ckx6dpc/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0057 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0046 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0056 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0051 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0175 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0017 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0177 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0176 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0015 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0297 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0052 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0182 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0237 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0351 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0150 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0005 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0168 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0020 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_json_last_modified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_log_requests_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_marker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_prefix_delim ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AllUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant AuthenticatedUsers READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_authenticated_users ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_items_post_overwrite_out_of_order ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_abort_multipart_upload_acl_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_deleted ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': 'OwaL35FcoygjuZOYTERk/g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '3b068bdf915ca32823b993984c4464fe', 'Content-Length': '230'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_error ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_fullcontrol_permission ... ok test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: ERROR with server-type server 10.0.0.0:1000/sda re: success: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/test_server.py", line 1210, in do_test raise Exception('kaboom1!') Exception: kaboom1! test ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx46bd0794e3024ae5ad59a-00665ca3de) test INFO: Client disconnected on read of '/a/c' (txn: tx46bd0794e3024ae5ad59a-00665ca3de) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.1.0:1100/sda (txn: tx6ab51852cdbe4e7187cdf-00665ca3de) test INFO: Client disconnected on read of '/a/c' (txn: tx6ab51852cdbe4e7187cdf-00665ca3de) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to UPDATE /a/c: Network unreachable (txn: tx6232599850a64ae1a45ca-00665ca3df) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to UPDATE /a/c: Network unreachable (txn: tx6232599850a64ae1a45ca-00665ca3df) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to UPDATE /a/c: Network unreachable (txn: tx6232599850a64ae1a45ca-00665ca3df) test ERROR: Container UPDATE returning 503 for (503, 503, 503) (txn: tx6232599850a64ae1a45ca-00665ca3df) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r3=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_overrides ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_per_policy_with_no_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_affinity_proxy_server_config ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_by_timing_per_policy ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '3b7ba7f265e748ed0a1b689c674b5ccd', '2a14a08e-c5ad-4331-866b-8aa7d6d4e968-', '1717347275.40254', '1717347275.39418', '0', '', '1717347275.39418', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347275.42289', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347275.42289', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347275.42289', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347275.42336', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347275.42336', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347275.42310', 'epoch': '1717347275.42310', 'reported': 0, 'tombstones': -1}], '2a14a08e-c5ad-4331-866b-8aa7d6d4e968-') test DEBUG: /tmp/tmpu0_wff13/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.42310.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpu0_wff13/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.42310.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '3b7ba7f265e748ed0a1b689c674b5ccd', '2a14a08e-c5ad-4331-866b-8aa7d6d4e968-', '1717347275.40254', '1717347275.53869', '0', '', '1717347275.39418', 99, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347275.42289', 'lower': '', 'upper': 'a', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347275.42289', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347275.42289', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347275.42336', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347275.42336', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347275.42310', 'epoch': '1717347275.42310', 'reported': 0, 'tombstones': -1}], '2a14a08e-c5ad-4331-866b-8aa7d6d4e968-') test DEBUG: /tmp/tmpu0_wff13/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.42310.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpu0_wff13/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347275.42310.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '1b8ba46b2d7967b2c6465147a8187e6e', 'e0c8ee18-e45d-4f74-aa0a-89b3fb7b38aa-', '1717347287.21240', '1717347287.08943', '0', '', '1717347287.08943', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e0c8ee18-e45d-4f74-aa0a-89b3fb7b38aa-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.49204.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '1b8ba46b2d7967b2c6465147a8187e6e', 'e0c8ee18-e45d-4f74-aa0a-89b3fb7b38aa-', '1717347287.21240', '1717347287.08943', '0', '', '1717347287.08943', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347287.64414', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347287.64414', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347287.64414', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347288.49257', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347288.49257', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347288.49204', 'epoch': '1717347288.49204', 'reported': 0, 'tombstones': -1}], 'e0c8ee18-e45d-4f74-aa0a-89b3fb7b38aa-') test DEBUG: /tmp/tmpfodfk_t8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.49204.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpfodfk_t8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.49204.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '07e8d99ebc46ab04cec539a9fa25f7b4', '32913f69-6b5e-4a93-8ba2-626d5e919f61-', '1717347288.72877', '1717347288.72099', '0', '', '1717347288.72099', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('32913f69-6b5e-4a93-8ba2-626d5e919f61-', '5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.78855.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (8, '07e8d99ebc46ab04cec539a9fa25f7b4', '32913f69-6b5e-4a93-8ba2-626d5e919f61-', '1717347288.72877', '1717347288.72099', '0', '', '1717347288.72099', 2, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347288.75032', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347288.75032', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347288.75032', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347288.78892', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347288.78892', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347288.78855', 'epoch': '1717347288.78855', 'reported': 0, 'tombstones': -1}], '32913f69-6b5e-4a93-8ba2-626d5e919f61-') test DEBUG: /tmp/tmpigrnbay7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.78855.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpigrnbay7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.78855.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '6d287da3aa7ecec127689cf41d818655', 'b7ce0dfc-0319-477f-8e56-b2c9640a10b4-', '1717347288.95803', '1717347288.95289', '0', '', '1717347288.95289', 3, 0) test WARNING: /tmp/tmpecxcsu_w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.98053.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmpecxcsu_w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.98053.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '6d287da3aa7ecec127689cf41d818655', 'b7ce0dfc-0319-477f-8e56-b2c9640a10b4-', '1717347288.95803', '1717347289.07438', '0', '', '1717347288.95289', 3, 0) test WARNING: /tmp/tmpecxcsu_w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.98053.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmpecxcsu_w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347288.98053.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'e76369fff84f1f782c874b2a7905753a', 'f39069af-662e-4143-9efc-b73acb09b5f6-', '1717347289.17982', '1717347289.16740', '0', '', '1717347289.16740', 3, 0) test WARNING: /tmp/tmplptsuwr1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347289.21662.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_presharding_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_sharding_large_diff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_enabled_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_error_with_suspended_versioning ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_error ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: Setting object count to 1032 (txn: txf14555fc03114b419182f-00665ca3f5) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: Setting object count to 48 (txn: tx3ff936f1155d410ebe143-00665ca3f5) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: Setting object count to 1064 (txn: txaead453413504da8940e4-00665ca3f5) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx10a849d2ce2b4fc28ca5c-00665ca3f5) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc9ff92e974064a4897b92-00665ca3f5) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb9005f5a92d745a8961f7-00665ca3f8) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_mixed_policies_error ... ok test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx6963e047fe864091be268-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx51fb6a10a9a3455ea0dd4-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx83c6ba7fe9fd494e8ea36-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx075c121335c64d8eb17f7-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx368c126f0b3549eab353a-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx368c126f0b3549eab353a-00665ca3e4) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx368c126f0b3549eab353a-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx054c7da0efa945ad9712a-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx054c7da0efa945ad9712a-00665ca3e4) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx054c7da0efa945ad9712a-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx3355edc61bfd456daa6e4-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx3355edc61bfd456daa6e4-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx3355edc61bfd456daa6e4-00665ca3e4) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx3355edc61bfd456daa6e4-00665ca3e4) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx42f8f496a6484dc493f51-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx42f8f496a6484dc493f51-00665ca3e4) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx42f8f496a6484dc493f51-00665ca3e4) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx42f8f496a6484dc493f51-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_mem_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_mem_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_mem_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_mem_server.TestContainerController.test_error_convert_index_to_name ... ok swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. Reassigned 768 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) Reassigned 192 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (0:24:00 remaining) No partitions could be reassigned. There is no need to do so at this time Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 1000.0 No partitions could be reassigned. The time between rebalances must be at least min_part_hours: 1 hours (1:00:00 remaining) ------------------------------------------------------------------------------- An error has occurred during ring validation. Common causes of failure are rings that are empty or do not have enough devices to accommodate the replica count. Original exception message: There are no devices in this ring, or all devices have been deleted ------------------------------------------------------------------------------- Partition power increase in progress. You need to finish the increase first before rebalancing. Reassigned 0 (0.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Reassigned 20 (31.25%) partitions. Balance is now 24.48. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 24.48 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 33.33 ------------------------------------------------------------------------------- NOTE: Dispersion of 33.333333 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 64 (100.00%) partitions. Balance is now 100.00. Dispersion is now 16.67 ------------------------------------------------------------------------------- NOTE: Dispersion of 16.666667 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" marked for removal and will be removed next rebalance. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" marked for removal and will be removed next rebalance. d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. test.unit.cli.test_ringbuilder.TestCommands.test_remove_device test.unit.cli.test_ringbuilder.TestCommands.test_remove_device ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase test.unit.cli.test_ringbuilder.TestCommands.test_remove_device_part_power_increase ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_search_device_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_search_device_no_matching ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server ERROR: Object returning 503 for [] (txn: txf9a58ad7bbc548e1ae044-00665ca3db) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx843f63429df04760ba9c6-00665ca3db) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from primary (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_mixed_frags_and_no_quorum_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_GET_with_no_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_simple ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBroker.test_reclaim_deadlock ... ok test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0216 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0019 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0086 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0177 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0013 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0159 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0290 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0048 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0057 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0353 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0261 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0206 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0228 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:06 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0035 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:07 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.5131 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 1.1849 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0117 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0153 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0024 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0004 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:08 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0167 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_listing ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_namespaces_unsharded_subshard_state_updating ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_non_ascii ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 501 Not Implemented test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_email_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_full_control ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_version_id_not_implemented ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r0=0', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 0 (zero): ProxyOverrideOptions({}, {'sorting_method': 'timing', 'read_affinity': 'r1=1,r2=2', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for policy 1 (one): ProxyOverrideOptions({}, {'sorting_method': 'affinity', 'read_affinity': 'r2=1', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default test.unit.proxy.test_mem_server.TestProxyServer.test_sort_nodes_default ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception test.unit.proxy.test_mem_server.TestProxyServer.test_unhandled_exception ... ok test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version test.unit.proxy.test_mem_server.TestProxyServer.test_valid_api_version ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_list_parts ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBroker.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBroker.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync test.unit.container.test_backend.TestContainerBroker.test_reconciler_sync ... ok test WARNING: /tmp/tmplptsuwr1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347289.21662.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, 'e76369fff84f1f782c874b2a7905753a', 'f39069af-662e-4143-9efc-b73acb09b5f6-', '1717347289.17982', '1717347294.62513', '0', '', '1717347289.16740', 3, 0) test WARNING: /tmp/tmplptsuwr1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347289.21662.db is unable to replicate shard ranges to peer 10.0.0.1:1001/sdb; peer may need upgrading test WARNING: /tmp/tmplptsuwr1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347289.21662.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '765f64dfd953bcba33f14fa6de0998bc', 'd83b6fad-8beb-42d4-98b7-2317a3b10ab2-', '1717347298.30015', '1717347298.29631', '0', '', '1717347298.29631', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347302.96861', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347302.96861', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347302.96861', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347302.99989', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347302.99989', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347300.00300', 'epoch': '1717347300.00300', 'reported': 0, 'tombstones': -1}], 'd83b6fad-8beb-42d4-98b7-2317a3b10ab2-') test DEBUG: /tmp/tmpg1qtulzz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347300.00300.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpg1qtulzz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347300.00300.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '765f64dfd953bcba33f14fa6de0998bc', 'd83b6fad-8beb-42d4-98b7-2317a3b10ab2-', '1717347298.30015', '1717347302.94760', '0', '', '1717347298.29631', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347302.96861', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347302.96861', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347302.96861', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347302.99989', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347302.99989', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347300.00300', 'epoch': '1717347300.00300', 'reported': 0, 'tombstones': -1}], 'd83b6fad-8beb-42d4-98b7-2317a3b10ab2-') test DEBUG: /tmp/tmpg1qtulzz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347300.00300.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpg1qtulzz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347300.00300.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '496fbd3d97933fb43657f2aab381a29d', '8275edba-615d-436a-838f-4c39b4972589-', '1717347303.25300', '1717347303.24610', '0', '', '1717347303.24610', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347303.41661', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.41661', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347303.41661', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347303.43198', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.43198', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347303.27595', 'epoch': '1717347303.27595', 'reported': 0, 'tombstones': -1}], '8275edba-615d-436a-838f-4c39b4972589-') test DEBUG: /tmp/tmp5xky_c3o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.27595.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp5xky_c3o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.27595.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (10, '496fbd3d97933fb43657f2aab381a29d', '8275edba-615d-436a-838f-4c39b4972589-', '1717347303.25300', '1717347303.39757', '0', '', '1717347303.24610', 5, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347303.41661', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.41661', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347303.41661', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347303.43198', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.43198', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347303.27595', 'epoch': '1717347303.27595', 'reported': 0, 'tombstones': -1}], '8275edba-615d-436a-838f-4c39b4972589-') test DEBUG: /tmp/tmp5xky_c3o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.27595.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp5xky_c3o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.27595.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '0f29b8a1f5b9f593f2c16fbe70df7a5b', '1b767b97-f785-41ff-8843-c59198139866-', '1717347303.64512', '1717347303.63513', '0', '', '1717347303.63513', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347303.66060', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.66060', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347303.66060', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347303.68231', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.68231', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347303.67989', 'epoch': '1717347303.67989', 'reported': 0, 'tombstones': -1}], '1b767b97-f785-41ff-8843-c59198139866-') test DEBUG: /tmp/tmp7_803ic1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.67989.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp7_803ic1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.67989.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '0f29b8a1f5b9f593f2c16fbe70df7a5b', '1b767b97-f785-41ff-8843-c59198139866-', '1717347303.64512', '1717347303.77600', '0', '', '1717347303.63513', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_sharding_remote_unsharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded ... ok test.unit.container.test_backend.TestContainerBroker.test_remove_objects test.unit.container.test_backend.TestContainerBroker.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBroker.test_resolve_shard_range_states ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_invalid_md5 ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_non_json ... ok test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBroker.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path test.unit.container.test_backend.TestContainerBroker.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBroker.test_save_and_load_record_from_pending_file ... ok test ERROR: Problem finding shard upper in '/tmp/tmp2bx8ns04/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp2bx8ns04/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpn2hh7sow/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpn2hh7sow/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBroker.test_set_db_states test.unit.container.test_backend.TestContainerBroker.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBroker.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBroker.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs test.unit.container.test_backend.TestContainerBroker.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBroker.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBroker.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBroker.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property test.unit.container.test_backend.TestContainerBroker.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBroker.test_sync_merging test.unit.container.test_backend.TestContainerBroker.test_sync_merging ... ok test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_finalize_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_most_cleave_progress ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_no_complete_path ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_two_sub_path_splits ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split test.unit.container.test_sharder.TestSharderFunctions.test_find_and_rank_whole_path_split ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_all_donors_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_donors_but_no_suitable_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_eligible_states ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_expansion_limit_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_four_donors_two_acceptors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_expanding ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_max_shrinking ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_nine_donors_one_acceptor ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_donors ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_no_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_overlapping_ranges_with_ineligible_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shard_ranges_in_found_state ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_shrink_threshold_with_tombstones ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root test.unit.container.test_sharder.TestSharderFunctions.test_find_compactible_single_donor_shrink_to_root ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges test.unit.container.test_sharder.TestSharderFunctions.test_find_overlapping_ranges ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps test.unit.container.test_sharder.TestSharderFunctions.test_find_paths_with_gaps ... ok test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates test.unit.container.test_sharder.TestSharderFunctions.test_find_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_sharding_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate test.unit.container.test_sharder.TestSharderFunctions.test_is_shrinking_candidate ... ok test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible test.unit.container.test_sharder.TestSharderFunctions.test_process_compactible ... ok test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats test.unit.container.test_sharder.TestSharderFunctions.test_update_own_shard_range_stats ... ok test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect test.unit.obj.test_server.TestObjectServer.test_expect_on_multiphase_put_diconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put test.unit.obj.test_server.TestObjectServer.test_expect_on_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict test.unit.obj.test_server.TestObjectServer.test_expect_on_put_conflict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer test.unit.obj.test_server.TestObjectServer.test_expect_on_put_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message test.unit.obj.test_server.TestObjectServer.test_multiphase_put_bad_commit_message ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_in_the_middle_of_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit test.unit.obj.test_server.TestObjectServer.test_multiphase_put_client_disconnect_right_before_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_drains_extra_commit_junk_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers test.unit.obj.test_server.TestObjectServer.test_multiphase_put_ec_fragment_in_headers_no_footers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect test.unit.obj.test_server.TestObjectServer.test_multiphase_put_metadata_footer_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_metadata_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary test.unit.obj.test_server.TestObjectServer.test_multiphase_put_no_mime_boundary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectServer.test_not_found test.unit.obj.test_server.TestObjectServer.test_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call test.unit.obj.test_ssync_sender.TestSender.test_call ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_metadata_legacy_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required test.unit.obj.test_ssync_sender.TestSender.test_call_and_missing_check_with_obj_list_but_required ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others test.unit.obj.test_ssync_sender.TestSender.test_call_calls_others ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout test.unit.obj.test_ssync_sender.TestSender.test_call_catches_MessageTimeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException test.unit.obj.test_ssync_sender.TestSender.test_call_catches_ReplicationException ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception test.unit.obj.test_ssync_sender.TestSender.test_call_catches_exception_handling_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions test.unit.obj.test_ssync_sender.TestSender.test_call_catches_other_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect test.unit.obj.test_ssync_sender.TestSender.test_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status test.unit.obj.test_ssync_sender.TestSender.test_connect_bad_status ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_no_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_none_frag_to_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated test.unit.obj.test_ssync_sender.TestSender.test_connect_handoff_replicated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_legacy_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_receive_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout test.unit.obj.test_ssync_sender.TestSender.test_connect_send_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable test.unit.obj.test_ssync_sender.TestSender.test_connect_upgraded_receiver_include_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect test.unit.obj.test_ssync_sender.TestSender.test_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout test.unit.obj.test_ssync_sender.TestSender.test_disconnect_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts test.unit.obj.test_ssync_sender.TestSender.test_missing_check_extra_line_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_disconnect2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected test.unit.obj.test_ssync_sender.TestSender.test_missing_check_far_end_unexpected ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes test.unit.obj.test_ssync_sender.TestSender.test_missing_check_has_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_exactly_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects test.unit.obj.test_ssync_sender.TestSender.test_missing_check_max_objects_less_than_actual_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map test.unit.obj.test_ssync_sender.TestSender.test_missing_check_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_send_line ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_missing_check_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all test.unit.obj.test_ssync_sender.TestSender.test_readline_all ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed test.unit.obj.test_ssync_sender.TestSender.test_readline_all_trailing_not_newline_termed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_at_start_of_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_broken_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow test.unit.obj.test_ssync_sender.TestSender.test_readline_buffer_exceeds_network_chunk_size_somehow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension test.unit.obj.test_ssync_sender.TestSender.test_readline_chunk_with_extension ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer test.unit.obj.test_ssync_sender.TestSender.test_readline_newline_in_buffer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk test.unit.obj.test_ssync_sender.TestSender.test_readline_terminated_chunk ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete test.unit.obj.test_ssync_sender.TestSender.test_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_delete_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post test.unit.obj.test_ssync_sender.TestSender.test_send_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_post_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put test.unit.obj.test_ssync_sender.TestSender.test_send_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_chunk_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout test.unit.obj.test_ssync_sender.TestSender.test_send_put_initial_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable test.unit.obj.test_ssync_sender.TestSender.test_send_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name test.unit.obj.test_ssync_sender.TestSender.test_send_put_unicode_header_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete test.unit.obj.test_ssync_sender.TestSender.test_update_send_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map test.unit.obj.test_ssync_sender.TestSender.test_updates_empty_send_map ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted test.unit.obj.test_ssync_sender.TestSender.test_updates_is_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_post test.unit.obj.test_ssync_sender.TestSender.test_updates_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put test.unit.obj.test_ssync_sender.TestSender.test_updates_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post test.unit.obj.test_ssync_sender.TestSender.test_updates_put_and_post ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_disconnect_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_timeout_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_end ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start test.unit.obj.test_ssync_sender.TestSender.test_updates_read_response_unexp_start ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index test.unit.obj.test_ssync_sender.TestSender.test_updates_storage_policy_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout test.unit.obj.test_ssync_sender.TestSender.test_updates_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 test.unit.obj.test_ssync_sender.TestSender.test_updates_unexpected_response_lines2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_admin_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info test.unit.proxy.controllers.test_info.TestInfoController.test_disabled_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info test.unit.proxy.controllers.test_info.TestInfoController.test_disallow_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_expires ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_key ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_method ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path test.unit.proxy.controllers.test_info.TestInfoController.test_get_admin_info_invalid_path ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info test.unit.proxy.controllers.test_info.TestInfoController.test_get_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors test.unit.proxy.controllers.test_info.TestInfoController.test_get_info_cors ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_admin_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_head_info test.unit.proxy.controllers.test_info.TestInfoController.test_head_info ... ok test.unit.proxy.controllers.test_info.TestInfoController.test_options_info test.unit.proxy.controllers.test_info.TestInfoController.test_options_info ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_errors ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_mostly_not_found ... ok d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" with 3.14159265359 weight got id 4 d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:9000/sda30_"other meta data" marked for removal and will be removed next rebalance. Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" marked for removal and will be removed next rebalance. Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Partition power increase in progress. You need to finish the increase first before removing devices. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 test.test.com 6200 r.test.com 7000 sda3 3.14 2 33.67 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 0 0 0 127.0.0.1 6200 127.0.0.1 6200 sda1 100.00 0 -100.00 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Devices: id region zone ip address port replication ip replication port name weight partitions balance meta 4 2 3 2001:0:1234::c1c0:abcd:876 6200 2::10 7000 sda3 3.14 2 33.67 some meta data No matching devices found swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_search_device_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high test.unit.cli.test_ringbuilder.TestCommands.test_server_overload_crazy_high ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info test.unit.cli.test_ringbuilder.TestCommands.test_set_info ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_all_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists test.unit.cli.test_ringbuilder.TestCommands.test_set_info_already_exists ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_info_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_info_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_info_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_info_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 96, in handle_acl return ah._handle_acl(app, method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_version_id_not_implemented ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_empty_versioning ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_group_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_permission ... ok test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0185 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0121 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0175 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0152 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0034 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0032 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0355 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0153 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0152 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0150 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0155 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0072 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0041 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:08 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0031 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:09 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0177 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0255 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmp32woxyx6/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0092 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0055 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:09 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0222 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0322 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmp13qc8r0y/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0047 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0039 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:09 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0303 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0222 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:09 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0046 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.8668 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 2.0199 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:13 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:13 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:13 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0026 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:14 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.4274 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:15 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 1.4009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:15 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:15 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:15 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0028 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:17 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 1.8652 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:18 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 1.5554 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:18 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:18 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0242 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0036 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.2145 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0149 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.3960 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0034 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.2717 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0409 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0006 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_over_limit ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_plain ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_prefix ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_auditing ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_encoding_type_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_merge_shard_ranges_state ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_newid ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_read_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_root_container ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347303.66060', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.66060', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347303.66060', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347303.68231', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347303.68231', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347303.67989', 'epoch': '1717347303.67989', 'reported': 0, 'tombstones': -1}], '1b767b97-f785-41ff-8843-c59198139866-') test DEBUG: /tmp/tmp7_803ic1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.67989.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmp7_803ic1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347303.67989.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '7f45e752cc04d38ed45d829062148db6', '09f871f3-7fd0-452c-afdd-bdf925d41ef2-', '1717347304.21094', '1717347304.20402', '0', '', '1717347304.20402', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347304.22271', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.22271', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347304.22271', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347304.25119', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.25119', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347304.24185', 'epoch': '1717347304.24185', 'reported': 0, 'tombstones': -1}], '09f871f3-7fd0-452c-afdd-bdf925d41ef2-') test DEBUG: /tmp/tmpybauz6ip/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347304.24185.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpybauz6ip/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347304.24185.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '7f45e752cc04d38ed45d829062148db6', '09f871f3-7fd0-452c-afdd-bdf925d41ef2-', '1717347304.21094', '1717347304.35732', '0', '', '1717347304.20402', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347304.22271', 'lower': '', 'upper': 'a', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.22271', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347304.22271', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347304.25119', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.25119', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347304.24185', 'epoch': '1717347304.24185', 'reported': 0, 'tombstones': -1}], '09f871f3-7fd0-452c-afdd-bdf925d41ef2-') test DEBUG: /tmp/tmpybauz6ip/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347304.24185.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpybauz6ip/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872_1717347304.24185.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0ab5abb4650db3d722ec5c0017f153e7', 'c4ff7eae-f357-44f3-b885-ab0d2995bf9a-sda', '1717347304.48742', '1717347304.48237', '0', '', '1717347304.48237', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c4ff7eae-f357-44f3-b885-ab0d2995bf9a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '08198fb8aa2801fce836f837a6c19e48', '5c892d4e-a059-4140-871e-5189dd06b120-sda', '1717347304.57376', '1717347304.56823', '0', '', '1717347304.56823', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347304.63484', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347304.63484', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347304.63484', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347304.65619', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.65619', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347304.65558', 'epoch': '1717347304.65558', 'reported': 0, 'tombstones': -1}], '5c892d4e-a059-4140-871e-5189dd06b120-sda') test DEBUG: /tmp/tmpa_icwda5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpa_icwda5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '08198fb8aa2801fce836f837a6c19e48', '5c892d4e-a059-4140-871e-5189dd06b120-sda', '1717347304.57376', '1717347304.61318', '0', '', '1717347304.56823', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347304.63484', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347304.63484', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347304.63484', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347304.65619', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.65619', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347304.65558', 'epoch': '1717347304.65558', 'reported': 0, 'tombstones': -1}], '5c892d4e-a059-4140-871e-5189dd06b120-sda') test DEBUG: /tmp/tmpa_icwda5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpa_icwda5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '08198fb8aa2801fce836f837a6c19e48', 'ca426d3c-8c40-49e3-8c74-3534c3789d8c-sda', '1717347304.85522', '1717347304.84994', '0', '', '1717347304.84994', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347304.89745', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347304.89745', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347304.89745', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347304.90983', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.90983', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347304.90948', 'epoch': '1717347304.90948', 'reported': 0, 'tombstones': -1}], 'ca426d3c-8c40-49e3-8c74-3534c3789d8c-sda') test DEBUG: /tmp/tmph6lppao_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff test.unit.container.test_replicator.TestReplicatorSync.test_replication_local_unsharded_remote_sharded_large_diff ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_after_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharded ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding test.unit.container.test_replicator.TestReplicatorSync.test_replication_rsync_then_merge_aborts_before_merge_sharding ... ok test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync test.unit.container.test_replicator.TestReplicatorSync.test_replicator_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date test.unit.container.test_replicator.TestReplicatorSync.test_report_up_to_date ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure test.unit.container.test_replicator.TestReplicatorSync.test_rsync_failure ... ok test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge test.unit.container.test_replicator.TestReplicatorSync.test_rsync_then_merge ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines test.unit.container.test_replicator.TestReplicatorSync.test_sync_bogus_db_quarantines ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_misplaced_object_does_not_effect_container_stats ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'Content-Md5': '1mSVh2rJhFbgxzf6YLHnJA==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd66495876ac98456e0c737fa60b1e724', 'Content-Length': '184', 'Content-Type': 'multipart/form-data'} test ERROR: Could not parse SLO delete response (200 OK): b'asdf': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/multi_delete.py", line 136, in do_delete delete_result = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_md5 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: txf7a79c11b7374866b7a5a-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txef4c0b28cf7541609f893-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txef4c0b28cf7541609f893-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txef4c0b28cf7541609f893-00665ca3e4) test ERROR: Account HEAD returning 503 for [] (txn: txef4c0b28cf7541609f893-00665ca3e4) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx35fea2dfb8c54cb190e50-00665ca3e4) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx35fea2dfb8c54cb190e50-00665ca3e4) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx35fea2dfb8c54cb190e50-00665ca3e4) test ERROR: Account HEAD returning 503 for [] (txn: tx35fea2dfb8c54cb190e50-00665ca3e4) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx811196060e214eb581401-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx2cc5b7ec872d4ca8af66f-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx2cc5b7ec872d4ca8af66f-00665ca3e4) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx2cc5b7ec872d4ca8af66f-00665ca3e4) test ERROR: Account HEAD returning 503 for [] (txn: tx2cc5b7ec872d4ca8af66f-00665ca3e4) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx171363c6549a465cb67ca-00665ca3e4) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx171363c6549a465cb67ca-00665ca3e4) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx171363c6549a465cb67ca-00665ca3e4) test ERROR: Account HEAD returning 503 for [] (txn: tx171363c6549a465cb67ca-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx3f4af478f140413fad93d-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: tx599e91b91d774eb591511-00665ca3e4) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: tx599e91b91d774eb591511-00665ca3e4) test INFO: Client disconnected on read of '/a/c' (txn: tx599e91b91d774eb591511-00665ca3e4) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txffcceab129554a9c82099-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx01459a6d45784dc791ed4-00665ca3e4) test INFO: Client disconnected on read of '/a/c' (txn: tx01459a6d45784dc791ed4-00665ca3e4) test INFO: Client disconnected on read of '/a/c/' (txn: tx01459a6d45784dc791ed4-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txa4c687d9ba594a01a6c95-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txa4c687d9ba594a01a6c95-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa4c687d9ba594a01a6c95-00665ca3e4) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txa4c687d9ba594a01a6c95-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txcd40f52c437d40c3a4af8-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txcd40f52c437d40c3a4af8-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txcd40f52c437d40c3a4af8-00665ca3e4) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txcd40f52c437d40c3a4af8-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx21f04002630340c6a5bee-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx21f04002630340c6a5bee-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx21f04002630340c6a5bee-00665ca3e4) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx21f04002630340c6a5bee-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx18d7b54073084bb58c3da-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx18d7b54073084bb58c3da-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx18d7b54073084bb58c3da-00665ca3e4) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx18d7b54073084bb58c3da-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txf488e76eebf14eec97ebd-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf488e76eebf14eec97ebd-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf488e76eebf14eec97ebd-00665ca3e4) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txf488e76eebf14eec97ebd-00665ca3e4) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txce6e1db56cf04279ab172-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txce6e1db56cf04279ab172-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txce6e1db56cf04279ab172-00665ca3e5) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txce6e1db56cf04279ab172-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx5e024980dd75448a927bb-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx5e024980dd75448a927bb-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx5e024980dd75448a927bb-00665ca3e5) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx5e024980dd75448a927bb-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx3ecbbdac7da94f4f8a58c-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx3ecbbdac7da94f4f8a58c-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx3ecbbdac7da94f4f8a58c-00665ca3e5) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx3ecbbdac7da94f4f8a58c-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx0196544db95c436490bd7-00665ca3e5) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: tx0196544db95c436490bd7-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx0196544db95c436490bd7-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx0196544db95c436490bd7-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx0196544db95c436490bd7-00665ca3e5) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx0196544db95c436490bd7-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx289a6c7b62f0413f9de10-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx289a6c7b62f0413f9de10-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx289a6c7b62f0413f9de10-00665ca3e5) test ERROR: Container GET returning 503 for [503, 503] (txn: tx289a6c7b62f0413f9de10-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test ERROR: Container GET returning 503 for [503, 503] (txn: tx049d6f9e96ff4e2aa2295-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx349ee51084774f10886ac-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx349ee51084774f10886ac-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx349ee51084774f10886ac-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx349ee51084774f10886ac-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx04bffbc308864a5bbf677-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx04bffbc308864a5bbf677-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx04bffbc308864a5bbf677-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx04bffbc308864a5bbf677-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx7b96f64d043b489387762-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx7b96f64d043b489387762-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx7b96f64d043b489387762-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx7b96f64d043b489387762-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx48df211b477f4e43981e2-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx48df211b477f4e43981e2-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx48df211b477f4e43981e2-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx48df211b477f4e43981e2-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx281eb2da4af94c6aad41d-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx281eb2da4af94c6aad41d-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx281eb2da4af94c6aad41d-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx281eb2da4af94c6aad41d-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx6847b711ac904362b9566-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx6847b711ac904362b9566-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx6847b711ac904362b9566-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx6847b711ac904362b9566-00665ca3e5) test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting test.unit.proxy.test_mem_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_mem_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_mem_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_get_accept_ranges_header ... ok test ERROR: Failed to unlink '/tmp/tmp9e91_37a/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp9e91_37a/containers/part/suffix/hash/container_1717347347.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp9e91_37a/containers/part/suffix/hash/container_1717347347.00000.db': ['/tmp/tmp9e91_37a/containers/part/suffix/hash/container.db', '/tmp/tmp9e91_37a/containers/part/suffix/hash/container_1717347348.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp9e91_37a/containers/part/suffix/hash/container_1717347348.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjh_vocfp/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/ascii/ascii From Object Server 10.0.0.1:1001/sdb (txn: txcb1bc174d1134df49c4fb-00665ca415) proxy-server ERROR: ERROR 500 Trying to DELETE /AUTH_kilroy/툎/鐉 From Object Server 10.0.0.1:1001/sdb (txn: txcee41e746c80491797b5a-00665ca415) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx582e595596da4497b55f3-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx95e1da3e1386484e82840-00665ca415) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx95e1da3e1386484e82840-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx46c760668c84469182ee0-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx46c760668c84469182ee0-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/ascii/ascii: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: tx46c760668c84469182ee0-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx8cdb1b487af747c19bb74-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx8cdb1b487af747c19bb74-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to DELETE /AUTH_kilroy/툎/鐉: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception (txn: tx8cdb1b487af747c19bb74-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: txc10f882a8a654a7faec40-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_newest_will_not_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_not_found_when_404_newer ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.0.1:6200/sda1_"other meta data" Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"something" Device d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" is now d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"something" Device d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" is now d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"something" Device d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" is now d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"something" Device 1 already uses 127.0.0.2:6201/sda2. Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" is now d4r2z3-test.test2.com:9000Rr.test2.com:9000/sda300_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.2.1:9000R127.0.2.1:9000/sda100_"other meta data" Device d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" is now d0r0z0-127.0.1.1:8000R127.0.1.1:8000/sda10_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[4001:0:1234::c1c0:abcd:876]:9000R[4::10]:9000/sda300_"other meta data" Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 3.14159265359 weight got id 4 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" is now d4r2z3-[3001:0:1234::c1c0:abcd:876]:8000R[3::10]:8000/sda30_"other meta data" Search value matched 0 devices. The on-disk ring builder is unchanged. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The minimum number of hours before a partition can be reassigned is now set to 24 test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_min_part_hours_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload test.unit.cli.test_ringbuilder.TestCommands.test_set_overload ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_negative ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_non_numeric ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input test.unit.cli.test_ringbuilder.TestCommands.test_set_overload_percent_strange_input ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region test.unit.cli.test_ringbuilder.TestCommands.test_set_region ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_region_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_region_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_region_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_region_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_invalid_value ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_replicas_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight test.unit.cli.test_ringbuilder.TestCommands.test_set_weight ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_all_devices ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_read_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_deadlock ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:full_control OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_with_write_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_deadlock ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server ERROR: Object returning 503 for [] (txn: tx56b2db2edb024ac983fc0-00665ca3db) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: txc7c35f222ba84468b20be-00665ca3e1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx924f3a790f4a4d788d4c0-00665ca3e1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reconciler_sync ... ok test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0006 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0006 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0005 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:19 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.1153 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0008 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0006 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:20 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0005 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 1.0175 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0053 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0032 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0015 "-" 3656251 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:21 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0005 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:21 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.4981 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:21 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.4195 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.5890 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0005 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0867 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0152 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0216 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0014 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0015 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:22 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0321 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0222 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0186 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0059 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0054 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0428 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0047 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0022 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_shard_ranges_with_format_header ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_target ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_default ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_maxuploads_over_max_32bit_int ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_s3acl_and_unknown_user ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_save_and_load_record_from_pending_file ... ok test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Setting object count to 1065 (txn: txb9005f5a92d745a8961f7-00665ca3f8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Setting object count to 1065 (txn: tx0e42a4bd809b49fe846e5-00665ca3fb) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txc8680a8ac7b24d23a3f4d-00665ca3fe) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: txc8680a8ac7b24d23a3f4d-00665ca3fe) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: tx591a4ca1b605407eb0a1f-00665ca401) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: tx591a4ca1b605407eb0a1f-00665ca401) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: tx591a4ca1b605407eb0a1f-00665ca401) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx2340533d30a14ad585a8e-00665ca401) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx2340533d30a14ad585a8e-00665ca401) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2340533d30a14ad585a8e-00665ca401) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: tx2340533d30a14ad585a8e-00665ca401) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx289433b7f3d84ccfae130-00665ca401) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx289433b7f3d84ccfae130-00665ca401) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx289433b7f3d84ccfae130-00665ca401) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx289433b7f3d84ccfae130-00665ca401) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: Setting object count to 3120 (txn: txc16678da80fe4cd38c2ec-00665ca405) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: Setting object count to 3120 (txn: tx5435e90752cb4c128cb45-00665ca405) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: Setting object count to 3120 (txn: tx99f126004b0f4bdfb12a3-00665ca405) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: Setting object count to 3120 (txn: txfec9cce2d89949a187eac-00665ca405) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: Setting object count to 3120 (txn: tx360c61b3531942e29c937-00665ca405) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx0e7eb32b50514581bcf2b-00665ca405) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: tx7beda59cae3640469ecf1-00665ca405) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf5fff4349a7e485b83fd9-00665ca405) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_save_and_load_record_from_pending_file ... ok test WARNING: /tmp/tmph6lppao_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (6, '08198fb8aa2801fce836f837a6c19e48', 'ca426d3c-8c40-49e3-8c74-3534c3789d8c-sda', '1717347304.85522', '1717347304.87839', '0', '', '1717347304.84994', 6, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.sharded_a/sr-a', 'timestamp': '1717347304.89745', 'lower': '', 'upper': 'a', 'object_count': 101, 'bytes_used': 1010, 'meta_timestamp': '1717347304.89745', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347304.89745', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347304.90983', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347304.90983', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347304.90948', 'epoch': '1717347304.90948', 'reported': 0, 'tombstones': -1}], 'ca426d3c-8c40-49e3-8c74-3534c3789d8c-sda') test DEBUG: /tmp/tmph6lppao_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 2 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmph6lppao_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '7d22209251b61225785e9ddee7dba7b6', '06eff3b9-c217-4d5e-93ca-663a195cc8ee-sda', '1717347310.57205', '1717347310.56644', '0', '', '1717347310.56644', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('06eff3b9-c217-4d5e-93ca-663a195cc8ee-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmp9b3fj40a/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, 'c6283a5484df4116ddcadc9adb6d4e1d', '6fafb14b-7758-4954-a867-b9ff719e9a64-sda', '1717347322.65450', '1717347322.44349', '0', '', '1717347322.44349', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('6fafb14b-7758-4954-a867-b9ff719e9a64-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '3c0fd9e279afe4b5b3693bfd12404bdf', '7527e3b1-373d-4b35-a853-05c0158852f1-sda', '1717347323.15285', '1717347323.14738', '0', '', '1717347323.14738', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('7527e3b1-373d-4b35-a853-05c0158852f1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Unlinked retiring db '/tmp/tmpeg_19302/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (3, '3c0fd9e279afe4b5b3693bfd12404bdf', '7e7fb85b-7bd3-42df-bb53-14baa895d834-sda', '1717347323.60513', '1717347323.59679', '0', '', '1717347323.59679', 3, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('7e7fb85b-7bd3-42df-bb53-14baa895d834-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c789ee48-8215-4247-bd18-4098ed8e6dab-sda', '1717347323.95698', '1717347323.94741', '0', '', '1717347323.94741', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpdbzipi_p/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '04b084e1-7831-47ce-9864-37e8b2382f05-sdb', '1717347324.76355', '1717347325.00000', '0', '', '1717347325.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04b084e1-7831-47ce-9864-37e8b2382f05-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '04b084e1-7831-47ce-9864-37e8b2382f05-sdb', '1717347324.76355', '1717347325.00000', '0', '', '1717347325.00000', 0, 0) test ERROR: Quarantining DB /tmp/tmpdbzipi_p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04b084e1-7831-47ce-9864-37e8b2382f05-sdb', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08361 seconds (11.96017/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpg9l16rra/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fea2b24d-1bff-48d8-afff-3296a3324c99-sda', '1717347324.88813', '1717347324', '0', '', '1717347324', 0, 0) test DEBUG: /tmp/tmpg9l16rra/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'fea2b24d-1bff-48d8-afff-3296a3324c99-sda', '1717347324.88813', '1717347324', '0', '', '1717347324', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fea2b24d-1bff-48d8-afff-3296a3324c99-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08389 seconds (11.92040/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmph7lr0qjv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '15092ae257a4f7776ea2e9d43738588a', '2ed327e1-48c7-4f84-8525-962a5b7ea168-sda', '1717347325.03807', '1717347325', '0', '', '1717347325', 1, 1) test DEBUG: /tmp/tmph7lr0qjv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347327.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '2ed327e1-48c7-4f84-8525-962a5b7ea168-sda') test DEBUG: /tmp/tmph7lr0qjv/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0163739s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '15092ae257a4f7776ea2e9d43738588a', '2ed327e1-48c7-4f84-8525-962a5b7ea168-sda', '1717347325.03807', '1717347325', '0', '', '1717347325', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2ed327e1-48c7-4f84-8525-962a5b7ea168-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.16007 seconds (6.24723/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp924t6tly/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a453dba3-3e31-46ca-9199-abe313e85806-sda', '1717347325.25731', '1717347325', '0', '', '1717347325', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_newer_remote_delete ... ok test ERROR: Problem finding shard upper in '/tmp/tmp76krc2lp/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp76krc2lp/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpyssbx2z1/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpyssbx2z1/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state ... ok test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx0621153c8260490b920b3-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx0621153c8260490b920b3-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx0621153c8260490b920b3-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx0621153c8260490b920b3-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txacfeb90f4db14624a4ebb-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txacfeb90f4db14624a4ebb-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txacfeb90f4db14624a4ebb-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: txacfeb90f4db14624a4ebb-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx98b039f57b7144009ddf2-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx98b039f57b7144009ddf2-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx98b039f57b7144009ddf2-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx98b039f57b7144009ddf2-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx04dd410dec934ba6acfdb-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx04dd410dec934ba6acfdb-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx04dd410dec934ba6acfdb-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx04dd410dec934ba6acfdb-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx65d6ba5ecfe444fea675c-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx65d6ba5ecfe444fea675c-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx65d6ba5ecfe444fea675c-00665ca3e5) test ERROR: Container GET returning 503 for [] (txn: tx65d6ba5ecfe444fea675c-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txa4f5f19ee65c42998a831-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txa4f5f19ee65c42998a831-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txa4f5f19ee65c42998a831-00665ca3e5) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txa4f5f19ee65c42998a831-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd774686c1de249ce94f9f-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd774686c1de249ce94f9f-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd774686c1de249ce94f9f-00665ca3e5) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txd774686c1de249ce94f9f-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx4547053d6e48407c8fa4a-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx4547053d6e48407c8fa4a-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx4547053d6e48407c8fa4a-00665ca3e5) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx4547053d6e48407c8fa4a-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txb3dfc96b79bf476089a3c-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txb3dfc96b79bf476089a3c-00665ca3e5) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txb3dfc96b79bf476089a3c-00665ca3e5) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txb3dfc96b79bf476089a3c-00665ca3e5) test INFO: Client disconnected on read of '/a/c' (txn: txb274140f42ff4a499c864-00665ca3e5) test INFO: Client disconnected on read of '/a/c/' (txn: txb274140f42ff4a499c864-00665ca3e5) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9acdd684986648f8bf149-00665ca3e5) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9acdd684986648f8bf149-00665ca3e5) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9acdd684986648f8bf149-00665ca3e5) test ERROR: Account HEAD returning 503 for [] (txn: tx9acdd684986648f8bf149-00665ca3e5) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txc37728f097ba46d9bf7c1-00665ca3e5) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txc37728f097ba46d9bf7c1-00665ca3e5) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txc37728f097ba46d9bf7c1-00665ca3e5) test ERROR: Account HEAD returning 503 for [] (txn: txc37728f097ba46d9bf7c1-00665ca3e5) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txe108df79faba4c98bf1f2-00665ca3e9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx3ecfb1715d4b419eb4c75-00665ca3e9) test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers test.unit.proxy.test_mem_server.TestContainerController.test_transfer_headers ... ok test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_mem_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_sharding_state_errors ... ok test ERROR: Problem finding shard upper in '/tmp/tmpht9zr172/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpht9zr172/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpp85zrvfk/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpp85zrvfk/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_set_x_syncs ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:51 -0000', 'Content-Md5': 'ud0C7/g6K1j77RpgTDYdwQ==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'b9dd02eff83a2b58fbed1a604c361dc1', 'Content-Length': '120'} test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission test.unit.common.middleware.s3api.test_multi_delete.TestS3ApiMultiDeleteAcl.test_object_multi_DELETE_without_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_cors_preflight ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sharding_sysmeta ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txfeddaea03f684d3ca0635-00665ca3ef) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txcaaef6871be848ed9bfda-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txcaaef6871be848ed9bfda-00665ca415) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txcaaef6871be848ed9bfda-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: txcaaef6871be848ed9bfda-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: my-txn-id) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txc2969e3fc72142dc80241-00665ca415) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.0:1000/sda (txn: txc2969e3fc72142dc80241-00665ca415) proxy-server ERROR: ERROR 503 b'' Trying to GET /a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txc2969e3fc72142dc80241-00665ca415) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txc2969e3fc72142dc80241-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txf845d919df9342fbadc36-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx495c566a24c945a1819b7-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx495c566a24c945a1819b7-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: tx495c566a24c945a1819b7-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx4bed0573053142cfa443e-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx4bed0573053142cfa443e-00665ca415) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_error_during_rebalance ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. The overload factor is now 19.88% (0.198780) The change will take effect after the next rebalance. Overload must be non-negative. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. 'swedish fish' is not a valid number. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 d0r314z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r314z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" region set to 314 swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. The replica count is now 3.141593. The change will take effect after the next rebalance. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. "test" is not a valid number. Replica count must be at least 1. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Matched more than one device: d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 200.0 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 200.0 d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 200.0 d3r3z3-127.0.0.4:6203R127.0.0.4:6203/sdd4_"" weight set to 200.0 test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_no_matching ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_old_format_two_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_weight_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone test.unit.cli.test_ringbuilder.TestCommands.test_set_zone ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_domain_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv4_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_new_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_ipv6_old_format ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_no_matching ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:50 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_keystone ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_s3acl_and_unknown_user ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:write OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other OWNER test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_DELETE_without_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_all_users ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0758 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0167 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0029 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0002 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0004 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:23 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0138 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0233 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0252 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0067 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0028 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0025 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmpi2rvqzli/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0081 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0047 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0040 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0197 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0252 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0134 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0184 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0029 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0247 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0298 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0214 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0229 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0085 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0109 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0336 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0053 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0140 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0351 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0106 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0239 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0020 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:23 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0028 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0210 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0304 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0156 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0028 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0019 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.2465 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.4065 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0156 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0076 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0058 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_GET_xml ... ok test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: txf5fff4349a7e485b83fd9-00665ca405) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx48db1395185d4f70849b7-00665ca405) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx48db1395185d4f70849b7-00665ca405) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx48db1395185d4f70849b7-00665ca405) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx48db1395185d4f70849b7-00665ca405) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx6af0bd70b16344e6b8b61-00665ca406) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx6af0bd70b16344e6b8b61-00665ca406) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx6af0bd70b16344e6b8b61-00665ca406) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx6af0bd70b16344e6b8b61-00665ca406) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: tx51336996c061463e92fa9-00665ca406) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx51336996c061463e92fa9-00665ca406) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx51336996c061463e92fa9-00665ca406) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx51336996c061463e92fa9-00665ca406) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: Setting object count to 1068 (txn: tx916c583df2084708b1c50-00665ca409) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Setting object count to 1068 (txn: txab4248d597ff4f3fb986d-00665ca409) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx85f2593d65904b34962a3-00665ca40c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx85f2593d65904b34962a3-00665ca40c) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx85f2593d65904b34962a3-00665ca40c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx85f2593d65904b34962a3-00665ca40c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx85f2593d65904b34962a3-00665ca40c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx85f2593d65904b34962a3-00665ca40c) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_errors_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_between_shards ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_OPTIONS ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_no_timestamp_update ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_negative_maxuploads ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_str_maxuploads ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:06 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_GET_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_state_is_not_reclaimable ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test ERROR: 400 Bad Request test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_invalid_uri_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read ... ok test DEBUG: /tmp/tmp924t6tly/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a453dba3-3e31-46ca-9199-abe313e85806-sda', '1717347325.25731', '1717347325', '0', '', '1717347325', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a453dba3-3e31-46ca-9199-abe313e85806-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13554 seconds (7.37806/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpz2b9_jov/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '15092ae257a4f7776ea2e9d43738588a', '028c8a91-e9e2-4996-ba31-91117b166afa-sda', '1717347325.43695', '1717347325', '0', '', '1717347325', 1, 1) test DEBUG: /tmp/tmpz2b9_jov/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '15092ae257a4f7776ea2e9d43738588a', '028c8a91-e9e2-4996-ba31-91117b166afa-sda', '1717347325.43695', '1717347325', '0', '', '1717347325', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('028c8a91-e9e2-4996-ba31-91117b166afa-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14373 seconds (6.95735/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpan1l4ieb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '083656b2-b41a-4bd2-8a7e-4e881da03465-sda', '1717347325.64031', '1717347325', '0', '', '1717347325', 0, 0) test DEBUG: /tmp/tmpan1l4ieb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '083656b2-b41a-4bd2-8a7e-4e881da03465-sda', '1717347325.64031', '1717347325', '0', '', '1717347325', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('083656b2-b41a-4bd2-8a7e-4e881da03465-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09421 seconds (10.61445/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpx9bkcr43/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'de6ad92b88a7c43fae9279c8737707e6', '7daedaef-0373-4e0a-ab0f-60ae237e30de-sda', '1717347325.80760', '1717347325', '0', '', '1717347325', 1, 1) test DEBUG: /tmp/tmpx9bkcr43/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347328.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '7daedaef-0373-4e0a-ab0f-60ae237e30de-sda') test DEBUG: /tmp/tmpx9bkcr43/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0138586s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'de6ad92b88a7c43fae9279c8737707e6', '7daedaef-0373-4e0a-ab0f-60ae237e30de-sda', '1717347325.80760', '1717347325', '0', '', '1717347325', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7daedaef-0373-4e0a-ab0f-60ae237e30de-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.11506 seconds (8.69078/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpkltkpeu7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ba92c12d-692c-49fe-907d-f9d721dedd4e-sda', '1717347325.99608', '1717347325', '0', '', '1717347325', 0, 1) test DEBUG: /tmp/tmpkltkpeu7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'ba92c12d-692c-49fe-907d-f9d721dedd4e-sda', '1717347325.99608', '1717347325', '0', '', '1717347325', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ba92c12d-692c-49fe-907d-f9d721dedd4e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10620 seconds (9.41617/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpa83cwjyl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e6458dd00b139b9a31c987da93852f90', 'f8572153-0aa5-4394-82ff-e4395a083b57-sda', '1717347326.15262', '1717347326', '0', '', '1717347326', 1, 1) test DEBUG: /tmp/tmpa83cwjyl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e6458dd00b139b9a31c987da93852f90', 'f8572153-0aa5-4394-82ff-e4395a083b57-sda', '1717347326.15262', '1717347326', '0', '', '1717347326', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f8572153-0aa5-4394-82ff-e4395a083b57-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.21240 seconds (4.70819/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmps187xupl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7504b8c1-7fee-4201-af11-d61383a00f52-sda', '1717347326.47799', '1717347328', '0', '', '1717347328', 0, 1) test DEBUG: /tmp/tmps187xupl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7504b8c1-7fee-4201-af11-d61383a00f52-sda', '1717347326.47799', '1717347328', '0', '', '1717347328', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('7504b8c1-7fee-4201-af11-d61383a00f52-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10092 seconds (9.90911/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_create_policy_over_older_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_half_delete_policy_over_newer_remote_create ... ok test-reaper ERROR: Exception with 10.10.10.1:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.2:6202/sda1: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.3:6202/None: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.10.10.1:6202/sda2: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 414, in reap_container direct_delete_container( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 167, in fake_direct_delete_container raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Exception with 10.0.0.0:1000/sda: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 490, in reap_object direct_delete_object( File "/build/reproducible-path/swift-2.33.0/test/unit/account/test_reaper.py", line 160, in fake_direct_delete_object raise self.myexp swift.common.exceptions.ClientException: : 404 test-reaper ERROR: Timeout Exception with 10.0.0.0:1000/sda test-reaper ERROR: Timeout Exception with 10.0.0.1:1001/sdb test-reaper ERROR: Timeout Exception with 10.0.0.2:1002/sdc test-reaper ERROR: Timeout Exception with 10.0.0.3:1003/sdd test-reaper DEBUG: Begin devices pass: thisdeviceisbad test-reaper ERROR: Exception in top-level account reaper loop: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/account/reaper.py", line 135, in run_once for device in os.listdir(self.devices): ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'thisdeviceisbad' test-reaper INFO: Devices pass completed: 0.00s test INFO: Disabling account_sdb test INFO: Disabling container_sdb test INFO: Disabling object_sdb test INFO: Enabling account_sdb test INFO: Enabling container_sdb test INFO: Enabling object_sdb =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 5 hosts (object) =============================================================================== =============================================================================== --> Starting reconnaissance on 2 hosts (object) =============================================================================== DEBUG: New builder; performing initial balance New builder; performing initial balance DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 768 new parts and 0 removed parts from replica-count change 768 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 87 (Last start was 0) Gather start is 87 (Last start was 0) DEBUG: Gathered 768 parts Gathered 768 parts DEBUG: Placed 115/0 onto dev r1z1-10.1.1.1/sdc Placed 115/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 115/1 onto dev r1z1-10.1.1.1/sdd Placed 115/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 115/2 onto dev r1z1-10.1.1.1/sda Placed 115/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 8/0 onto dev r1z1-10.1.1.1/sdb Placed 8/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 8/1 onto dev r1z1-10.1.1.1/sdc Placed 8/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 8/2 onto dev r1z1-10.1.1.1/sdd Placed 8/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 65/0 onto dev r1z1-10.1.1.1/sda Placed 65/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 65/1 onto dev r1z1-10.1.1.1/sdb Placed 65/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 65/2 onto dev r1z1-10.1.1.1/sdc Placed 65/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 170/0 onto dev r1z1-10.1.1.1/sdd Placed 170/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 170/1 onto dev r1z1-10.1.1.1/sda Placed 170/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 170/2 onto dev r1z1-10.1.1.1/sdb Placed 170/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 208/0 onto dev r1z1-10.1.1.1/sdc Placed 208/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 208/1 onto dev r1z1-10.1.1.1/sdd Placed 208/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 208/2 onto dev r1z1-10.1.1.1/sda Placed 208/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 28/0 onto dev r1z1-10.1.1.1/sdb Placed 28/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 28/1 onto dev r1z1-10.1.1.1/sdc Placed 28/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 28/2 onto dev r1z1-10.1.1.1/sdd Placed 28/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 101/0 onto dev r1z1-10.1.1.1/sda Placed 101/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 101/1 onto dev r1z1-10.1.1.1/sdb Placed 101/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 101/2 onto dev r1z1-10.1.1.1/sdc Placed 101/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 108/0 onto dev r1z1-10.1.1.1/sdd Placed 108/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 108/1 onto dev r1z1-10.1.1.1/sda Placed 108/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 108/2 onto dev r1z1-10.1.1.1/sdb Placed 108/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 89/0 onto dev r1z1-10.1.1.1/sdc Placed 89/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 89/1 onto dev r1z1-10.1.1.1/sdd Placed 89/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 89/2 onto dev r1z1-10.1.1.1/sda Placed 89/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 127/0 onto dev r1z1-10.1.1.1/sdb Placed 127/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 127/1 onto dev r1z1-10.1.1.1/sdc Placed 127/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 127/2 onto dev r1z1-10.1.1.1/sdd Placed 127/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 203/0 onto dev r1z1-10.1.1.1/sda Placed 203/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 203/1 onto dev r1z1-10.1.1.1/sdb Placed 203/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 203/2 onto dev r1z1-10.1.1.1/sdc Placed 203/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 75/0 onto dev r1z1-10.1.1.1/sdd Placed 75/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 75/1 onto dev r1z1-10.1.1.1/sda Placed 75/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 75/2 onto dev r1z1-10.1.1.1/sdb Placed 75/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 155/0 onto dev r1z1-10.1.1.1/sdc Placed 155/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 155/1 onto dev r1z1-10.1.1.1/sdd Placed 155/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 155/2 onto dev r1z1-10.1.1.1/sda Placed 155/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 72/0 onto dev r1z1-10.1.1.1/sdb Placed 72/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 72/1 onto dev r1z1-10.1.1.1/sdc Placed 72/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 72/2 onto dev r1z1-10.1.1.1/sdd Placed 72/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 142/0 onto dev r1z1-10.1.1.1/sda Placed 142/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 142/1 onto dev r1z1-10.1.1.1/sdb Placed 142/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 142/2 onto dev r1z1-10.1.1.1/sdc Placed 142/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 252/0 onto dev r1z1-10.1.1.1/sdd Placed 252/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 252/1 onto dev r1z1-10.1.1.1/sda Placed 252/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 252/2 onto dev r1z1-10.1.1.1/sdb Placed 252/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 180/0 onto dev r1z1-10.1.1.1/sdc Placed 180/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 180/1 onto dev r1z1-10.1.1.1/sdd Placed 180/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 180/2 onto dev r1z1-10.1.1.1/sda Placed 180/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 118/0 onto dev r1z1-10.1.1.1/sdb Placed 118/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 118/1 onto dev r1z1-10.1.1.1/sdc Placed 118/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 118/2 onto dev r1z1-10.1.1.1/sdd Placed 118/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 244/0 onto dev r1z1-10.1.1.1/sda Placed 244/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 244/1 onto dev r1z1-10.1.1.1/sdb Placed 244/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 244/2 onto dev r1z1-10.1.1.1/sdc Placed 244/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 177/0 onto dev r1z1-10.1.1.1/sdd Placed 177/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 177/1 onto dev r1z1-10.1.1.1/sda Placed 177/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 177/2 onto dev r1z1-10.1.1.1/sdb Placed 177/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 26/0 onto dev r1z1-10.1.1.1/sdc Placed 26/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 26/1 onto dev r1z1-10.1.1.1/sdd Placed 26/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 26/2 onto dev r1z1-10.1.1.1/sda Placed 26/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 150/0 onto dev r1z1-10.1.1.1/sdb Placed 150/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 150/1 onto dev r1z1-10.1.1.1/sdc Placed 150/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 150/2 onto dev r1z1-10.1.1.1/sdd Placed 150/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 103/0 onto dev r1z1-10.1.1.1/sda Placed 103/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 103/1 onto dev r1z1-10.1.1.1/sdb Placed 103/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 103/2 onto dev r1z1-10.1.1.1/sdc Placed 103/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 92/0 onto dev r1z1-10.1.1.1/sdd Placed 92/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 92/1 onto dev r1z1-10.1.1.1/sda Placed 92/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 92/2 onto dev r1z1-10.1.1.1/sdb Placed 92/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 159/0 onto dev r1z1-10.1.1.1/sdc Placed 159/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 159/1 onto dev r1z1-10.1.1.1/sdd Placed 159/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 159/2 onto dev r1z1-10.1.1.1/sda Placed 159/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 138/0 onto dev r1z1-10.1.1.1/sdb Placed 138/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 138/1 onto dev r1z1-10.1.1.1/sdc Placed 138/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 138/2 onto dev r1z1-10.1.1.1/sdd Placed 138/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 76/0 onto dev r1z1-10.1.1.1/sda Placed 76/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 76/1 onto dev r1z1-10.1.1.1/sdb Placed 76/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 76/2 onto dev r1z1-10.1.1.1/sdc Placed 76/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 229/0 onto dev r1z1-10.1.1.1/sdd Placed 229/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 229/1 onto dev r1z1-10.1.1.1/sda Placed 229/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 229/2 onto dev r1z1-10.1.1.1/sdb Placed 229/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 13/0 onto dev r1z1-10.1.1.1/sdc Placed 13/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 13/1 onto dev r1z1-10.1.1.1/sdd Placed 13/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 13/2 onto dev r1z1-10.1.1.1/sda Placed 13/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 240/0 onto dev r1z1-10.1.1.1/sdb Placed 240/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 240/1 onto dev r1z1-10.1.1.1/sdc Placed 240/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 240/2 onto dev r1z1-10.1.1.1/sdd Placed 240/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 171/0 onto dev r1z1-10.1.1.1/sda Placed 171/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 171/1 onto dev r1z1-10.1.1.1/sdb Placed 171/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 171/2 onto dev r1z1-10.1.1.1/sdc Placed 171/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 10/0 onto dev r1z1-10.1.1.1/sdd Placed 10/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 10/1 onto dev r1z1-10.1.1.1/sda Placed 10/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 10/2 onto dev r1z1-10.1.1.1/sdb Placed 10/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 251/0 onto dev r1z1-10.1.1.1/sdc Placed 251/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 251/1 onto dev r1z1-10.1.1.1/sdd Placed 251/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 251/2 onto dev r1z1-10.1.1.1/sda Placed 251/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 173/0 onto dev r1z1-10.1.1.1/sdb Placed 173/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 173/1 onto dev r1z1-10.1.1.1/sdc Placed 173/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 173/2 onto dev r1z1-10.1.1.1/sdd Placed 173/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 45/0 onto dev r1z1-10.1.1.1/sda Placed 45/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 45/1 onto dev r1z1-10.1.1.1/sdb Placed 45/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 45/2 onto dev r1z1-10.1.1.1/sdc Placed 45/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 84/0 onto dev r1z1-10.1.1.1/sdd Placed 84/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 84/1 onto dev r1z1-10.1.1.1/sda Placed 84/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 84/2 onto dev r1z1-10.1.1.1/sdb Placed 84/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 225/0 onto dev r1z1-10.1.1.1/sdc Placed 225/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 225/1 onto dev r1z1-10.1.1.1/sdd Placed 225/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 225/2 onto dev r1z1-10.1.1.1/sda Placed 225/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 217/0 onto dev r1z1-10.1.1.1/sdb Placed 217/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 217/1 onto dev r1z1-10.1.1.1/sdc Placed 217/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 217/2 onto dev r1z1-10.1.1.1/sdd Placed 217/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 17/0 onto dev r1z1-10.1.1.1/sda Placed 17/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 17/1 onto dev r1z1-10.1.1.1/sdb Placed 17/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 17/2 onto dev r1z1-10.1.1.1/sdc Placed 17/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 255/0 onto dev r1z1-10.1.1.1/sdd Placed 255/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 255/1 onto dev r1z1-10.1.1.1/sda Placed 255/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 255/2 onto dev r1z1-10.1.1.1/sdb Placed 255/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 9/0 onto dev r1z1-10.1.1.1/sdc Placed 9/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 9/1 onto dev r1z1-10.1.1.1/sdd Placed 9/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 9/2 onto dev r1z1-10.1.1.1/sda Placed 9/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 78/0 onto dev r1z1-10.1.1.1/sdb Placed 78/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 78/1 onto dev r1z1-10.1.1.1/sdc Placed 78/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 78/2 onto dev r1z1-10.1.1.1/sdd Placed 78/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 215/0 onto dev r1z1-10.1.1.1/sda Placed 215/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 215/1 onto dev r1z1-10.1.1.1/sdb Placed 215/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 215/2 onto dev r1z1-10.1.1.1/sdc Placed 215/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 114/0 onto dev r1z1-10.1.1.1/sdd Placed 114/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 114/1 onto dev r1z1-10.1.1.1/sda Placed 114/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 114/2 onto dev r1z1-10.1.1.1/sdb Placed 114/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 56/0 onto dev r1z1-10.1.1.1/sdc Placed 56/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 56/1 onto dev r1z1-10.1.1.1/sdd Placed 56/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 56/2 onto dev r1z1-10.1.1.1/sda Placed 56/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 121/0 onto dev r1z1-10.1.1.1/sdb Placed 121/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 121/1 onto dev r1z1-10.1.1.1/sdc Placed 121/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 121/2 onto dev r1z1-10.1.1.1/sdd Placed 121/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 235/0 onto dev r1z1-10.1.1.1/sda Placed 235/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 235/1 onto dev r1z1-10.1.1.1/sdb Placed 235/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 235/2 onto dev r1z1-10.1.1.1/sdc Placed 235/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 88/0 onto dev r1z1-10.1.1.1/sdd Placed 88/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 88/1 onto dev r1z1-10.1.1.1/sda Placed 88/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 88/2 onto dev r1z1-10.1.1.1/sdb Placed 88/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 102/0 onto dev r1z1-10.1.1.1/sdc Placed 102/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 102/1 onto dev r1z1-10.1.1.1/sdd Placed 102/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 102/2 onto dev r1z1-10.1.1.1/sda Placed 102/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 211/0 onto dev r1z1-10.1.1.1/sdb Placed 211/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 211/1 onto dev r1z1-10.1.1.1/sdc Placed 211/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 211/2 onto dev r1z1-10.1.1.1/sdd Placed 211/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 206/0 onto dev r1z1-10.1.1.1/sda Placed 206/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 206/1 onto dev r1z1-10.1.1.1/sdb Placed 206/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 206/2 onto dev r1z1-10.1.1.1/sdc Placed 206/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 129/0 onto dev r1z1-10.1.1.1/sdd Placed 129/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 129/1 onto dev r1z1-10.1.1.1/sda Placed 129/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 129/2 onto dev r1z1-10.1.1.1/sdb Placed 129/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 20/0 onto dev r1z1-10.1.1.1/sdc Placed 20/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 20/1 onto dev r1z1-10.1.1.1/sdd Placed 20/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 20/2 onto dev r1z1-10.1.1.1/sda Placed 20/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 71/0 onto dev r1z1-10.1.1.1/sdb Placed 71/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 71/1 onto dev r1z1-10.1.1.1/sdc Placed 71/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 71/2 onto dev r1z1-10.1.1.1/sdd Placed 71/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 122/0 onto dev r1z1-10.1.1.1/sda Placed 122/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 122/1 onto dev r1z1-10.1.1.1/sdb Placed 122/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 122/2 onto dev r1z1-10.1.1.1/sdc Placed 122/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 174/0 onto dev r1z1-10.1.1.1/sdd Placed 174/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 174/1 onto dev r1z1-10.1.1.1/sda Placed 174/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 174/2 onto dev r1z1-10.1.1.1/sdb Placed 174/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 79/0 onto dev r1z1-10.1.1.1/sdc Placed 79/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 79/1 onto dev r1z1-10.1.1.1/sdd Placed 79/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 79/2 onto dev r1z1-10.1.1.1/sda Placed 79/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 195/0 onto dev r1z1-10.1.1.1/sdb Placed 195/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 195/1 onto dev r1z1-10.1.1.1/sdc Placed 195/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 195/2 onto dev r1z1-10.1.1.1/sdd Placed 195/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 3/0 onto dev r1z1-10.1.1.1/sda Placed 3/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 3/1 onto dev r1z1-10.1.1.1/sdb Placed 3/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 3/2 onto dev r1z1-10.1.1.1/sdc Placed 3/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 167/0 onto dev r1z1-10.1.1.1/sdd Placed 167/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 167/1 onto dev r1z1-10.1.1.1/sda Placed 167/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 167/2 onto dev r1z1-10.1.1.1/sdb Placed 167/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 31/0 onto dev r1z1-10.1.1.1/sdc Placed 31/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 31/1 onto dev r1z1-10.1.1.1/sdd Placed 31/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 31/2 onto dev r1z1-10.1.1.1/sda Placed 31/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 136/0 onto dev r1z1-10.1.1.1/sdb Placed 136/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 136/1 onto dev r1z1-10.1.1.1/sdc Placed 136/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 136/2 onto dev r1z1-10.1.1.1/sdd Placed 136/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 82/0 onto dev r1z1-10.1.1.1/sda Placed 82/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 82/1 onto dev r1z1-10.1.1.1/sdb Placed 82/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 82/2 onto dev r1z1-10.1.1.1/sdc Placed 82/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 74/0 onto dev r1z1-10.1.1.1/sdd Placed 74/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 74/1 onto dev r1z1-10.1.1.1/sda Placed 74/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 74/2 onto dev r1z1-10.1.1.1/sdb Placed 74/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 184/0 onto dev r1z1-10.1.1.1/sdc Placed 184/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 184/1 onto dev r1z1-10.1.1.1/sdd Placed 184/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 184/2 onto dev r1z1-10.1.1.1/sda Placed 184/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 44/0 onto dev r1z1-10.1.1.1/sdb Placed 44/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 44/1 onto dev r1z1-10.1.1.1/sdc Placed 44/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 44/2 onto dev r1z1-10.1.1.1/sdd Placed 44/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 137/0 onto dev r1z1-10.1.1.1/sda Placed 137/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 137/1 onto dev r1z1-10.1.1.1/sdb Placed 137/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 137/2 onto dev r1z1-10.1.1.1/sdc Placed 137/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 141/0 onto dev r1z1-10.1.1.1/sdd Placed 141/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 141/1 onto dev r1z1-10.1.1.1/sda Placed 141/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 141/2 onto dev r1z1-10.1.1.1/sdb Placed 141/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 140/0 onto dev r1z1-10.1.1.1/sdc Placed 140/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 140/1 onto dev r1z1-10.1.1.1/sdd Placed 140/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 140/2 onto dev r1z1-10.1.1.1/sda Placed 140/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 70/0 onto dev r1z1-10.1.1.1/sdb Placed 70/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 70/1 onto dev r1z1-10.1.1.1/sdc Placed 70/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 70/2 onto dev r1z1-10.1.1.1/sdd Placed 70/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 6/0 onto dev r1z1-10.1.1.1/sda Placed 6/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 6/1 onto dev r1z1-10.1.1.1/sdb Placed 6/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 6/2 onto dev r1z1-10.1.1.1/sdc Placed 6/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 80/0 onto dev r1z1-10.1.1.1/sdd Placed 80/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 80/1 onto dev r1z1-10.1.1.1/sda Placed 80/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 80/2 onto dev r1z1-10.1.1.1/sdb Placed 80/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 34/0 onto dev r1z1-10.1.1.1/sdc Placed 34/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 34/1 onto dev r1z1-10.1.1.1/sdd Placed 34/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 34/2 onto dev r1z1-10.1.1.1/sda Placed 34/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 55/0 onto dev r1z1-10.1.1.1/sdb Placed 55/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 55/1 onto dev r1z1-10.1.1.1/sdc Placed 55/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 55/2 onto dev r1z1-10.1.1.1/sdd Placed 55/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 5/0 onto dev r1z1-10.1.1.1/sda Placed 5/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 5/1 onto dev r1z1-10.1.1.1/sdb Placed 5/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 5/2 onto dev r1z1-10.1.1.1/sdc Placed 5/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 191/0 onto dev r1z1-10.1.1.1/sdd Placed 191/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 191/1 onto dev r1z1-10.1.1.1/sda Placed 191/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 191/2 onto dev r1z1-10.1.1.1/sdb Placed 191/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 85/0 onto dev r1z1-10.1.1.1/sdc Placed 85/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 85/1 onto dev r1z1-10.1.1.1/sdd Placed 85/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 85/2 onto dev r1z1-10.1.1.1/sda Placed 85/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 156/0 onto dev r1z1-10.1.1.1/sdb Placed 156/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 156/1 onto dev r1z1-10.1.1.1/sdc Placed 156/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 156/2 onto dev r1z1-10.1.1.1/sdd Placed 156/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 61/0 onto dev r1z1-10.1.1.1/sda Placed 61/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 61/1 onto dev r1z1-10.1.1.1/sdb Placed 61/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 61/2 onto dev r1z1-10.1.1.1/sdc Placed 61/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 164/0 onto dev r1z1-10.1.1.1/sdd Placed 164/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 164/1 onto dev r1z1-10.1.1.1/sda Placed 164/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 164/2 onto dev r1z1-10.1.1.1/sdb Placed 164/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 246/0 onto dev r1z1-10.1.1.1/sdc Placed 246/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 246/1 onto dev r1z1-10.1.1.1/sdd Placed 246/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 246/2 onto dev r1z1-10.1.1.1/sda Placed 246/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 190/0 onto dev r1z1-10.1.1.1/sdb Placed 190/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 190/1 onto dev r1z1-10.1.1.1/sdc Placed 190/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 190/2 onto dev r1z1-10.1.1.1/sdd Placed 190/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 126/0 onto dev r1z1-10.1.1.1/sda Placed 126/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 126/1 onto dev r1z1-10.1.1.1/sdb Placed 126/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 126/2 onto dev r1z1-10.1.1.1/sdc Placed 126/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 69/0 onto dev r1z1-10.1.1.1/sdd Placed 69/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 69/1 onto dev r1z1-10.1.1.1/sda Placed 69/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 69/2 onto dev r1z1-10.1.1.1/sdb Placed 69/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 40/0 onto dev r1z1-10.1.1.1/sdc Placed 40/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 40/1 onto dev r1z1-10.1.1.1/sdd Placed 40/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 40/2 onto dev r1z1-10.1.1.1/sda Placed 40/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 165/0 onto dev r1z1-10.1.1.1/sdb Placed 165/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 165/1 onto dev r1z1-10.1.1.1/sdc Placed 165/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 165/2 onto dev r1z1-10.1.1.1/sdd Placed 165/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 220/0 onto dev r1z1-10.1.1.1/sda Placed 220/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 220/1 onto dev r1z1-10.1.1.1/sdb Placed 220/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 220/2 onto dev r1z1-10.1.1.1/sdc Placed 220/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 169/0 onto dev r1z1-10.1.1.1/sdd Placed 169/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 169/1 onto dev r1z1-10.1.1.1/sda Placed 169/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 169/2 onto dev r1z1-10.1.1.1/sdb Placed 169/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 91/0 onto dev r1z1-10.1.1.1/sdc Placed 91/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 91/1 onto dev r1z1-10.1.1.1/sdd Placed 91/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 91/2 onto dev r1z1-10.1.1.1/sda Placed 91/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 232/0 onto dev r1z1-10.1.1.1/sdb Placed 232/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 232/1 onto dev r1z1-10.1.1.1/sdc Placed 232/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 232/2 onto dev r1z1-10.1.1.1/sdd Placed 232/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 218/0 onto dev r1z1-10.1.1.1/sda Placed 218/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 218/1 onto dev r1z1-10.1.1.1/sdb Placed 218/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 218/2 onto dev r1z1-10.1.1.1/sdc Placed 218/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 237/0 onto dev r1z1-10.1.1.1/sdd Placed 237/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 237/1 onto dev r1z1-10.1.1.1/sda Placed 237/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 237/2 onto dev r1z1-10.1.1.1/sdb Placed 237/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 227/0 onto dev r1z1-10.1.1.1/sdc Placed 227/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 227/1 onto dev r1z1-10.1.1.1/sdd Placed 227/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 227/2 onto dev r1z1-10.1.1.1/sda Placed 227/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 199/0 onto dev r1z1-10.1.1.1/sdb Placed 199/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 199/1 onto dev r1z1-10.1.1.1/sdc Placed 199/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 199/2 onto dev r1z1-10.1.1.1/sdd Placed 199/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 158/0 onto dev r1z1-10.1.1.1/sda Placed 158/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 158/1 onto dev r1z1-10.1.1.1/sdb Placed 158/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 158/2 onto dev r1z1-10.1.1.1/sdc Placed 158/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 90/0 onto dev r1z1-10.1.1.1/sdd Placed 90/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 90/1 onto dev r1z1-10.1.1.1/sda Placed 90/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 90/2 onto dev r1z1-10.1.1.1/sdb Placed 90/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 52/0 onto dev r1z1-10.1.1.1/sdc Placed 52/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 52/1 onto dev r1z1-10.1.1.1/sdd Placed 52/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 52/2 onto dev r1z1-10.1.1.1/sda Placed 52/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 54/0 onto dev r1z1-10.1.1.1/sdb Placed 54/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 54/1 onto dev r1z1-10.1.1.1/sdc Placed 54/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 54/2 onto dev r1z1-10.1.1.1/sdd Placed 54/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 207/0 onto dev r1z1-10.1.1.1/sda Placed 207/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 207/1 onto dev r1z1-10.1.1.1/sdb Placed 207/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 207/2 onto dev r1z1-10.1.1.1/sdc Placed 207/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 16/0 onto dev r1z1-10.1.1.1/sdd Placed 16/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 16/1 onto dev r1z1-10.1.1.1/sda Placed 16/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 16/2 onto dev r1z1-10.1.1.1/sdb Placed 16/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 163/0 onto dev r1z1-10.1.1.1/sdc Placed 163/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 163/1 onto dev r1z1-10.1.1.1/sdd Placed 163/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 163/2 onto dev r1z1-10.1.1.1/sda Placed 163/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 14/0 onto dev r1z1-10.1.1.1/sdb Placed 14/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 14/1 onto dev r1z1-10.1.1.1/sdc Placed 14/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 14/2 onto dev r1z1-10.1.1.1/sdd Placed 14/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 223/0 onto dev r1z1-10.1.1.1/sda Placed 223/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 223/1 onto dev r1z1-10.1.1.1/sdb Placed 223/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 223/2 onto dev r1z1-10.1.1.1/sdc Placed 223/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 222/0 onto dev r1z1-10.1.1.1/sdd Placed 222/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 222/1 onto dev r1z1-10.1.1.1/sda Placed 222/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 222/2 onto dev r1z1-10.1.1.1/sdb Placed 222/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 176/0 onto dev r1z1-10.1.1.1/sdc Placed 176/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 176/1 onto dev r1z1-10.1.1.1/sdd Placed 176/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 176/2 onto dev r1z1-10.1.1.1/sda Placed 176/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 131/0 onto dev r1z1-10.1.1.1/sdb Placed 131/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 131/1 onto dev r1z1-10.1.1.1/sdc Placed 131/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 131/2 onto dev r1z1-10.1.1.1/sdd Placed 131/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 148/0 onto dev r1z1-10.1.1.1/sda Placed 148/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 148/1 onto dev r1z1-10.1.1.1/sdb Placed 148/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 148/2 onto dev r1z1-10.1.1.1/sdc Placed 148/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 254/0 onto dev r1z1-10.1.1.1/sdd Placed 254/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 254/1 onto dev r1z1-10.1.1.1/sda Placed 254/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 254/2 onto dev r1z1-10.1.1.1/sdb Placed 254/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 238/0 onto dev r1z1-10.1.1.1/sdc Placed 238/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 238/1 onto dev r1z1-10.1.1.1/sdd Placed 238/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 238/2 onto dev r1z1-10.1.1.1/sda Placed 238/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 243/0 onto dev r1z1-10.1.1.1/sdb Placed 243/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 243/1 onto dev r1z1-10.1.1.1/sdc Placed 243/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 243/2 onto dev r1z1-10.1.1.1/sdd Placed 243/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 63/0 onto dev r1z1-10.1.1.1/sda Placed 63/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 63/1 onto dev r1z1-10.1.1.1/sdb Placed 63/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 63/2 onto dev r1z1-10.1.1.1/sdc Placed 63/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 249/0 onto dev r1z1-10.1.1.1/sdd Placed 249/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 249/1 onto dev r1z1-10.1.1.1/sda Placed 249/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 249/2 onto dev r1z1-10.1.1.1/sdb Placed 249/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 46/0 onto dev r1z1-10.1.1.1/sdc Placed 46/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 46/1 onto dev r1z1-10.1.1.1/sdd Placed 46/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 46/2 onto dev r1z1-10.1.1.1/sda Placed 46/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 202/0 onto dev r1z1-10.1.1.1/sdb Placed 202/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 202/1 onto dev r1z1-10.1.1.1/sdc Placed 202/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 202/2 onto dev r1z1-10.1.1.1/sdd Placed 202/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 135/0 onto dev r1z1-10.1.1.1/sda Placed 135/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 135/1 onto dev r1z1-10.1.1.1/sdb Placed 135/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 135/2 onto dev r1z1-10.1.1.1/sdc Placed 135/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 178/0 onto dev r1z1-10.1.1.1/sdd Placed 178/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 178/1 onto dev r1z1-10.1.1.1/sda Placed 178/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 178/2 onto dev r1z1-10.1.1.1/sdb Placed 178/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 106/0 onto dev r1z1-10.1.1.1/sdc Placed 106/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 106/1 onto dev r1z1-10.1.1.1/sdd Placed 106/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 106/2 onto dev r1z1-10.1.1.1/sda Placed 106/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 146/0 onto dev r1z1-10.1.1.1/sdb Placed 146/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 146/1 onto dev r1z1-10.1.1.1/sdc Placed 146/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 146/2 onto dev r1z1-10.1.1.1/sdd Placed 146/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 183/0 onto dev r1z1-10.1.1.1/sda Placed 183/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 183/1 onto dev r1z1-10.1.1.1/sdb Placed 183/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 183/2 onto dev r1z1-10.1.1.1/sdc Placed 183/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 151/0 onto dev r1z1-10.1.1.1/sdd Placed 151/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 151/1 onto dev r1z1-10.1.1.1/sda Placed 151/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 151/2 onto dev r1z1-10.1.1.1/sdb Placed 151/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 19/0 onto dev r1z1-10.1.1.1/sdc Placed 19/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 19/1 onto dev r1z1-10.1.1.1/sdd Placed 19/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 19/2 onto dev r1z1-10.1.1.1/sda Placed 19/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 204/0 onto dev r1z1-10.1.1.1/sdb Placed 204/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 204/1 onto dev r1z1-10.1.1.1/sdc Placed 204/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 204/2 onto dev r1z1-10.1.1.1/sdd Placed 204/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 196/0 onto dev r1z1-10.1.1.1/sda Placed 196/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 196/1 onto dev r1z1-10.1.1.1/sdb Placed 196/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 196/2 onto dev r1z1-10.1.1.1/sdc Placed 196/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 242/0 onto dev r1z1-10.1.1.1/sdd Placed 242/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 242/1 onto dev r1z1-10.1.1.1/sda Placed 242/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 242/2 onto dev r1z1-10.1.1.1/sdb Placed 242/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 189/0 onto dev r1z1-10.1.1.1/sdc Placed 189/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 189/1 onto dev r1z1-10.1.1.1/sdd Placed 189/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 189/2 onto dev r1z1-10.1.1.1/sda Placed 189/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 245/0 onto dev r1z1-10.1.1.1/sdb Placed 245/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 245/1 onto dev r1z1-10.1.1.1/sdc Placed 245/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 245/2 onto dev r1z1-10.1.1.1/sdd Placed 245/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 144/0 onto dev r1z1-10.1.1.1/sda Placed 144/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 144/1 onto dev r1z1-10.1.1.1/sdb Placed 144/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 144/2 onto dev r1z1-10.1.1.1/sdc Placed 144/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 228/0 onto dev r1z1-10.1.1.1/sdd Placed 228/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 228/1 onto dev r1z1-10.1.1.1/sda Placed 228/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 228/2 onto dev r1z1-10.1.1.1/sdb Placed 228/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 186/0 onto dev r1z1-10.1.1.1/sdc Placed 186/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 186/1 onto dev r1z1-10.1.1.1/sdd Placed 186/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 186/2 onto dev r1z1-10.1.1.1/sda Placed 186/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 201/0 onto dev r1z1-10.1.1.1/sdb Placed 201/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 201/1 onto dev r1z1-10.1.1.1/sdc Placed 201/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 201/2 onto dev r1z1-10.1.1.1/sdd Placed 201/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 241/0 onto dev r1z1-10.1.1.1/sda Placed 241/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 241/1 onto dev r1z1-10.1.1.1/sdb Placed 241/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 241/2 onto dev r1z1-10.1.1.1/sdc Placed 241/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 248/0 onto dev r1z1-10.1.1.1/sdd Placed 248/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 248/1 onto dev r1z1-10.1.1.1/sda Placed 248/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 248/2 onto dev r1z1-10.1.1.1/sdb Placed 248/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 253/0 onto dev r1z1-10.1.1.1/sdc Placed 253/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 253/1 onto dev r1z1-10.1.1.1/sdd Placed 253/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 253/2 onto dev r1z1-10.1.1.1/sda Placed 253/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 192/0 onto dev r1z1-10.1.1.1/sdb Placed 192/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 192/1 onto dev r1z1-10.1.1.1/sdc Placed 192/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 192/2 onto dev r1z1-10.1.1.1/sdd Placed 192/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 113/0 onto dev r1z1-10.1.1.1/sda Placed 113/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 113/1 onto dev r1z1-10.1.1.1/sdb Placed 113/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 113/2 onto dev r1z1-10.1.1.1/sdc Placed 113/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 7/0 onto dev r1z1-10.1.1.1/sdd Placed 7/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 7/1 onto dev r1z1-10.1.1.1/sda Placed 7/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 7/2 onto dev r1z1-10.1.1.1/sdb Placed 7/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 234/0 onto dev r1z1-10.1.1.1/sdc Placed 234/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 234/1 onto dev r1z1-10.1.1.1/sdd Placed 234/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 234/2 onto dev r1z1-10.1.1.1/sda Placed 234/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 116/0 onto dev r1z1-10.1.1.1/sdb Placed 116/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 116/1 onto dev r1z1-10.1.1.1/sdc Placed 116/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 116/2 onto dev r1z1-10.1.1.1/sdd Placed 116/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 53/0 onto dev r1z1-10.1.1.1/sda Placed 53/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 53/1 onto dev r1z1-10.1.1.1/sdb Placed 53/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 53/2 onto dev r1z1-10.1.1.1/sdc Placed 53/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 119/0 onto dev r1z1-10.1.1.1/sdd Placed 119/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 119/1 onto dev r1z1-10.1.1.1/sda Placed 119/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 119/2 onto dev r1z1-10.1.1.1/sdb Placed 119/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 247/0 onto dev r1z1-10.1.1.1/sdc Placed 247/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 247/1 onto dev r1z1-10.1.1.1/sdd Placed 247/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 247/2 onto dev r1z1-10.1.1.1/sda Placed 247/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 47/0 onto dev r1z1-10.1.1.1/sdb Placed 47/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 47/1 onto dev r1z1-10.1.1.1/sdc Placed 47/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 47/2 onto dev r1z1-10.1.1.1/sdd Placed 47/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 221/0 onto dev r1z1-10.1.1.1/sda Placed 221/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 221/1 onto dev r1z1-10.1.1.1/sdb Placed 221/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 221/2 onto dev r1z1-10.1.1.1/sdc Placed 221/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 145/0 onto dev r1z1-10.1.1.1/sdd Placed 145/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 145/1 onto dev r1z1-10.1.1.1/sda Placed 145/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 145/2 onto dev r1z1-10.1.1.1/sdb Placed 145/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 147/0 onto dev r1z1-10.1.1.1/sdc Placed 147/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 147/1 onto dev r1z1-10.1.1.1/sdd Placed 147/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 147/2 onto dev r1z1-10.1.1.1/sda Placed 147/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 250/0 onto dev r1z1-10.1.1.1/sdb Placed 250/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 250/1 onto dev r1z1-10.1.1.1/sdc Placed 250/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 250/2 onto dev r1z1-10.1.1.1/sdd Placed 250/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 132/0 onto dev r1z1-10.1.1.1/sda Placed 132/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 132/1 onto dev r1z1-10.1.1.1/sdb Placed 132/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 132/2 onto dev r1z1-10.1.1.1/sdc Placed 132/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 224/0 onto dev r1z1-10.1.1.1/sdd Placed 224/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 224/1 onto dev r1z1-10.1.1.1/sda Placed 224/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 224/2 onto dev r1z1-10.1.1.1/sdb Placed 224/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 200/0 onto dev r1z1-10.1.1.1/sdc Placed 200/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 200/1 onto dev r1z1-10.1.1.1/sdd Placed 200/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 200/2 onto dev r1z1-10.1.1.1/sda Placed 200/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 59/0 onto dev r1z1-10.1.1.1/sdb Placed 59/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 59/1 onto dev r1z1-10.1.1.1/sdc Placed 59/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 59/2 onto dev r1z1-10.1.1.1/sdd Placed 59/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 98/0 onto dev r1z1-10.1.1.1/sda Placed 98/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 98/1 onto dev r1z1-10.1.1.1/sdb Placed 98/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 98/2 onto dev r1z1-10.1.1.1/sdc Placed 98/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 50/0 onto dev r1z1-10.1.1.1/sdd Placed 50/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 50/1 onto dev r1z1-10.1.1.1/sda Placed 50/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 50/2 onto dev r1z1-10.1.1.1/sdb Placed 50/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 97/0 onto dev r1z1-10.1.1.1/sdc Placed 97/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 97/1 onto dev r1z1-10.1.1.1/sdd Placed 97/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 97/2 onto dev r1z1-10.1.1.1/sda Placed 97/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 107/0 onto dev r1z1-10.1.1.1/sdb Placed 107/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 107/1 onto dev r1z1-10.1.1.1/sdc Placed 107/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 107/2 onto dev r1z1-10.1.1.1/sdd Placed 107/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 95/0 onto dev r1z1-10.1.1.1/sda Placed 95/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 95/1 onto dev r1z1-10.1.1.1/sdb Placed 95/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 95/2 onto dev r1z1-10.1.1.1/sdc Placed 95/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 149/0 onto dev r1z1-10.1.1.1/sdd Placed 149/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 149/1 onto dev r1z1-10.1.1.1/sda Placed 149/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 149/2 onto dev r1z1-10.1.1.1/sdb Placed 149/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 105/0 onto dev r1z1-10.1.1.1/sdc Placed 105/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 105/1 onto dev r1z1-10.1.1.1/sdd Placed 105/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 105/2 onto dev r1z1-10.1.1.1/sda Placed 105/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 18/0 onto dev r1z1-10.1.1.1/sdb Placed 18/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 18/1 onto dev r1z1-10.1.1.1/sdc Placed 18/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 18/2 onto dev r1z1-10.1.1.1/sdd Placed 18/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 111/0 onto dev r1z1-10.1.1.1/sda Placed 111/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 111/1 onto dev r1z1-10.1.1.1/sdb Placed 111/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 111/2 onto dev r1z1-10.1.1.1/sdc Placed 111/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 109/0 onto dev r1z1-10.1.1.1/sdd Placed 109/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 109/1 onto dev r1z1-10.1.1.1/sda Placed 109/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 109/2 onto dev r1z1-10.1.1.1/sdb Placed 109/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 133/0 onto dev r1z1-10.1.1.1/sdc Placed 133/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 133/1 onto dev r1z1-10.1.1.1/sdd Placed 133/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 133/2 onto dev r1z1-10.1.1.1/sda Placed 133/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 37/0 onto dev r1z1-10.1.1.1/sdb Placed 37/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 37/1 onto dev r1z1-10.1.1.1/sdc Placed 37/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 37/2 onto dev r1z1-10.1.1.1/sdd Placed 37/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 209/0 onto dev r1z1-10.1.1.1/sda Placed 209/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 209/1 onto dev r1z1-10.1.1.1/sdb Placed 209/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 209/2 onto dev r1z1-10.1.1.1/sdc Placed 209/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 216/0 onto dev r1z1-10.1.1.1/sdd Placed 216/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 216/1 onto dev r1z1-10.1.1.1/sda Placed 216/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 216/2 onto dev r1z1-10.1.1.1/sdb Placed 216/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 219/0 onto dev r1z1-10.1.1.1/sdc Placed 219/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 219/1 onto dev r1z1-10.1.1.1/sdd Placed 219/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 219/2 onto dev r1z1-10.1.1.1/sda Placed 219/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 154/0 onto dev r1z1-10.1.1.1/sdb Placed 154/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 154/1 onto dev r1z1-10.1.1.1/sdc Placed 154/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 154/2 onto dev r1z1-10.1.1.1/sdd Placed 154/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 233/0 onto dev r1z1-10.1.1.1/sda Placed 233/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 233/1 onto dev r1z1-10.1.1.1/sdb Placed 233/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 233/2 onto dev r1z1-10.1.1.1/sdc Placed 233/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 236/0 onto dev r1z1-10.1.1.1/sdd Placed 236/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 236/1 onto dev r1z1-10.1.1.1/sda Placed 236/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 236/2 onto dev r1z1-10.1.1.1/sdb Placed 236/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 175/0 onto dev r1z1-10.1.1.1/sdc Placed 175/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 175/1 onto dev r1z1-10.1.1.1/sdd Placed 175/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 175/2 onto dev r1z1-10.1.1.1/sda Placed 175/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 38/0 onto dev r1z1-10.1.1.1/sdb Placed 38/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 38/1 onto dev r1z1-10.1.1.1/sdc Placed 38/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 38/2 onto dev r1z1-10.1.1.1/sdd Placed 38/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 230/0 onto dev r1z1-10.1.1.1/sda Placed 230/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 230/1 onto dev r1z1-10.1.1.1/sdb Placed 230/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 230/2 onto dev r1z1-10.1.1.1/sdc Placed 230/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 123/0 onto dev r1z1-10.1.1.1/sdd Placed 123/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 123/1 onto dev r1z1-10.1.1.1/sda Placed 123/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 123/2 onto dev r1z1-10.1.1.1/sdb Placed 123/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 139/0 onto dev r1z1-10.1.1.1/sdc Placed 139/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 139/1 onto dev r1z1-10.1.1.1/sdd Placed 139/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 139/2 onto dev r1z1-10.1.1.1/sda Placed 139/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 77/0 onto dev r1z1-10.1.1.1/sdb Placed 77/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 77/1 onto dev r1z1-10.1.1.1/sdc Placed 77/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 77/2 onto dev r1z1-10.1.1.1/sdd Placed 77/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 57/0 onto dev r1z1-10.1.1.1/sda Placed 57/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 57/1 onto dev r1z1-10.1.1.1/sdb Placed 57/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 57/2 onto dev r1z1-10.1.1.1/sdc Placed 57/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 86/0 onto dev r1z1-10.1.1.1/sdd Placed 86/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 86/1 onto dev r1z1-10.1.1.1/sda Placed 86/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 86/2 onto dev r1z1-10.1.1.1/sdb Placed 86/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 73/0 onto dev r1z1-10.1.1.1/sdc Placed 73/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 73/1 onto dev r1z1-10.1.1.1/sdd Placed 73/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 73/2 onto dev r1z1-10.1.1.1/sda Placed 73/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 62/0 onto dev r1z1-10.1.1.1/sdb Placed 62/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 62/1 onto dev r1z1-10.1.1.1/sdc Placed 62/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 62/2 onto dev r1z1-10.1.1.1/sdd Placed 62/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 117/0 onto dev r1z1-10.1.1.1/sda Placed 117/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 117/1 onto dev r1z1-10.1.1.1/sdb Placed 117/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 117/2 onto dev r1z1-10.1.1.1/sdc Placed 117/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 48/0 onto dev r1z1-10.1.1.1/sdd Placed 48/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 48/1 onto dev r1z1-10.1.1.1/sda Placed 48/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 48/2 onto dev r1z1-10.1.1.1/sdb Placed 48/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 161/0 onto dev r1z1-10.1.1.1/sdc Placed 161/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 161/1 onto dev r1z1-10.1.1.1/sdd Placed 161/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 161/2 onto dev r1z1-10.1.1.1/sda Placed 161/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 1/0 onto dev r1z1-10.1.1.1/sdb Placed 1/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 1/1 onto dev r1z1-10.1.1.1/sdc Placed 1/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 1/2 onto dev r1z1-10.1.1.1/sdd Placed 1/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 0/0 onto dev r1z1-10.1.1.1/sda Placed 0/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 0/1 onto dev r1z1-10.1.1.1/sdb Placed 0/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 0/2 onto dev r1z1-10.1.1.1/sdc Placed 0/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 25/0 onto dev r1z1-10.1.1.1/sdd Placed 25/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 25/1 onto dev r1z1-10.1.1.1/sda Placed 25/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 25/2 onto dev r1z1-10.1.1.1/sdb Placed 25/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 4/0 onto dev r1z1-10.1.1.1/sdc Placed 4/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 4/1 onto dev r1z1-10.1.1.1/sdd Placed 4/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 4/2 onto dev r1z1-10.1.1.1/sda Placed 4/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 60/0 onto dev r1z1-10.1.1.1/sdb Placed 60/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 60/1 onto dev r1z1-10.1.1.1/sdc Placed 60/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 60/2 onto dev r1z1-10.1.1.1/sdd Placed 60/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 23/0 onto dev r1z1-10.1.1.1/sda Placed 23/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 23/1 onto dev r1z1-10.1.1.1/sdb Placed 23/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 23/2 onto dev r1z1-10.1.1.1/sdc Placed 23/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 104/0 onto dev r1z1-10.1.1.1/sdd Placed 104/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 104/1 onto dev r1z1-10.1.1.1/sda Placed 104/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 104/2 onto dev r1z1-10.1.1.1/sdb Placed 104/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 134/0 onto dev r1z1-10.1.1.1/sdc Placed 134/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 134/1 onto dev r1z1-10.1.1.1/sdd Placed 134/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 134/2 onto dev r1z1-10.1.1.1/sda Placed 134/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 99/0 onto dev r1z1-10.1.1.1/sdb Placed 99/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 99/1 onto dev r1z1-10.1.1.1/sdc Placed 99/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 99/2 onto dev r1z1-10.1.1.1/sdd Placed 99/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 168/0 onto dev r1z1-10.1.1.1/sda Placed 168/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 168/1 onto dev r1z1-10.1.1.1/sdb Placed 168/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 168/2 onto dev r1z1-10.1.1.1/sdc Placed 168/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 205/0 onto dev r1z1-10.1.1.1/sdd Placed 205/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 205/1 onto dev r1z1-10.1.1.1/sda Placed 205/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 205/2 onto dev r1z1-10.1.1.1/sdb Placed 205/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 64/0 onto dev r1z1-10.1.1.1/sdc Placed 64/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 64/1 onto dev r1z1-10.1.1.1/sdd Placed 64/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 64/2 onto dev r1z1-10.1.1.1/sda Placed 64/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 11/0 onto dev r1z1-10.1.1.1/sdb Placed 11/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 11/1 onto dev r1z1-10.1.1.1/sdc Placed 11/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 11/2 onto dev r1z1-10.1.1.1/sdd Placed 11/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 212/0 onto dev r1z1-10.1.1.1/sda Placed 212/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 212/1 onto dev r1z1-10.1.1.1/sdb Placed 212/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 212/2 onto dev r1z1-10.1.1.1/sdc Placed 212/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 100/0 onto dev r1z1-10.1.1.1/sdd Placed 100/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 100/1 onto dev r1z1-10.1.1.1/sda Placed 100/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 100/2 onto dev r1z1-10.1.1.1/sdb Placed 100/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 36/0 onto dev r1z1-10.1.1.1/sdc Placed 36/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 36/1 onto dev r1z1-10.1.1.1/sdd Placed 36/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 36/2 onto dev r1z1-10.1.1.1/sda Placed 36/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 83/0 onto dev r1z1-10.1.1.1/sdb Placed 83/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 83/1 onto dev r1z1-10.1.1.1/sdc Placed 83/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 83/2 onto dev r1z1-10.1.1.1/sdd Placed 83/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 12/0 onto dev r1z1-10.1.1.1/sda Placed 12/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 12/1 onto dev r1z1-10.1.1.1/sdb Placed 12/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 12/2 onto dev r1z1-10.1.1.1/sdc Placed 12/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 187/0 onto dev r1z1-10.1.1.1/sdd Placed 187/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 187/1 onto dev r1z1-10.1.1.1/sda Placed 187/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 187/2 onto dev r1z1-10.1.1.1/sdb Placed 187/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 27/0 onto dev r1z1-10.1.1.1/sdc Placed 27/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 27/1 onto dev r1z1-10.1.1.1/sdd Placed 27/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 27/2 onto dev r1z1-10.1.1.1/sda Placed 27/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 58/0 onto dev r1z1-10.1.1.1/sdb Placed 58/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 58/1 onto dev r1z1-10.1.1.1/sdc Placed 58/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 58/2 onto dev r1z1-10.1.1.1/sdd Placed 58/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 112/0 onto dev r1z1-10.1.1.1/sda Placed 112/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 112/1 onto dev r1z1-10.1.1.1/sdb Placed 112/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 112/2 onto dev r1z1-10.1.1.1/sdc Placed 112/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 153/0 onto dev r1z1-10.1.1.1/sdd Placed 153/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 153/1 onto dev r1z1-10.1.1.1/sda Placed 153/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 153/2 onto dev r1z1-10.1.1.1/sdb Placed 153/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 130/0 onto dev r1z1-10.1.1.1/sdc Placed 130/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 130/1 onto dev r1z1-10.1.1.1/sdd Placed 130/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 130/2 onto dev r1z1-10.1.1.1/sda Placed 130/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 32/0 onto dev r1z1-10.1.1.1/sdb Placed 32/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 32/1 onto dev r1z1-10.1.1.1/sdc Placed 32/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 32/2 onto dev r1z1-10.1.1.1/sdd Placed 32/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 51/0 onto dev r1z1-10.1.1.1/sda Placed 51/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 51/1 onto dev r1z1-10.1.1.1/sdb Placed 51/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 51/2 onto dev r1z1-10.1.1.1/sdc Placed 51/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 162/0 onto dev r1z1-10.1.1.1/sdd Placed 162/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 162/1 onto dev r1z1-10.1.1.1/sda Placed 162/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 162/2 onto dev r1z1-10.1.1.1/sdb Placed 162/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 41/0 onto dev r1z1-10.1.1.1/sdc Placed 41/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 41/1 onto dev r1z1-10.1.1.1/sdd Placed 41/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 41/2 onto dev r1z1-10.1.1.1/sda Placed 41/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 33/0 onto dev r1z1-10.1.1.1/sdb Placed 33/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 33/1 onto dev r1z1-10.1.1.1/sdc Placed 33/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 33/2 onto dev r1z1-10.1.1.1/sdd Placed 33/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 24/0 onto dev r1z1-10.1.1.1/sda Placed 24/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 24/1 onto dev r1z1-10.1.1.1/sdb Placed 24/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 24/2 onto dev r1z1-10.1.1.1/sdc Placed 24/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 94/0 onto dev r1z1-10.1.1.1/sdd Placed 94/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 94/1 onto dev r1z1-10.1.1.1/sda Placed 94/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 94/2 onto dev r1z1-10.1.1.1/sdb Placed 94/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 143/0 onto dev r1z1-10.1.1.1/sdc Placed 143/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 143/1 onto dev r1z1-10.1.1.1/sdd Placed 143/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 143/2 onto dev r1z1-10.1.1.1/sda Placed 143/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 166/0 onto dev r1z1-10.1.1.1/sdb Placed 166/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 166/1 onto dev r1z1-10.1.1.1/sdc Placed 166/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 166/2 onto dev r1z1-10.1.1.1/sdd Placed 166/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 152/0 onto dev r1z1-10.1.1.1/sda Placed 152/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 152/1 onto dev r1z1-10.1.1.1/sdb Placed 152/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 152/2 onto dev r1z1-10.1.1.1/sdc Placed 152/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 22/0 onto dev r1z1-10.1.1.1/sdd Placed 22/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 22/1 onto dev r1z1-10.1.1.1/sda Placed 22/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 22/2 onto dev r1z1-10.1.1.1/sdb Placed 22/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 198/0 onto dev r1z1-10.1.1.1/sdc Placed 198/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 198/1 onto dev r1z1-10.1.1.1/sdd Placed 198/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 198/2 onto dev r1z1-10.1.1.1/sda Placed 198/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 96/0 onto dev r1z1-10.1.1.1/sdb Placed 96/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 96/1 onto dev r1z1-10.1.1.1/sdc Placed 96/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 96/2 onto dev r1z1-10.1.1.1/sdd Placed 96/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 197/0 onto dev r1z1-10.1.1.1/sda Placed 197/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 197/1 onto dev r1z1-10.1.1.1/sdb Placed 197/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 197/2 onto dev r1z1-10.1.1.1/sdc Placed 197/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 110/0 onto dev r1z1-10.1.1.1/sdd Placed 110/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 110/1 onto dev r1z1-10.1.1.1/sda Placed 110/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 110/2 onto dev r1z1-10.1.1.1/sdb Placed 110/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 179/0 onto dev r1z1-10.1.1.1/sdc Placed 179/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 179/1 onto dev r1z1-10.1.1.1/sdd Placed 179/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 179/2 onto dev r1z1-10.1.1.1/sda Placed 179/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 239/0 onto dev r1z1-10.1.1.1/sdb Placed 239/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 239/1 onto dev r1z1-10.1.1.1/sdc Placed 239/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 239/2 onto dev r1z1-10.1.1.1/sdd Placed 239/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 2/0 onto dev r1z1-10.1.1.1/sda Placed 2/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 2/1 onto dev r1z1-10.1.1.1/sdb Placed 2/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 2/2 onto dev r1z1-10.1.1.1/sdc Placed 2/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 120/0 onto dev r1z1-10.1.1.1/sdd Placed 120/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 120/1 onto dev r1z1-10.1.1.1/sda Placed 120/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 120/2 onto dev r1z1-10.1.1.1/sdb Placed 120/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 193/0 onto dev r1z1-10.1.1.1/sdc Placed 193/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 193/1 onto dev r1z1-10.1.1.1/sdd Placed 193/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 193/2 onto dev r1z1-10.1.1.1/sda Placed 193/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 226/0 onto dev r1z1-10.1.1.1/sdb Placed 226/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 226/1 onto dev r1z1-10.1.1.1/sdc Placed 226/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 226/2 onto dev r1z1-10.1.1.1/sdd Placed 226/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 29/0 onto dev r1z1-10.1.1.1/sda Placed 29/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 29/1 onto dev r1z1-10.1.1.1/sdb Placed 29/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 29/2 onto dev r1z1-10.1.1.1/sdc Placed 29/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 43/0 onto dev r1z1-10.1.1.1/sdd Placed 43/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 43/1 onto dev r1z1-10.1.1.1/sda Placed 43/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 43/2 onto dev r1z1-10.1.1.1/sdb Placed 43/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 124/0 onto dev r1z1-10.1.1.1/sdc Placed 124/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 124/1 onto dev r1z1-10.1.1.1/sdd Placed 124/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 124/2 onto dev r1z1-10.1.1.1/sda Placed 124/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 68/0 onto dev r1z1-10.1.1.1/sdb Placed 68/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 68/1 onto dev r1z1-10.1.1.1/sdc Placed 68/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 68/2 onto dev r1z1-10.1.1.1/sdd Placed 68/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 185/0 onto dev r1z1-10.1.1.1/sda Placed 185/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 185/1 onto dev r1z1-10.1.1.1/sdb Placed 185/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 185/2 onto dev r1z1-10.1.1.1/sdc Placed 185/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 181/0 onto dev r1z1-10.1.1.1/sdd Placed 181/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 181/1 onto dev r1z1-10.1.1.1/sda Placed 181/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 181/2 onto dev r1z1-10.1.1.1/sdb Placed 181/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 49/0 onto dev r1z1-10.1.1.1/sdc Placed 49/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 49/1 onto dev r1z1-10.1.1.1/sdd Placed 49/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 49/2 onto dev r1z1-10.1.1.1/sda Placed 49/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 21/0 onto dev r1z1-10.1.1.1/sdb Placed 21/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 21/1 onto dev r1z1-10.1.1.1/sdc Placed 21/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 21/2 onto dev r1z1-10.1.1.1/sdd Placed 21/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 87/0 onto dev r1z1-10.1.1.1/sda Placed 87/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 87/1 onto dev r1z1-10.1.1.1/sdb Placed 87/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 87/2 onto dev r1z1-10.1.1.1/sdc Placed 87/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 39/0 onto dev r1z1-10.1.1.1/sdd Placed 39/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 39/1 onto dev r1z1-10.1.1.1/sda Placed 39/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 39/2 onto dev r1z1-10.1.1.1/sdb Placed 39/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 93/0 onto dev r1z1-10.1.1.1/sdc Placed 93/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 93/1 onto dev r1z1-10.1.1.1/sdd Placed 93/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 93/2 onto dev r1z1-10.1.1.1/sda Placed 93/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 160/0 onto dev r1z1-10.1.1.1/sdb Placed 160/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 160/1 onto dev r1z1-10.1.1.1/sdc Placed 160/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 160/2 onto dev r1z1-10.1.1.1/sdd Placed 160/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 231/0 onto dev r1z1-10.1.1.1/sda Placed 231/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 231/1 onto dev r1z1-10.1.1.1/sdb Placed 231/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 231/2 onto dev r1z1-10.1.1.1/sdc Placed 231/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 182/0 onto dev r1z1-10.1.1.1/sdd Placed 182/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 182/1 onto dev r1z1-10.1.1.1/sda Placed 182/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 182/2 onto dev r1z1-10.1.1.1/sdb Placed 182/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 172/0 onto dev r1z1-10.1.1.1/sdc Placed 172/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 172/1 onto dev r1z1-10.1.1.1/sdd Placed 172/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 172/2 onto dev r1z1-10.1.1.1/sda Placed 172/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 210/0 onto dev r1z1-10.1.1.1/sdb Placed 210/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 210/1 onto dev r1z1-10.1.1.1/sdc Placed 210/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 210/2 onto dev r1z1-10.1.1.1/sdd Placed 210/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 213/0 onto dev r1z1-10.1.1.1/sda Placed 213/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 213/1 onto dev r1z1-10.1.1.1/sdb Placed 213/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 213/2 onto dev r1z1-10.1.1.1/sdc Placed 213/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 35/0 onto dev r1z1-10.1.1.1/sdd Placed 35/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 35/1 onto dev r1z1-10.1.1.1/sda Placed 35/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 35/2 onto dev r1z1-10.1.1.1/sdb Placed 35/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 214/0 onto dev r1z1-10.1.1.1/sdc Placed 214/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 214/1 onto dev r1z1-10.1.1.1/sdd Placed 214/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 214/2 onto dev r1z1-10.1.1.1/sda Placed 214/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 66/0 onto dev r1z1-10.1.1.1/sdb Placed 66/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 66/1 onto dev r1z1-10.1.1.1/sdc Placed 66/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 66/2 onto dev r1z1-10.1.1.1/sdd Placed 66/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 125/0 onto dev r1z1-10.1.1.1/sda Placed 125/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 125/1 onto dev r1z1-10.1.1.1/sdb Placed 125/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 125/2 onto dev r1z1-10.1.1.1/sdc Placed 125/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 15/0 onto dev r1z1-10.1.1.1/sdd Placed 15/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 15/1 onto dev r1z1-10.1.1.1/sda Placed 15/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 15/2 onto dev r1z1-10.1.1.1/sdb Placed 15/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 81/0 onto dev r1z1-10.1.1.1/sdc Placed 81/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 81/1 onto dev r1z1-10.1.1.1/sdd Placed 81/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 81/2 onto dev r1z1-10.1.1.1/sda Placed 81/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 30/0 onto dev r1z1-10.1.1.1/sdb Placed 30/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 30/1 onto dev r1z1-10.1.1.1/sdc Placed 30/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 30/2 onto dev r1z1-10.1.1.1/sdd Placed 30/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 188/0 onto dev r1z1-10.1.1.1/sda Placed 188/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 188/1 onto dev r1z1-10.1.1.1/sdb Placed 188/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 188/2 onto dev r1z1-10.1.1.1/sdc Placed 188/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 42/0 onto dev r1z1-10.1.1.1/sdd Placed 42/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 42/1 onto dev r1z1-10.1.1.1/sda Placed 42/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 42/2 onto dev r1z1-10.1.1.1/sdb Placed 42/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 67/0 onto dev r1z1-10.1.1.1/sdc Placed 67/0 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 67/1 onto dev r1z1-10.1.1.1/sdd Placed 67/1 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 67/2 onto dev r1z1-10.1.1.1/sda Placed 67/2 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 194/0 onto dev r1z1-10.1.1.1/sdb Placed 194/0 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 194/1 onto dev r1z1-10.1.1.1/sdc Placed 194/1 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 194/2 onto dev r1z1-10.1.1.1/sdd Placed 194/2 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 157/0 onto dev r1z1-10.1.1.1/sda Placed 157/0 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 157/1 onto dev r1z1-10.1.1.1/sdb Placed 157/1 onto dev r1z1-10.1.1.1/sdb DEBUG: Placed 157/2 onto dev r1z1-10.1.1.1/sdc Placed 157/2 onto dev r1z1-10.1.1.1/sdc DEBUG: Placed 128/0 onto dev r1z1-10.1.1.1/sdd Placed 128/0 onto dev r1z1-10.1.1.1/sdd DEBUG: Placed 128/1 onto dev r1z1-10.1.1.1/sda Placed 128/1 onto dev r1z1-10.1.1.1/sda DEBUG: Placed 128/2 onto dev r1z1-10.1.1.1/sdb Placed 128/2 onto dev r1z1-10.1.1.1/sdb DEBUG: Assigned 768 parts Assigned 768 parts DEBUG: Finished rebalance plan after 1 attempts Finished rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 169 (Last start was 87) DEBUG: Gather start is 169 (Last start was 87) Gather start is 169 (Last start was 87) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1,) Planning 3.0 on (1,) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1) Planning 3.0 on (1, 1) DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 3.0 on (1, 1, '10.1.1.1') Planning 3.0 on (1, 1, '10.1.1.1') DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 0) Planning 0.75 on (1, 1, '10.1.1.1', 0) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 1) Planning 0.75 on (1, 1, '10.1.1.1', 1) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 2) Planning 0.75 on (1, 1, '10.1.1.1', 2) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: Planning 0.75 on (1, 1, '10.1.1.1', 3) Planning 0.75 on (1, 1, '10.1.1.1', 3) DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sda wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdb wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdc wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload r1z1-10.1.1.1/sdd wants 0.75 and is weighted for 0.75 so therefore requires 0.0 overload DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change DEBUG: 0 new parts and 0 removed parts from replica-count change 0 new parts and 0 removed parts from replica-count change DEBUG: Gather start is 169 (Last start was 87) DEBUG: Gather start is 169 (Last start was 87) DEBUG: Gather start is 169 (Last start was 87) Gather start is 169 (Last start was 87) DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts DEBUG: Unable to finish rebalance plan after 1 attempts Unable to finish rebalance plan after 1 attempts test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_debug ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_appears ... ok test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload test.unit.cli.test_ringbuilder.TestRebalanceCommand.test_rebalance_warning_with_overload ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location ... ok test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 test.unit.common.middleware.s3api.test_location.TestS3ApiLocation.test_object_location_setting_as_us_west_1 ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_all_users ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_authenticated_users ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_full_control ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_bucket_owner_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_canonical_user ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_elem ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_from_elem_by_id_only ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_private ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_acl_public_read_write ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_base_grantee ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_canned_acl_grantees ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_empty_list ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_object ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_undefined ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_decode_acl_with_invalid_json ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_container ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_many_grant ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_encode_acl_object ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl ... ok test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid test.unit.common.middleware.s3api.test_subresource.TestS3ApiSubresource.test_from_headers_x_amz_acl_invalid ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types test.unit.common.middleware.test_slo.TestRespAttrs.test_default_types ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers test.unit.common.middleware.test_slo.TestRespAttrs.test_from_empty_headers ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp test.unit.common.middleware.test_slo.TestRespAttrs.test_from_only_timestamp ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object test.unit.common.middleware.test_slo.TestRespAttrs.test_from_regular_object ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_valid_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_from_zero_byte_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_init_calculates_is_legacy ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings test.unit.common.middleware.test_slo.TestRespAttrs.test_init_converts_timestamps_from_strings ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset test.unit.common.middleware.test_slo.TestRespAttrs.test_init_with_no_sysmeta_offset ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_invalid_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_legacy_slo_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_non_slo_with_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta test.unit.common.middleware.test_slo.TestRespAttrs.test_partial_modern_sysmeta ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_not_legacy ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_range ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_raw_data ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo ... ok test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range test.unit.common.middleware.test_slo.TestRespAttrs.test_update_from_segments_with_sub_slo_range ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_both ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_host ... ok test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme test.unit.common.middleware.test_staticweb.TestStaticWebUrlBase.test_container3subdirz_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_delete_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions test.unit.common.middleware.test_tempauth.TestAuth.test_account_put_permissions ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon test.unit.common.middleware.test_tempauth.TestAuth.test_anon ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath test.unit.common.middleware.test_tempauth.TestAuth.test_anon_badpath ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override test.unit.common.middleware.test_tempauth.TestAuth.test_auth_deny_non_reseller_prefix_no_override ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail test.unit.common.middleware.test_tempauth.TestAuth.test_auth_fail ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token test.unit.common.middleware.test_tempauth.TestAuth.test_auth_no_reseller_prefix_no_token ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_auth_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny test.unit.common.middleware.test_tempauth.TestAuth.test_auth_reseller_prefix_with_s3_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_auth_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_s3api_details ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_old_swift3_details ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_s3api_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_good ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid test.unit.common.middleware.test_tempauth.TestAuth.test_auth_with_swift3_authorization_invalid ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_account_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_group_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referer_after_user_groups ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_acl_referrer_access ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_authorize_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller test.unit.common.middleware.test_tempauth.TestAuth.test_deny_cross_reseller ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request test.unit.common.middleware.test_tempauth.TestAuth.test_detect_reseller_request ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_bad_path ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_invalid_x_auth_user_format ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_missing_key ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_fail_non_matching_account_in_request ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_memcache_error ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_get_token_success_other_auth_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group test.unit.common.middleware.test_tempauth.TestAuth.test_get_user_group ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash test.unit.common.middleware.test_tempauth.TestAuth.test_object_name_containing_slash ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_overdate ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data test.unit.common.middleware.test_tempauth.TestAuth.test_old_token_with_old_data ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_options_call test.unit.common.middleware.test_tempauth.TestAuth.test_options_call ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_override_default_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner test.unit.common.middleware.test_tempauth.TestAuth.test_regular_is_not_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed test.unit.common.middleware.test_tempauth.TestAuth.test_request_method_not_allowed ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_admin_is_owner ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init test.unit.common.middleware.test_tempauth.TestAuth.test_reseller_prefix_init ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix test.unit.common.middleware.test_tempauth.TestAuth.test_statsd_prefix ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_based_on_host ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_default ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme test.unit.common.middleware.test_tempauth.TestAuth.test_storage_url_overridden_scheme ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user test.unit.common.middleware.test_tempauth.TestAuth.test_successful_token_unicode_user ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_key ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_fail_no_timestamp ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host test.unit.common.middleware.test_tempauth.TestAuth.test_sync_request_success_lb_sync_host ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny test.unit.common.middleware.test_tempauth.TestAuth.test_top_level_deny ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached test.unit.common.middleware.test_tempauth.TestAuth.test_use_old_token_from_memcached ... ok test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success test.unit.common.middleware.test_tempauth.TestAuth.test_use_token_success ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall test.unit.common.middleware.test_xprofile.TestProfilers.test_runcall ... ok test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx test.unit.common.middleware.test_xprofile.TestProfilers.test_runctx ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ test.unit.common.test_http_protocol.TestProxyProtocol.test_address_and_environ ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines test.unit.common.test_http_protocol.TestProxyProtocol.test_malformed_proxy_lines ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line test.unit.common.test_http_protocol.TestProxyProtocol.test_missing_proxy_line ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_multiple_requests_with_proxy ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https test.unit.common.test_http_protocol.TestProxyProtocol.test_request_with_proxy_https ... ok test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr test.unit.common.test_http_protocol.TestProxyProtocol.test_unknown_client_addr ... ok test.unit.common.test_splice.TestSplice.test_available test.unit.common.test_splice.TestSplice.test_available ... ok test.unit.common.test_splice.TestSplice.test_errno test.unit.common.test_splice.TestSplice.test_errno ... ok test.unit.common.test_splice.TestSplice.test_fileno test.unit.common.test_splice.TestSplice.test_fileno ... ok test.unit.common.test_splice.TestSplice.test_flags test.unit.common.test_splice.TestSplice.test_flags ... ok test.unit.common.test_splice.TestSplice.test_flags_list test.unit.common.test_splice.TestSplice.test_flags_list ... ok test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe test.unit.common.test_splice.TestSplice.test_splice_file_to_pipe ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file test.unit.common.test_splice.TestSplice.test_splice_pipe_to_file ... ok test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe test.unit.common.test_splice.TestSplice.test_splice_pipe_to_pipe ... ok test.unit.common.test_splice.TestSplice.test_unavailable test.unit.common.test_splice.TestSplice.test_unavailable ... ok test.unit.common.test_splice.TestSplice.test_unavailable_in_libc test.unit.common.test_splice.TestSplice.test_unavailable_in_libc ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_list_no_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star test.unit.common.test_swob.TestConditionalIfNoneMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfNoneMatch.test_simple_match ... ok test.unit.common.test_utils.TestCooperativeIterator.test_close test.unit.common.test_utils.TestCooperativeIterator.test_close ... ok test.unit.common.test_utils.TestCooperativeIterator.test_init test.unit.common.test_utils.TestCooperativeIterator.test_init ... ok test.unit.common.test_utils.TestCooperativeIterator.test_iter test.unit.common.test_utils.TestCooperativeIterator.test_iter ... ok test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_no_sleeps ... ok test.unit.common.test_utils.TestCooperativeIterator.test_sleeps test.unit.common.test_utils.TestCooperativeIterator.test_sleeps ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_fails ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_force ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_add_to_reconciler_queue_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_direct_delete_container_entry_with_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_cache ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_deleted ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_brain ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_for_recently_split_recreated ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_socket_error ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors test.unit.container.test_reconciler.TestReconcilerUtils.test_get_container_policy_index_with_too_many_errors ... ok test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj test.unit.container.test_reconciler.TestReconcilerUtils.test_parse_raw_obj ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_add_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_container_to_synced_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers test.unit.container.test_sync_store.TestContainerSyncStore.test_iterate_synced_containers ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container test.unit.container.test_sync_store.TestContainerSyncStore.test_remove_synced_container ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion test.unit.container.test_sync_store.TestContainerSyncStore.test_synced_container_to_container_path_conversion ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove test.unit.container.test_sync_store.TestContainerSyncStore.test_unhandled_exceptions_in_add_remove ... ok test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted test.unit.container.test_sync_store.TestContainerSyncStore.test_update_sync_store_according_to_metadata_and_deleted ... ok test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_builtin_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_agreement ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_watcher_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects test.unit.obj.test_auditor.TestAuditWatchers.test_dark_data_with_sharding_fallback_to_root_no_objects ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_auditor.TestAuditWatchers.test_watchers test.unit.obj.test_auditor.TestAuditWatchers.test_watchers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init test.unit.obj.test_diskfile.TestDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_fails_with_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_content_type_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_missing_check_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable test.unit.obj.test_ssync_sender.TestSenderEC.test_updates_put_non_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacy.test_get_update_shard_empty_body ... ok proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txc55c03afa2d048b5bdb04-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txa0c3dcec0e4546bdb4673-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: txa0c3dcec0e4546bdb4673-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx82b54aad36e9475998e81-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: tx82b54aad36e9475998e81-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx36ade5751663436b9bda1-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx36ade5751663436b9bda1-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx9a58216bc99349c281075-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx9a58216bc99349c281075-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: tx9a58216bc99349c281075-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx11a3a326581e486d9705e-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx11a3a326581e486d9705e-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx91b209cbc34443cba9dbc-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6919b3e6776048319ebe9-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6919b3e6776048319ebe9-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6415c568e2af4aac90118-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx6415c568e2af4aac90118-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: tx6415c568e2af4aac90118-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx5fa5f5f564d842c787bf5-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx5fa5f5f564d842c787bf5-00665ca415) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_explode ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_mixed_explode_and_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_primaries_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_resuming_ignores_416 ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_simple ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sharding_sysmeta ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_old_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_Response ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_copy_self_metadata_replace ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeMetadata.test_sync_merging ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values test.unit.container.test_sharder.TestContainerSharderConf.test_bad_values ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges test.unit.container.test_sharder.TestContainerSharderConf.test_combine_shard_ranges ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_conf test.unit.container.test_sharder.TestContainerSharderConf.test_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_default test.unit.container.test_sharder.TestContainerSharderConf.test_default ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf test.unit.container.test_sharder.TestContainerSharderConf.test_deprecated_percent_conf ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate test.unit.container.test_sharder.TestContainerSharderConf.test_validate ... ok test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset test.unit.container.test_sharder.TestContainerSharderConf.test_validate_subset ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bool test.unit.obj.test_updater.TestRateLimiterBucket.test_bool ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering test.unit.obj.test_updater.TestRateLimiterBucket.test_bucket_ordering ... ok test.unit.obj.test_updater.TestRateLimiterBucket.test_len test.unit.obj.test_updater.TestRateLimiterBucket.test_len ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_not_available ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_storage_policy_property ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardLegacyUTF8.test_get_update_shard_empty_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_error ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_body_and_footers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeXSync.test_sync_merging ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerMimePutter.test_PUT_with_no_body_and_footers ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_mostly_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestECObjController.test_DELETE_write_affinity_before_replication ... ok Device d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-test.test.com:6200Rr.test.com:7000/sda3_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z3-[2001:0:1234::c1c0:abcd:876]:6200R[2::10]:7000/sda3_"some meta data" weight set to 3.14159265359 swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. d2r2z2-127.0.0.3:6202R127.0.0.3:6202/sdc3_"" weight set to 3.14 d1r1z1-127.0.0.2:6201R127.0.0.2:6201/sda2_"" weight set to 6.28 Search value matched 0 devices. The on-disk ring builder is unchanged. d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-test.test.com:6000Rr.test.com:7000/sda3_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 d0r0z314-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 Device d4r2z3-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" with 100.0 weight got id 4 d4r2z314-[2001:0:1234::c1c0:abcd:876]:6000R[2::10]:7000/sda3_"some meta data" zone set to 314 swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_number_of_arguments ... ok test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices test.unit.cli.test_ringbuilder.TestCommands.test_set_zone_values_no_devices ... ok test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining test.unit.cli.test_ringbuilder.TestCommands.test_time_remaining ... ok test.unit.cli.test_ringbuilder.TestCommands.test_unknown test.unit.cli.test_ringbuilder.TestCommands.test_unknown ... ok test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0168 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0175 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0222 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0249 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0185 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0233 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0042 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0192 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0015 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0239 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0055 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0053 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0049 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:24 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0054 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0057 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0175 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0240 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0006 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0007 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0007 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0006 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0009 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0212 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0185 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0209 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0112 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0251 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0061 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0271 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0313 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0660 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0041 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0249 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0157 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:25 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0190 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0166 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0270 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 - test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_object_timing_metrics ... ok test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx85f2593d65904b34962a3-00665ca40c) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: tx85f2593d65904b34962a3-00665ca40c) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txd441015a5d6244be9b449-00665ca410) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txd441015a5d6244be9b449-00665ca410) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txd441015a5d6244be9b449-00665ca410) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txd441015a5d6244be9b449-00665ca410) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd441015a5d6244be9b449-00665ca410) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txd441015a5d6244be9b449-00665ca410) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txd441015a5d6244be9b449-00665ca410) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: txd441015a5d6244be9b449-00665ca410) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Setting object count to 1058 (txn: txc90ff480809643b18da48-00665ca413) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Setting object count to 1024 (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Setting object count to 1024 (txn: tx479dfa261fd14e6f95f83-00665ca417) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharded_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_complete ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_GET_without_permission ... ok proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx48c2626611624f1cb4397-00665ca3e1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_some_missing ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: public None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: authenticated None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:12 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_authenticated_users ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_delimiter ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_id_and_key_marker ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:54:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester READ_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_read_acp ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_canned_acl ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpcvsngq3y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e6458dd00b139b9a31c987da93852f90', 'ea4ba763-1435-4f5a-9a78-cb9168290f19-sda', '1717347326.64467', '1717347328', '0', '', '1717347328', 1, 1) test DEBUG: /tmp/tmpcvsngq3y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347329.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'ea4ba763-1435-4f5a-9a78-cb9168290f19-sda') test DEBUG: /tmp/tmpcvsngq3y/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0174723s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e6458dd00b139b9a31c987da93852f90', 'ea4ba763-1435-4f5a-9a78-cb9168290f19-sda', '1717347326.64467', '1717347328', '0', '', '1717347328', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ea4ba763-1435-4f5a-9a78-cb9168290f19-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13590 seconds (7.35849/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpyn7s4iu_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e6458dd00b139b9a31c987da93852f90', 'f2c9166a-cd20-4a7a-8ba1-6dc1ce62114e-sda', '1717347326.83303', '1717347326', '1717347327', '', '1717347327', 1, 0) test DEBUG: /tmp/tmpyn7s4iu_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347329.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'f2c9166a-cd20-4a7a-8ba1-6dc1ce62114e-sda') test DEBUG: /tmp/tmpyn7s4iu_/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0196619s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e6458dd00b139b9a31c987da93852f90', 'f2c9166a-cd20-4a7a-8ba1-6dc1ce62114e-sda', '1717347326.83303', '1717347326', '1717347327', '', '1717347327', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f2c9166a-cd20-4a7a-8ba1-6dc1ce62114e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13598 seconds (7.35400/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpfo_i73x2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '23e2d7dd2c7017c239dbc0373de02424', '5a761681-c208-40ec-ac48-b056e9b758ee-sda', '1717347327.02987', '1717347327', '1717347328', '', '1717347328', 1, 0) test DEBUG: /tmp/tmpfo_i73x2/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '23e2d7dd2c7017c239dbc0373de02424', '5a761681-c208-40ec-ac48-b056e9b758ee-sda', '1717347327.02987', '1717347327', '1717347328', '', '1717347328', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5a761681-c208-40ec-ac48-b056e9b758ee-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12962 seconds (7.71478/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpfjkpy_w5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bcc8ef0c-0ae9-4bb0-ba91-f579387a9b23-sda', '1717347327.21629', '1717347329', '1717347328', '', '1717347329', 0, 0) test DEBUG: /tmp/tmpfjkpy_w5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bcc8ef0c-0ae9-4bb0-ba91-f579387a9b23-sda', '1717347327.21629', '1717347329', '1717347328', '', '1717347329', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('bcc8ef0c-0ae9-4bb0-ba91-f579387a9b23-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07989 seconds (12.51663/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_3kuxq06/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b937a508003e70f50693a607cd2dba64', '04dfe824-6e58-44eb-8fcc-2c44a5297ba9-sda', '1717347327.36337', '1717347329', '1717347328', '', '1717347329', 1, 0) test DEBUG: /tmp/tmp_3kuxq06/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347331.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '04dfe824-6e58-44eb-8fcc-2c44a5297ba9-sda') test DEBUG: /tmp/tmp_3kuxq06/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0237026s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'b937a508003e70f50693a607cd2dba64', '04dfe824-6e58-44eb-8fcc-2c44a5297ba9-sda', '1717347327.36337', '1717347329', '1717347328', '', '1717347329', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04dfe824-6e58-44eb-8fcc-2c44a5297ba9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.17865 seconds (5.59761/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp4rbxrg3v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd9a2fa2e-768f-4181-bd46-22c2bbb5669a-sda', '1717347327.62827', '1717347329', '1717347328', '', '1717347329', 0, 0) test DEBUG: /tmp/tmp4rbxrg3v/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_newer_remote_delete ... ok test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTC.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_from ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_not_exceed_bytes_quota_copy_verb ... ok test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src test.unit.common.middleware.test_account_quotas.AccountQuotaCopyingTestCases.test_quota_copy_from_bad_src ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bad_application_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_bogus_quota_is_ignored ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_container_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_delete_quotas_with_remove_header_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_from ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_bytes_quota_reseller_copy_verb ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_per_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_bytes_on_empty_account_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_exceed_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_policy_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_invalid_request_exception ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_info_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_no_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_not_exceed_bytes_quota ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_obj_request_ignores_attempt_to_set_quotas ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_create_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_container_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_over_quota_obj_post_still_works ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_policy_quota_translation ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_unauthorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_under_quota_not_authorized ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_policy_quota_reseller ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_admin ... ok test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller test.unit.common.middleware.test_account_quotas.TestAccountQuota.test_valid_quotas_reseller ... ok test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata test.unit.common.middleware.test_bulk.TestUntarMetadata.test_extract_metadata ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validating_x_object_manifest ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_accounts ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_ignores_containers ... ok test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes test.unit.common.middleware.test_dlo.TestDloPutManifest.test_validation_watches_manifests_with_slashes ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_formpost.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_formpost.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_formpost.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_formpost.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_referrer ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_invalid_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_referrer_without_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_for_unauthorized_user ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_fails_same_user_and_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_insensitive_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_as_owner_for_operator_role ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_container_sync ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_insensitive_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_project_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_obj ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_referrer_with_rlistings ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_reseller_admin ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_id_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_tenant_name_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_user_role_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_authorize_succeeds_for_wildcard_tenant_user_in_roles ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_allow_names ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_failure ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_cross_tenant_authorization_success ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_delete_own_account_when_reseller_allowed ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_get_project_domain_id ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_identity_set_up_at_call ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_keystone_identity ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_allowed_in_acls_inside_default_domain_with_config ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_inside_default_domain ... ok test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain test.unit.common.middleware.test_keystoneauth.TestAuthorize.test_names_disallowed_in_acls_outside_default_domain ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_segment_with_non_ascii_path ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_get_subrange_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_first_byte ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_on_segment_boundaries ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_overlapping_end ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_sub_slo ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_manifest_unsatisfiable ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_range_manifest ... ok test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest test.unit.common.middleware.test_slo.TestSloRangeRequests.test_range_get_subrange_manifest ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_failures ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_account_acl_success ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acl_syntax_verification ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_acls_propagate_to_sysmeta ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_admin_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_bad_acls_get_denied ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readonly_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_readwrite_privileges ... ok test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl test.unit.common.middleware.test_tempauth.TestAuthMultiplePrefixes.test_user_gets_best_acl ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_case_insensitive ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_del_contains ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_get ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_init_from_dict ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_keys ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_pop ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_set_none ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_setdefault ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_unicode ... ok test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update test.unit.common.test_header_key_dict.TestHeaderKeyDict.test_update ... ok test.unit.common.test_linkat.TestLinkat.test_available test.unit.common.test_linkat.TestLinkat.test_available ... ok test.unit.common.test_linkat.TestLinkat.test_errno test.unit.common.test_linkat.TestLinkat.test_errno ... ok test.unit.common.test_linkat.TestLinkat.test_flags test.unit.common.test_linkat.TestLinkat.test_flags ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer test.unit.common.test_linkat.TestLinkat.test_linkat_fd_not_integer ... ok test.unit.common.test_linkat.TestLinkat.test_linkat_success test.unit.common.test_linkat.TestLinkat.test_linkat_success ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable test.unit.common.test_linkat.TestLinkat.test_unavailable ... ok test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc test.unit.common.test_linkat.TestLinkat.test_unavailable_in_libc ... ok test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file test.unit.common.test_recon.TestCommonRecon.test_server_type_to_recon_file ... ok test.unit.common.test_utils.TestLRUCache.test_maxsize test.unit.common.test_utils.TestLRUCache.test_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_maxtime test.unit.common.test_utils.TestLRUCache.test_maxtime ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxsize test.unit.common.test_utils.TestLRUCache.test_set_maxsize ... ok test.unit.common.test_utils.TestLRUCache.test_set_maxtime test.unit.common.test_utils.TestLRUCache.test_set_maxtime ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_epoch_reset ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_none ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_merge_reported ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_gte ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_meta_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_bounds ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_retain_deleted ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_eq ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_gt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_eq_state_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_gt ... ok test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt test.unit.container.test_backend.TestModuleFunctions.test_merge_shards_existing_ts_lt ... ok test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges test.unit.container.test_backend.TestModuleFunctions.test_sift_shard_ranges ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_init_node_timeout ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward ... ok test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout test.unit.proxy.controllers.test_base.TestGetOrHeadHandler.test_range_fast_forward_after_data_timeout ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_object ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_record_type_shard_with_listing_state ... ok test ERROR: Failed to unlink '/tmp/tmp8b0xpxg9/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp8b0xpxg9/containers/part/suffix/hash/container_1717347373.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp8b0xpxg9/containers/part/suffix/hash/container_1717347373.00000.db': ['/tmp/tmp8b0xpxg9/containers/part/suffix/hash/container.db', '/tmp/tmp8b0xpxg9/containers/part/suffix/hash/container_1717347374.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp8b0xpxg9/containers/part/suffix/hash/container_1717347374.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx4d693918b8b244ed9354d-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txd674b6ee32c6439aa8bc5-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx940fa56057554823a7e62-00665ca436) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx940fa56057554823a7e62-00665ca436) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx940fa56057554823a7e62-00665ca436) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: tx940fa56057554823a7e62-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp01ybefcg/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp01ybefcg/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp01ybefcg/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp01ybefcg/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0147 "-" 3656266 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0109 "-" 3656266 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0133 "-" 3656266 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0124 "-" 3656266 - test.unit.proxy.test_server.TestAccountController.test_DELETE test.unit.proxy.test_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.proxy.test_server.TestAccountController.test_GET test.unit.proxy.test_server.TestAccountController.test_GET ... ok test.unit.proxy.test_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_server.TestAccountController.test_GET_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD test.unit.proxy.test_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_OPTIONS test.unit.proxy.test_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_server.TestAccountController.test_POST_autocreate ... ok test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_POST_metadata test.unit.proxy.test_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT test.unit.proxy.test_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestAccountController.test_PUT_status test.unit.proxy.test_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.proxy.test_server.TestAccountController.test_connection_refused test.unit.proxy.test_server.TestAccountController.test_connection_refused ... ok test.unit.proxy.test_server.TestAccountController.test_other_socket_error test.unit.proxy.test_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestAccountController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY jojo', (txn: txb379165002eb4c94bfd19-00665ca3fc) proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXYjojo a b c d e', (txn: tx101007fbb10c45ad8046e-00665ca3fc) proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'PROXY a b c d e', (txn: txc98d0bfef02a4ec29a284-00665ca3fc) proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: ERROR WSGI: code 400, message Invalid PROXY line 'GET /someurl HTTP/1.0\r\n', (txn: test-bad-req-trans-id) proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:55:24] "GET /someurl HTTP/1.1" 200 160 0.000171 proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:55:24] "GET /otherurl HTTP/1.1" 200 179 0.000089 proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:55:24] "GET /someurl HTTP/1.0" 200 186 0.000082 proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: 192.168.0.1 - - [02/Jun/2024 04:55:24] "GET /someurl HTTP/1.0" 200 179 0.000094 proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:55:31] "GET /someurl HTTP/1.0" 200 180 0.000112 proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:55:31] "GET /someurl HTTP/1.0" 200 180 0.000068 proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy INFO: (3656243) wsgi starting up on http:('127.0.0.1', 8359) proxy DEBUG: (3656243) accepted ('127.0.0.1', 8359) proxy INFO: 127.0.0.1 - - [02/Jun/2024 04:55:31] "GET /someurl HTTP/1.0" 200 180 0.000075 proxy INFO: wsgi exiting proxy INFO: (3656243) wsgi exited, is_accepting=True proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=x1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx7fdf5ddf172d4bc48f99e-00665ca431) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: tx17ed6bda0b88407ebcf95-00665ca431) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmp98ysgeb9/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp98ysgeb9/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp98ysgeb9/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp98ysgeb9/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:18 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 1.0540 "-" 3656243 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:19 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.7103 "-" 3656243 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:20 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.5572 "-" 3656243 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:20 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.5345 "-" 3656243 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:20 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf8c9b4dac79241c8863ef-00665ca434" "proxy-server 3656243" 0.0024 "-" 3656243 - test.unit.proxy.test_mem_server.TestController.test_account_info_200 test.unit.proxy.test_mem_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_404 test.unit.proxy.test_mem_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_container_count test.unit.proxy.test_mem_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_account test.unit.proxy.test_mem_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache test.unit.proxy.test_mem_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_200 test.unit.proxy.test_mem_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_404 test.unit.proxy.test_mem_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account test.unit.proxy.test_mem_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache test.unit.proxy.test_mem_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_mem_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_make_requests test.unit.proxy.test_mem_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestController.test_transfer_headers test.unit.proxy.test_mem_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_expose_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_non_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_strict ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_CORS_valid_with_obj_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_DELETE_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_bad_range_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_newest_large_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_ranges_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_respects_read_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_GET_short_read_resuming ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_absolute_uri ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_HEAD_newest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_backend_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_converts_delete_after_to_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_key_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_meta_val_len ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_POST_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_GET_unicode_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_last_modified ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_POST_requires_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_auto_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_bad_message_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_calls_authorize ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_connect_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_error_limiting_last_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_nonzero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_expect_header_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_getresponse_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_max_size ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_too_large ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_unsup_xfr_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_both_with_crazy_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_message_length_using_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_no_etag_fallocate ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_not_autodetect_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_all_containers_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_container_exist_on_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_containers_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_object_to_primary_timeout_container_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_respects_write_affinity_with_507s ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_send_exceptions ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_equal_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_many_object_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_container_headers_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_fewer_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_PUT_x_delete_at_with_more_container_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_acc_or_con_missing_returns_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_cache_set_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_can_skip_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_empty_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_backend_headers_update_shard_container_with_live_infocache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_bad_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_best_response_sets_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_path_no_controller ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_bad_version ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_head_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_unhandled_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_chunked_put_utf8_all_the_way_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_GET_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_conditional_range_get ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_custom_mime_types_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_error_limiting_survives_ring_reload ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_calls_sort_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_gives_extra_if_error_limited_inline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_skips_error_limited ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_with_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_iter_nodes_without_replication_network ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_leak_1 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_mismatched_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_first_two_time_out_etags_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_one_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_one_time_out_two_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_only_first_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_three_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_read_timeout_retry_two_time_out ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_request_setting ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_node_write_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_policy_IO_override ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_length_on_head ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_proxy_passes_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_get_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestReplicatedObjectController.test_response_head_accept_ranges_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx5fa5f5f564d842c787bf5-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: tx5fa5f5f564d842c787bf5-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txccb0cdb29cd64a8998752-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: txccb0cdb29cd64a8998752-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txccb0cdb29cd64a8998752-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: txccb0cdb29cd64a8998752-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx1d71be4ecf0b45f8a2c57-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx1d71be4ecf0b45f8a2c57-00665ca415) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx1d71be4ecf0b45f8a2c57-00665ca415) proxy-server ERROR: Object GET returning 503 for [] (txn: tx1d71be4ecf0b45f8a2c57-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx43a21c1f9cb945569d22d-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txb7c095d581ee4b4c99dba-00665ca415) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: txb7c095d581ee4b4c99dba-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: tx567639d2cbc64c20880ad-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txdc2c8c6da8794141bd04e-00665ca415) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_slow_read ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_transfer_encoding_chunked ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_after_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_resumes_before_body_started ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_with_multirange_slow_body_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_GET_x_newest_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_error_limit_supression_count ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_removes_swift_bytes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_simple ... ok test ERROR: Failed to unlink '/tmp/tmpbzdvgdn_/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpbzdvgdn_/containers/part/suffix/hash/container_1717347373.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpbzdvgdn_/containers/part/suffix/hash/container_1717347373.00000.db': ['/tmp/tmpbzdvgdn_/containers/part/suffix/hash/container.db', '/tmp/tmpbzdvgdn_/containers/part/suffix/hash/container_1717347374.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpbzdvgdn_/containers/part/suffix/hash/container_1717347374.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.11:1011/sdl, error: ERROR Insufficient Storage (txn: tx39b15f9fe74542238a16c-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.4:1004/sde (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.8:1008/sdi (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.7:1007/sdh (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.9:1009/sdj (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.6:1006/sdg (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_disconnect ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_bucket_upload_part_missing_key ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Setting object count to 1072 (txn: txad59c17d9a5c40ba90ce1-00665ca41a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Setting object count to 1072 (txn: tx4520f34b1bc443b1add8c-00665ca41d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_sharding_shard_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_deleted_shard ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txc88412cb0cb74c96baf17-00665ca436" "proxy-server 3656266" 0.0020 "-" 3656266 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:45331/sda1/1/a/c" "txc88412cb0cb74c96baf17-00665ca436" "container-server 3656266" 0.0025 "-" 3656266 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:38045/sdb1/1/a/c" "txc88412cb0cb74c96baf17-00665ca436" "container-server 3656266" 0.0040 "-" 3656266 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txc88412cb0cb74c96baf17-00665ca436" "proxy-server 3656266" 0.0593 "-" 3656266 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txc88412cb0cb74c96baf17-00665ca436" "proxy-server 3656266" 0.0313 "-" 3656266 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/22 PUT /v1/a/c HTTP/1.0 201 - - t - - - txc88412cb0cb74c96baf17-00665ca436 - 0.0758 - - 1717347382.539592743 1717347382.615358829 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txeff8ef4aabbe438c834d0-00665ca436" "proxy-server 3656266" 0.0030 "-" 3656266 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:45331/sda1/1/a1/c1" "txeff8ef4aabbe438c834d0-00665ca436" "container-server 3656266" 0.0021 "-" 3656266 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:38045/sdb1/1/a1/c1" "txeff8ef4aabbe438c834d0-00665ca436" "container-server 3656266" 0.0023 "-" 3656266 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txeff8ef4aabbe438c834d0-00665ca436" "proxy-server 3656266" 0.0429 "-" 3656266 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txeff8ef4aabbe438c834d0-00665ca436" "proxy-server 3656266" 0.0291 "-" 3656266 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/22 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txeff8ef4aabbe438c834d0-00665ca436 - 0.0603 - - 1717347382.618152857 1717347382.678413391 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx5e9205d43f9d4f338ce22-00665ca436" "proxy-server 3656266" 0.1191 "-" 3656266 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:38045/sdb1/2/a/c1" "tx5e9205d43f9d4f338ce22-00665ca436" "container-server 3656266" 0.0080 "-" 3656266 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:45331/sda1/2/a/c1" "tx5e9205d43f9d4f338ce22-00665ca436" "container-server 3656266" 0.0021 "-" 3656266 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx5e9205d43f9d4f338ce22-00665ca436" "proxy-server 3656266" 0.0476 "-" 3656266 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx5e9205d43f9d4f338ce22-00665ca436" "proxy-server 3656266" 0.0310 "-" 3656266 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/22 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx5e9205d43f9d4f338ce22-00665ca436 - 0.1823 - - 1717347382.681370258 1717347382.863675356 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa724a6c2f901419595776-00665ca436" "proxy-server 3656266" 0.0120 "-" 3656266 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:45331/sda1/1/a/c2" "txa724a6c2f901419595776-00665ca436" "container-server 3656266" 0.0083 "-" 3656266 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:38045/sdb1/1/a/c2" "txa724a6c2f901419595776-00665ca436" "container-server 3656266" 0.0023 "-" 3656266 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa724a6c2f901419595776-00665ca436" "proxy-server 3656266" 0.0511 "-" 3656266 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa724a6c2f901419595776-00665ca436" "proxy-server 3656266" 0.0369 "-" 3656266 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/22 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txa724a6c2f901419595776-00665ca436 - 0.0780 - - 1717347382.866450071 1717347382.944408417 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:49823/sda1/0/a/c" "txf8c9b4dac79241c8863ef-00665ca434" "container-server 3656243" 0.0036 "-" 3656243 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:58213/sdb1/0/a/c" "txf8c9b4dac79241c8863ef-00665ca434" "container-server 3656243" 0.0086 "-" 3656243 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txf8c9b4dac79241c8863ef-00665ca434" "proxy-server 3656243" 0.9126 "-" 3656243 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txf8c9b4dac79241c8863ef-00665ca434" "proxy-server 3656243" 0.3244 "-" 3656243 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a/c HTTP/1.0 201 - - t - - - txf8c9b4dac79241c8863ef-00665ca434 - 0.9338 - - 1717347380.729690075 1717347381.663532972 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "HEAD /sda1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx41b448fe7b1144b990cda-00665ca435" "proxy-server 3656243" 0.0036 "-" 3656243 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:49823/sda1/2/a1/c1" "tx41b448fe7b1144b990cda-00665ca435" "container-server 3656243" 0.0023 "-" 3656243 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:58213/sdb1/2/a1/c1" "tx41b448fe7b1144b990cda-00665ca435" "container-server 3656243" 0.0045 "-" 3656243 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx41b448fe7b1144b990cda-00665ca435" "proxy-server 3656243" 0.0338 "-" 3656243 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx41b448fe7b1144b990cda-00665ca435" "proxy-server 3656243" 0.0595 "-" 3656243 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx41b448fe7b1144b990cda-00665ca435 - 0.0776 - - 1717347381.666840792 1717347381.744408607 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txda2530f142a54ad29493b-00665ca435" "proxy-server 3656243" 0.0221 "-" 3656243 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:49823/sda1/0/a/c1" "txda2530f142a54ad29493b-00665ca435" "container-server 3656243" 0.0030 "-" 3656243 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:58213/sdb1/0/a/c1" "txda2530f142a54ad29493b-00665ca435" "container-server 3656243" 0.0120 "-" 3656243 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txda2530f142a54ad29493b-00665ca435" "proxy-server 3656243" 0.0674 "-" 3656243 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txda2530f142a54ad29493b-00665ca435" "proxy-server 3656243" 0.0474 "-" 3656243 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txda2530f142a54ad29493b-00665ca435 - 0.1083 - - 1717347381.747737408 1717347381.856061935 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf33bdc858d114452b3a9b-00665ca435" "proxy-server 3656243" 0.0124 "-" 3656243 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:58213/sdb1/2/a/c2" "txf33bdc858d114452b3a9b-00665ca435" "container-server 3656243" 0.0184 "-" 3656243 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:49823/sda1/2/a/c2" "txf33bdc858d114452b3a9b-00665ca435" "container-server 3656243" 0.0042 "-" 3656243 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf33bdc858d114452b3a9b-00665ca435" "proxy-server 3656243" 0.0500 "-" 3656243 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txf33bdc858d114452b3a9b-00665ca435" "proxy-server 3656243" 0.0783 "-" 3656243 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txf33bdc858d114452b3a9b-00665ca435 - 0.1090 - - 1717347381.859619379 1717347381.968585730 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:25 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0231 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:25 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0195 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0016 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0196 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0091 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0044 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0157 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0077 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0100 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0083 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0085 "-" 3656251 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0004 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0203 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0123 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0003 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0272 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0207 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0419 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0051 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0236 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0041 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0048 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0201 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0052 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0221 "-" 3656251 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3656251 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0295 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0121 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0124 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0052 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0047 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0046 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:31 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0042 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0043 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0120 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0041 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0043 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0047 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shard_ranges ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_shards_timing_metrics ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 345, in GET resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_error ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_non_json ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_with_both_header_and_xml ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/55/28 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0016 - - 1717347328.338137627 1717347328.339730263 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/55/28 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0015 - - 1717347328.340748310 1717347328.342211723 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test WARNING: The following digest algorithms are configured but not supported: not-a-valid-digest, md5 test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test DEBUG: tenant mismatch: BLAN_a != tenant_id test DEBUG: allow user with role(s) swiftoperator,admin as account admin test DEBUG: allow user with role(s) swiftoperator,admin as account admin test DEBUG: allowing proxy 127.0.0.1 for container-sync test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: Names allowed in acls. test DEBUG: authorizing http://example.com/index.html via referer ACL test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:* allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL: allowme authorizing test DEBUG: Names allowed in acls. test DEBUG: user *:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user *:* allowed in ACL authorizing. test DEBUG: User tenant_name:user_name is not allowed to delete its own account test DEBUG: User tenant_id has reseller admin authorizing test DEBUG: Using identity: {'user': ('user_id', 'user_name'), 'tenant': ('test', 'tenant_name'), 'roles': ['reselleradmin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: User test has reseller admin authorizing test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_name:user_id allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_name allowed in ACL authorizing. test DEBUG: Names allowed in acls. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: user tenant_id:user_id allowed in ACL authorizing. test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_empty_shard_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_key_marker ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_multi_chars_delimiter ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Search value matched 0 devices. The on-disk ring builder is unchanged. Unknown command: unknown swift-ring-builder 1.3 swift-ring-builder Shows information about the ring and the devices within. Output includes a table that describes the report parameters (id, region, port, flags, etc). flags: possible values are 'DEL' and '' DEL - indicates that the device is marked for removal from ring and will be removed in next rebalance. swift-ring-builder add [r]z-:[R:]/_ [[r]z-:[R:]/_ ] ... Where and are replication ip and port. or swift-ring-builder add --region --zone --ip --port [--replication-ip ] [--replication-port ] --device --weight [--meta ] Adds devices to the ring with the given information. No partitions will be assigned to the new device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder cancel_increase_partition_power Cancel the increase of the partition power. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup unneeded links. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder create Creates with 2^ partitions and . is number of hours to restrict moving a partition more than once. swift-ring-builder dispersion [options] Output report on dispersion. --recalculate option will rebuild cached dispersion info and save builder --verbose option will display dispersion graph broken down by tier You can filter which tiers are evaluated to drill down using a regex in the optional search_filter argument. i.e. swift-ring-builder dispersion "r\d+z\d+$" -v ... would only display rows for the zone tiers swift-ring-builder dispersion ".*\-[^/]*$" -v ... would only display rows for the server tiers The reports columns are: Tier : the name of the tier parts : the total number of partitions with assignment in the tier % : the percentage of parts in the tier with replicas over assigned max : maximum replicas a part should have assigned at the tier 0 - N : the number of parts with that many replicas assigned e.g. Tier: parts % max 0 1 2 3 r1z1 1022 79.45 1 2 210 784 28 r1z1 has 1022 total parts assigned, 79% of them have more than the recommend max replica count of 1 assigned. Only 2 parts in the ring are *not* assigned in this tier (0 replica count), 210 parts have the recommend replica count of 1, 784 have 2 replicas, and 28 sadly have all three replicas in this tier. swift-ring-builder finish_increase_partition_power Finally removes the next_part_power flag. Has to be run after the swift-object-relinker tool has been used to cleanup old existing data. A write_ring command is needed to make the change take effect. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder increase_partition_power Increases the partition power by one. Needs to be run after prepare_increase_partition_power has been run and all existing data has been relinked using the swift-object-relinker tool. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to cleanup old data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder list_parts [] .. or swift-ring-builder list_parts --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Returns a 2 column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned partition number and the second column is the number of device matches for that partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command could take a while to run. swift-ring-builder prepare_increase_partition_power Prepare the ring to increase the partition power by one. A write_ring command is needed to make the change take effect. Once the updated rings have been deployed to all servers you need to run the swift-object-relinker tool to relink existing data. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you increase the partition power and deploy changed rings, you may introduce unavailability in your cluster. This has an end-user impact. Make sure you execute required operations to increase the partition power accurately. swift-ring-builder pretend_min_part_hours_passed Resets the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. ***************************** USE THIS WITH EXTREME CAUTION ***************************** If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user impact. swift-ring-builder rebalance [options] Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder remove [search-value ...] [--yes] or swift-ring-builder remove --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder search or swift-ring-builder search --region --zone --ip --port --replication-ip --replication-port --device --meta --weight Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Shows information about matching devices. swift-ring-builder set_info :[R:]/_ [ :[R:]/_] ... [--yes] or swift-ring-builder set_info --ip --port --replication-ip --replication-port --device --meta --change-ip --change-port --change-replication-ip --change-replication-port --change-device --change-meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. For each search-value, resets the matched device's information. This information isn't used to assign partitions, so you can use 'write_ring' afterward to rewrite the current ring with the newer device information. Any of the parts are optional in the final :/_ parameter; just give what you want to change. For instance set_info d74 _"snet: 5.6.7.8" would just update the meta data for device id 74. Option --yes assume a yes response to all questions. swift-ring-builder set_min_part_hours Changes the to the given . This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder set_overload [%] Changes the overload factor to the given . A rebalance is needed to make the change take effect. swift-ring-builder set_region [ set_region --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' regions. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_replicas Changes the replica count to the given . may be a floating-point value, in which case some partitions will have floor() replicas and some will have ceiling() in the correct proportions. A rebalance is needed to make the change take effect. swift-ring-builder set_weight [ ] ... [--yes] or swift-ring-builder set_weight --region --zone --ip --port --replication-ip --replication-port --device --meta --weight [--yes] Where , and are replication ip, hostname and port. and are the search weight and new weight values respectively. Any of the options are optional in both cases. Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder set_zone [ set_zone --region --zone --ip --port --replication-ip --replication-port --device --meta [--yes] Where , and are replication ip, hostname and port. Any of the options are optional in both cases. Resets the devices' zones. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. Option --yes assume a yes response to all questions. swift-ring-builder validate Just runs the validation routines on the ring. swift-ring-builder write_builder [min_part_hours] Recreate a builder from a ring file (lossy) if you lost your builder backups. (Protip: don't lose your builder backups). [min_part_hours] is one of those numbers lost to the builder, you can change it with set_min_part_hours. swift-ring-builder write_ring Just rewrites the distributable ring file. This is done automatically after a successful rebalance, so really this is only useful after one or more 'set_info' calls when no rebalance is needed but you want to send out the new device information. test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile test.unit.cli.test_ringbuilder.TestCommands.test_use_ringfile_as_builderfile ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate test.unit.cli.test_ringbuilder.TestCommands.test_validate ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_composite_builder_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_corrupted_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_empty_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error test.unit.cli.test_ringbuilder.TestCommands.test_validate_generic_error ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_accessible_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file test.unit.cli.test_ringbuilder.TestCommands.test_validate_non_existent_file ... ok test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk test.unit.cli.test_ringbuilder.TestCommands.test_warn_at_risk ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder test.unit.cli.test_ringbuilder.TestCommands.test_write_builder ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_after_device_removal ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_fractional_replicas ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version test.unit.cli.test_ringbuilder.TestCommands.test_write_builder_no_version ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_empty_ring ... ok test.unit.cli.test_ringbuilder.TestCommands.test_write_ring test.unit.cli.test_ringbuilder.TestCommands.test_write_ring ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_in_json ... ok test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json test.unit.common.middleware.s3api.test_s3api.TestListingMiddleware.test_s3_etag_non_json ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting test.unit.common.middleware.test_container_sync.TestContainerSync.test_allow_full_urls_setting ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_in_realms_conf ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_invalid ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set test.unit.common.middleware.test_container_sync.TestContainerSync.test_current_not_set ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter test.unit.common.middleware.test_container_sync.TestContainerSync.test_filter ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info test.unit.common.middleware.test_container_sync.TestContainerSync.test_info ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh test.unit.common.middleware.test_container_sync.TestContainerSync.test_info_always_fresh ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_invalid_sig ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf test.unit.common.middleware.test_container_sync.TestContainerSync.test_missing_from_realms_conf ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args test.unit.common.middleware.test_container_sync.TestContainerSync.test_not_enough_args ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through test.unit.common.middleware.test_container_sync.TestContainerSync.test_pass_through ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_realm_miss ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss test.unit.common.middleware.test_container_sync.TestContainerSync.test_user_key_miss ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig ... ok test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 test.unit.common.middleware.test_container_sync.TestContainerSync.test_valid_sig2 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_and_path ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_root_obj ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_path_trailing_slash ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_container_with_root_path_obj_slash_v1 ... ok test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_account_with_path_root_container ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_Range_with_invalid_value ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd9a2fa2e-768f-4181-bd46-22c2bbb5669a-sda', '1717347327.62827', '1717347329', '1717347328', '', '1717347329', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d9a2fa2e-768f-4181-bd46-22c2bbb5669a-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.37968 seconds (0.42022/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpj8rxl0y1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2c65a3dc215d2da9f24dc35d7006fa99', 'ee96e8b0-8649-4e9d-934f-cf50f6410674-sda', '1717347330.13450', '1717347332', '1717347331', '', '1717347332', 1, 0) test DEBUG: /tmp/tmpj8rxl0y1/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2c65a3dc215d2da9f24dc35d7006fa99', 'ee96e8b0-8649-4e9d-934f-cf50f6410674-sda', '1717347330.13450', '1717347332', '1717347331', '', '1717347332', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ee96e8b0-8649-4e9d-934f-cf50f6410674-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12760 seconds (7.83693/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp01w6gjrq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bc98c299-62d8-49f6-82ed-d18ea5d46b81-sda', '1717347337.05259', '1717347340', '1717347339', '', '1717347340', 0, 1) test DEBUG: /tmp/tmp01w6gjrq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bc98c299-62d8-49f6-82ed-d18ea5d46b81-sda', '1717347337.05259', '1717347340', '1717347339', '', '1717347340', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('bc98c299-62d8-49f6-82ed-d18ea5d46b81-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09873 seconds (10.12837/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp7sp6422p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '44e4e54c1bb6630b60830faf04e15e95', 'ac14b019-e878-4f9e-845c-4b7e9bcf58a9-sda', '1717347337.28522', '1717347340', '1717347339', '', '1717347340', 1, 0) test DEBUG: /tmp/tmp7sp6422p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347342.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], 'ac14b019-e878-4f9e-845c-4b7e9bcf58a9-sda') test DEBUG: /tmp/tmp7sp6422p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0150776s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '44e4e54c1bb6630b60830faf04e15e95', 'ac14b019-e878-4f9e-845c-4b7e9bcf58a9-sda', '1717347337.28522', '1717347340', '1717347339', '', '1717347340', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('ac14b019-e878-4f9e-845c-4b7e9bcf58a9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.12459 seconds (8.02649/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpgwd1gtm3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8416d166-19eb-4c9f-936d-ba820ff6d7e5-sda', '1717347337.52567', '1717347340', '1717347339', '', '1717347340', 0, 0) test DEBUG: /tmp/tmpgwd1gtm3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8416d166-19eb-4c9f-936d-ba820ff6d7e5-sda', '1717347337.52567', '1717347340', '1717347339', '', '1717347340', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8416d166-19eb-4c9f-936d-ba820ff6d7e5-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.11188 seconds (8.93853/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp2fdx7ydh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '44e4e54c1bb6630b60830faf04e15e95', '5d84b69a-4e6f-42d3-b612-c35ed683530d-sda', '1717347337.74556', '1717347340', '1717347339', '', '1717347340', 1, 0) test DEBUG: /tmp/tmp2fdx7ydh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '44e4e54c1bb6630b60830faf04e15e95', '5d84b69a-4e6f-42d3-b612-c35ed683530d-sda', '1717347337.74556', '1717347340', '1717347339', '', '1717347340', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('5d84b69a-4e6f-42d3-b612-c35ed683530d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13560 seconds (7.37456/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpzpmtg2il/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2df2d5ce-7378-46f3-a001-f98e1729fc80-sda', '1717347337.99703', '1717347340', '1717347339', '', '1717347340', 0, 1) test DEBUG: /tmp/tmpzpmtg2il/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2df2d5ce-7378-46f3-a001-f98e1729fc80-sda', '1717347337.99703', '1717347340', '1717347339', '', '1717347340', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2df2d5ce-7378-46f3-a001-f98e1729fc80-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07421 seconds (13.47466/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_create ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txfd64d771d47e4b3fafce2-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txdcfe6dde5a68493d87f39-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.2:1002/sdc (txn: txa16fb7b3e4bf4c5b8b050-00665ca415) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txa16fb7b3e4bf4c5b8b050-00665ca415) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.1:1001/sdb (txn: txa16fb7b3e4bf4c5b8b050-00665ca415) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: txa16fb7b3e4bf4c5b8b050-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: tx07e4262d46364b08b99d9-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read object during GET (retrying) 10.0.0.0:1000/sda (txn: txb9837847321a48f39122d-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.0:1000/sda (txn: txe2739acc8e38493d912b1-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.0:1000/sda (txn: txe8e01dee62ab4b39af719-00665ca415) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.2:1002/sdc (txn: txe8e01dee62ab4b39af719-00665ca415) proxy-server ERROR: Trying to read next part of object multi-part GET (retrying) 10.0.0.1:1001/sdb (txn: txe8e01dee62ab4b39af719-00665ca415) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txbde2582213ea4f3f95db6-00665ca416) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txbde2582213ea4f3f95db6-00665ca416) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txb6f6e3e5272440508a558-00665ca416) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txbf9a5c1ae8604f0b97081-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: txbf9a5c1ae8604f0b97081-00665ca416) proxy-server ERROR: Object HEAD returning 503 for [] (txn: txbf9a5c1ae8604f0b97081-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx7591b6a8f68548bf88208-00665ca416) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx7591b6a8f68548bf88208-00665ca416) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx7591b6a8f68548bf88208-00665ca416) proxy-server ERROR: Object HEAD returning 503 for [507] (txn: tx7591b6a8f68548bf88208-00665ca416) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx2eae378dc09f45469b24f-00665ca416) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx2eae378dc09f45469b24f-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Timeout (Nones) (txn: tx2eae378dc09f45469b24f-00665ca416) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_POST_non_int_delete_after ... ok proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1eebfd27ed4743d6a80b0-00665ca3e1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.28:1028/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.29:1029/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.30:1030/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.31:1031/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.32:1032/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.33:1033/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.34:1034/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.35:1035/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.36:1036/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.37:1037/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.38:1038/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.39:1039/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.40:1040/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.41:1041/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.42:1042/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.43:1043/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.44:1044/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.45:1045/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.46:1046/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.47:1047/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.48:1048/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.49:1049/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_PUT_requires_length ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Setting object count to 1072 (txn: txe6cdf8be6b9f428cbdaf4-00665ca420) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Setting object count to 1072 (txn: tx6729e729366c4f5aa108b-00665ca423) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx3d8991e9f5ca42c091c99-00665ca427) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx3d8991e9f5ca42c091c99-00665ca427) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3d8991e9f5ca42c091c99-00665ca427) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx3d8991e9f5ca42c091c99-00665ca427) test ERROR: Aborting listing from shards due to bad response: [404] (txn: tx3d8991e9f5ca42c091c99-00665ca427) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: Got non-JSON response trying to list /bucket: b'obj0\nobj1\nobj2\nobj3\nobj4\nobj5\nobj6\nobj7\nobj8\nobj9\nobj10\nobj1...' test ERROR: Expecting value: line 1 column 1 (char 0): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 348, in GET objects = json.loads(resp.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_owner_permission ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=0-3', 'Date': 'Sun, 02 Jun 2024 16:55:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=3-3', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1-', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=-3', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': '', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'hoge', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=1', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-1', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Range': 'bytes=5-10', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_Range_with_invalid_value ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9992&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1032 (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: Setting object count to 1032 (txn: txb5c602d59ce14aa4ba45c-00665ca435) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=10000&marker=&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Found 8 objects in shard (state=unsharded), total = 8 (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=9992&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Found 40 objects in shard (state=unsharded), total = 48 (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: Setting object count to 48 (txn: tx02a5c408fcc546e29154e-00665ca435) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Found 0 objects in shard (state=unsharded), total = 40 (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9960&marker=h&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1064 (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: Setting object count to 1064 (txn: txd397dedae9d7409d93aea-00665ca435) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&format=json&states=listing (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&format=json&limit=10000&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: Get listing from /v1/.shards_a/c_?marker=koolaid&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txd5f331811c554defbf9a7-00665ca436) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=koolaid&reverse=true&format=json&states=listing (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Get listing from /v1/.shards_a/c_pie?marker=koolaid&reverse=true&format=json&limit=10000&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Get listing from /v1/.shards_a/c_ham?marker=koolaid&reverse=true&format=json&limit=10000&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txef9d27cca6184aaca74f1-00665ca436) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_gap_in_shards_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_marker_beyond_end_marker_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mix_ok_and_unavailable_shards ... ok The can be of the form:: drz-:R:/ _ Where and are replication ip and port. Any part is optional, but you must include at least one part. Examples:: d74 Matches the device id 74 r4 Matches devices in region 4 z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 R5.6.7.8 Matches devices that use replication ip 5.6.7.8 R:5678 Matches devices that use replication port 5678 1.2.3.4R5.6.7.8 Matches devices that use ip 1.2.3.4 and replication ip 5.6.7.8 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 Most specific example:: d74r4z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8" Nerd explanation: All items require their single character prefix except the ip, in which case the - is optional unless the device id or zone is also included. Quick list: add cancel_increase_partition_power create dispersion finish_increase_partition_power increase_partition_power list_parts prepare_increase_partition_power pretend_min_part_hours_passed rebalance remove search set_info set_min_part_hours set_overload set_region set_replicas set_weight set_zone validate write_builder write_ring Exit codes: 0 = operation successful 1 = operation completed with warnings 2 = error Problem occurred while reading builder file: /tmp/tmpe3nxx8xc/tmpbqpy3m32. Generic error occurred Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 6.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 6.00 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 Device d4r1z1-127.0.0.1:6200R127.0.0.1:6200/sdb_"" with 1.0 weight got id 4 Device d5r1z1-127.0.0.1:6200R127.0.0.1:6200/sdc_"" with 1.0 weight got id 5 Reassigned 192 (300.00%) partitions. Balance is now 109.38. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 109.38 indicates you should push this ring, wait at least 1 hours, and rebalance/repush. ------------------------------------------------------------------------------- d0r0z0-127.0.0.1:6200R127.0.0.1:6200/sda1_"some meta data" marked for removal and will be removed next rebalance. Reassigned 48 (75.00%) partitions. Balance is now 57.29. Dispersion is now 0.52 ------------------------------------------------------------------------------- NOTE: Dispersion of 0.520833 indicates some parts are not optimally dispersed. You may want to adjust some device weights, increase the overload or review the dispersion report. ------------------------------------------------------------------------------- Note: using /tmp/tmpdqrmx2bq/tmpxf_ga6h8.builder instead of /tmp/tmpdqrmx2bq/tmpxf_ga6h8.ring.gz as builder file Reassigned 1 (1.56%) partitions. Balance is now 100.00. Dispersion is now 0.00 ------------------------------------------------------------------------------- NOTE: Balance of 100.00 indicates you should push this ring, wait at least 24 hours, and rebalance/repush. ------------------------------------------------------------------------------- Reassigned 24 (37.50%) partitions. Balance is now 1.04. Dispersion is now 0.00 Note: using /tmp/tmpi73szxrt/tmpkcgub9fy.builder instead of /tmp/tmpi73szxrt/tmpkcgub9fy.ring.gz as builder file Note: using /tmp/tmppqxtbwtn/tmpd9mkt2lu.builder instead of /tmp/tmppqxtbwtn/tmpd9mkt2lu.ring.gz as builder file Reassigned 192 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 test ERROR: Invalid current //REALM/CLUSTER (foo) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) test ERROR: Unknown current //REALM/CLUSTER (//FOO/BAR) {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/obj/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c//v1', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash test.unit.common.middleware.test_domain_remap.TestDomainRemapClientMangling.test_domain_remap_with_path_root_and_path_no_slash ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_account ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_blank_container ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through test.unit.common.middleware.test_listing_formats.TestListingFormats.test_pass_through ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_but_not_terrible_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_bad_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_not_really_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json test.unit.common.middleware.test_listing_formats.TestListingFormats.test_static_web_pretend_to_be_giant_json ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_does_not_duplicate ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present test.unit.common.middleware.test_listing_formats.TestListingFormats.test_update_vary_if_present ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_account_with_reserved ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_container_with_reserved ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_txt_head ... ok test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head test.unit.common.middleware.test_listing_formats.TestListingFormats.test_valid_content_type_on_xml_head ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_ratelimiting ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_get_manifest_with_submanifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_passthrough ... ok test INFO: - - - [02/Jun/2024:16:55:32 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0041 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:36 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:36 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0043 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:36 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0043 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:36 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:36 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0092 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0042 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0076 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0033 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0082 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0032 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0036 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0085 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0056 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0055 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0057 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmpyhfviaoy/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0052 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0093 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0057 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0158 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0064 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0056 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0043 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0139 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0039 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:37 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0041 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:43 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.3048 "-" 3656251 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3656251 2 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:44 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0423 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0093 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:44 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0128 "-" 3656251 - test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_sys_metadata ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_ratelimiting ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_account_update ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_container_timing_metrics ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_repeated_segments_uses_multi_range_requests ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_get_manifest_with_submanifest_bytes_in_content_type ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_multiple_ranges_including_suffix_get_manifest ... ok test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest test.unit.common.middleware.test_slo.TestSloGetOldManifests.test_range_get_manifest ... ok test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception test.unit.common.middleware.test_tempauth.TestUtilityMethods.test_account_acls_bad_path_raises_exception ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error test.unit.common.test_base_storage_server.TestBaseStorageServer.test_OPTIONS_error ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods test.unit.common.test_base_storage_server.TestBaseStorageServer.test_allowed_methods ... ok test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type test.unit.common.test_base_storage_server.TestBaseStorageServer.test_server_type ... ok test.unit.common.test_daemon.TestRunDaemon.test_fork_workers test.unit.common.test_daemon.TestRunDaemon.test_fork_workers ... ok test.unit.common.test_daemon.TestRunDaemon.test_forked_worker test.unit.common.test_daemon.TestRunDaemon.test_forked_worker ... ok test.unit.common.test_daemon.TestRunDaemon.test_restart_workers test.unit.common.test_daemon.TestRunDaemon.test_restart_workers ... ok test.unit.common.test_daemon.TestRunDaemon.test_run test.unit.common.test_daemon.TestRunDaemon.test_run ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon test.unit.common.test_daemon.TestRunDaemon.test_run_daemon ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_change ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_for_existing_non_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_no_policy_specified ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_diff_tz ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_dir_with_duplicate_var ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var test.unit.common.test_daemon.TestRunDaemon.test_run_daemon_from_conf_file_with_duplicate_var ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_dir ... ok test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file test.unit.common.test_daemon.TestRunDaemon.test_run_deamon_from_conf_file ... ok test.unit.common.test_daemon.TestRunDaemon.test_signal test.unit.common.test_daemon.TestRunDaemon.test_signal ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_disappears_in_cleanup ... ok test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup test.unit.common.test_daemon.TestRunDaemon.test_worker_kills_pids_in_cleanup ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star test.unit.common.test_swob.TestConditionalIfMatch.test_match_star ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 test.unit.common.test_swob.TestConditionalIfMatch.test_match_star_on_404 ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_quoted_simple_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_conditional_etag_no_match ... ok test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match test.unit.common.test_swob.TestConditionalIfMatch.test_simple_match ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent test.unit.common.test_swob.TestConditionalIfModifiedSince.test_absent ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before test.unit.common.test_swob.TestConditionalIfModifiedSince.test_before ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater test.unit.common.test_swob.TestConditionalIfModifiedSince.test_greater ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored test.unit.common.test_swob.TestConditionalIfModifiedSince.test_out_of_range_is_ignored ... ok test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same test.unit.common.test_swob.TestConditionalIfModifiedSince.test_same ... ok test.unit.common.test_utils.TestFallocate.test_arg_bounds test.unit.common.test_utils.TestFallocate.test_arg_bounds ... ok test.unit.common.test_utils.TestFallocate.test_fallocate test.unit.common.test_utils.TestFallocate.test_fallocate ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error test.unit.common.test_utils.TestFallocate.test_fallocate_fatal_error ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_offset test.unit.common.test_utils.TestFallocate.test_fallocate_offset ... ok test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors test.unit.common.test_utils.TestFallocate.test_fallocate_silent_errors ... ok test.unit.common.test_utils.TestFallocate.test_no_fallocates_available test.unit.common.test_utils.TestFallocate.test_no_fallocates_available ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback test.unit.common.test_utils.TestFallocate.test_posix_fallocate_fallback ... ok test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset test.unit.common.test_utils.TestFallocate.test_posix_fallocate_offset ... ok test.unit.common.test_utils.TestShardRange.test_constants test.unit.common.test_utils.TestShardRange.test_constants ... ok test.unit.common.test_utils.TestShardRange.test_copy test.unit.common.test_utils.TestShardRange.test_copy ... ok test.unit.common.test_utils.TestShardRange.test_deleted_setter test.unit.common.test_utils.TestShardRange.test_deleted_setter ... ok test.unit.common.test_utils.TestShardRange.test_epoch_setter test.unit.common.test_utils.TestShardRange.test_epoch_setter ... ok test.unit.common.test_utils.TestShardRange.test_find_ancestors test.unit.common.test_utils.TestShardRange.test_find_ancestors ... ok test.unit.common.test_utils.TestShardRange.test_find_root test.unit.common.test_utils.TestShardRange.test_find_root ... ok test.unit.common.test_utils.TestShardRange.test_increment_meta test.unit.common.test_utils.TestShardRange.test_increment_meta ... ok test.unit.common.test_utils.TestShardRange.test_is_child_of test.unit.common.test_utils.TestShardRange.test_is_child_of ... ok test.unit.common.test_utils.TestShardRange.test_make_path test.unit.common.test_utils.TestShardRange.test_make_path ... ok test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter test.unit.common.test_utils.TestShardRange.test_meta_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_min_max_bounds test.unit.common.test_utils.TestShardRange.test_min_max_bounds ... ok test.unit.common.test_utils.TestShardRange.test_name test.unit.common.test_utils.TestShardRange.test_name ... ok test.unit.common.test_utils.TestShardRange.test_name_validation test.unit.common.test_utils.TestShardRange.test_name_validation ... ok test.unit.common.test_utils.TestShardRange.test_repr test.unit.common.test_utils.TestShardRange.test_repr ... ok test.unit.common.test_utils.TestShardRange.test_resolve_state test.unit.common.test_utils.TestShardRange.test_resolve_state ... ok test.unit.common.test_utils.TestShardRange.test_row_count test.unit.common.test_utils.TestShardRange.test_row_count ... ok test.unit.common.test_utils.TestShardRange.test_set_deleted test.unit.common.test_utils.TestShardRange.test_set_deleted ... ok test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation test.unit.common.test_utils.TestShardRange.test_shard_range_initialisation ... ok test.unit.common.test_utils.TestShardRange.test_sort_key test.unit.common.test_utils.TestShardRange.test_sort_key ... ok test.unit.common.test_utils.TestShardRange.test_sort_key_order test.unit.common.test_utils.TestShardRange.test_sort_key_order ... ok test.unit.common.test_utils.TestShardRange.test_state_setter test.unit.common.test_utils.TestShardRange.test_state_setter ... ok test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter test.unit.common.test_utils.TestShardRange.test_state_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_timestamp_setter test.unit.common.test_utils.TestShardRange.test_timestamp_setter ... ok test.unit.common.test_utils.TestShardRange.test_to_from_dict test.unit.common.test_utils.TestShardRange.test_to_from_dict ... ok test.unit.common.test_utils.TestShardRange.test_update_meta test.unit.common.test_utils.TestShardRange.test_update_meta ... ok test.unit.common.test_utils.TestShardRange.test_update_state test.unit.common.test_utils.TestShardRange.test_update_state ... ok test.unit.common.test_utils.TestShardRange.test_update_tombstones test.unit.common.test_utils.TestShardRange.test_update_tombstones ... ok test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser test.unit.common.test_utils.TestSocketStringParser.test_socket_string_parser ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_sharding_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_attempts ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_fails_to_migrate_table ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_migrates_table ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reconciler_sync ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester WRITE_ACP permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_grant_write_acp ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_invalid_canned_acl ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_save_and_load_record_from_pending_file ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_with_prefix_and_delimiter ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_POST_error ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpc79an49f/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'aed8527c499c8191c1cf945d74edca97', '04c65577-68d7-4992-8eee-51803f478a0d-sda', '1717347342.02849', '1717347344', '1717347343', '', '1717347344', 1, 1) test DEBUG: /tmp/tmpc79an49f/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347345.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '04c65577-68d7-4992-8eee-51803f478a0d-sda') test DEBUG: /tmp/tmpc79an49f/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0229146s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'aed8527c499c8191c1cf945d74edca97', '04c65577-68d7-4992-8eee-51803f478a0d-sda', '1717347342.02849', '1717347344', '1717347343', '', '1717347344', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('04c65577-68d7-4992-8eee-51803f478a0d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.24425 seconds (4.09416/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpnplr2i62/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '78248298-098b-46aa-8c0e-672232a785e1-sda', '1717347344.47950', '1717347347', '1717347346', '', '1717347347', 0, 1) test DEBUG: /tmp/tmpnplr2i62/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '78248298-098b-46aa-8c0e-672232a785e1-sda', '1717347344.47950', '1717347347', '1717347346', '', '1717347347', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('78248298-098b-46aa-8c0e-672232a785e1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10910 seconds (9.16629/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp018r217d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a75c44c8646b5eec3489192ae81085b7', '700a31b5-6606-40e4-96b1-008d11f5f5d9-sda', '1717347344.67377', '1717347347', '1717347346', '', '1717347347', 1, 0) test DEBUG: /tmp/tmp018r217d/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a75c44c8646b5eec3489192ae81085b7', '700a31b5-6606-40e4-96b1-008d11f5f5d9-sda', '1717347344.67377', '1717347347', '1717347346', '', '1717347347', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('700a31b5-6606-40e4-96b1-008d11f5f5d9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14276 seconds (7.00496/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpdc2rbxkn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9d93e832-70f4-4d32-94ed-14dfeaf479a1-sda', '1717347344.93063', '1717347348', '1717347347', '', '1717347348', 0, 0) test DEBUG: /tmp/tmpdc2rbxkn/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '9d93e832-70f4-4d32-94ed-14dfeaf479a1-sda', '1717347344.93063', '1717347348', '1717347347', '', '1717347348', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('9d93e832-70f4-4d32-94ed-14dfeaf479a1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09225 seconds (10.83986/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmprrimlwco/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f8ee1dbb11605e8b8ba4d09b811180aa', '53831db6-3a93-4210-af4c-aa881a353163-sda', '1717347345.11616', '1717347349', '1717347348', '', '1717347349', 1, 1) test DEBUG: /tmp/tmprrimlwco/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347350.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '53831db6-3a93-4210-af4c-aa881a353163-sda') test DEBUG: /tmp/tmprrimlwco/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0155213s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'f8ee1dbb11605e8b8ba4d09b811180aa', '53831db6-3a93-4210-af4c-aa881a353163-sda', '1717347345.11616', '1717347349', '1717347348', '', '1717347349', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('53831db6-3a93-4210-af4c-aa881a353163-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15778 seconds (6.33796/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpa36bthki/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c901a421-e43b-41a4-aeda-f553d2f90687-sda', '1717347345.38762', '1717347349', '1717347348', '', '1717347349', 0, 1) test DEBUG: /tmp/tmpa36bthki/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'c901a421-e43b-41a4-aeda-f553d2f90687-sda', '1717347345.38762', '1717347349', '1717347348', '', '1717347349', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('c901a421-e43b-41a4-aeda-f553d2f90687-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09257 seconds (10.80217/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_local_recreate_policy_over_older_remote_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps test.unit.container.test_replicator.TestReplicatorSync.test_sync_merge_timestamps ... ok proxy-server ERROR: ERROR with Object server 10.0.0.50:1050/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.51:1051/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.52:1052/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.53:1053/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.54:1054/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server ERROR: ERROR with Object server 10.0.0.55:1055/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx82bc5b9db2864c199f428-00665ca3f5) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_determine_chunk_destinations_prioritize_more_missing ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_not_configured ... ok proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7397069e6a474c84914fb-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7397069e6a474c84914fb-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7397069e6a474c84914fb-00665ca416) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx21f6c22df71b4532b4d5b-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx21f6c22df71b4532b4d5b-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx21f6c22df71b4532b4d5b-00665ca416) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx40840c6e7d0f45dc99bd5-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx40840c6e7d0f45dc99bd5-00665ca416) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx40840c6e7d0f45dc99bd5-00665ca416) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_connect_exceptions ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_converts_delete_after_to_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_in_past ... ok {'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'QUERY_STRING': '', 'PATH_INFO': '/v1/AUTH_a/c/v1obj', 'SERVER_NAME': 'localhost', 'SERVER_PORT': '80', 'HTTP_HOST': 'c.AUTH_a.example.com', 'SERVER_PROTOCOL': 'HTTP/1.0', 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.errors': <_io.StringIO object at 0x7f7f324aad40>, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.input': } test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in name: '\x00bar\x00versions' test-listing WARNING: Account listing for a%E2%98%83 had reserved byte in subdir: '\x00foo_' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in name: '\x00bar\x00extra data' test-listing WARNING: Container listing for a%E2%98%83/c%F0%9F%8C%B4 had reserved byte in subdir: '\x00foo/' my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Worker mock-pid-0 exited my-daemon DEBUG: Worker mock-pid-1 exited my-daemon DEBUG: Worker mock-pid-2 exited my-daemon NOTICE: Finished 3656202 my-daemon DEBUG: Forked worker 3656202 finished my-daemon DEBUG: Spawned worker mock-pid-0 with {} my-daemon DEBUG: Spawned worker mock-pid-1 with {} my-daemon DEBUG: Spawned worker mock-pid-2 with {} my-daemon DEBUG: Daemon needs to change options, aborting workers my-daemon DEBUG: Cleaned up worker mock-pid-0 my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-3 with {} my-daemon DEBUG: Spawned worker mock-pid-4 with {} my-daemon DEBUG: Spawned worker mock-pid-5 with {} my-daemon DEBUG: Cleaned up worker mock-pid-3 my-daemon DEBUG: Cleaned up worker mock-pid-4 my-daemon DEBUG: Cleaned up worker mock-pid-5 my-daemon NOTICE: SIGTERM received () my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon DEBUG: Worker mock-pid exited my-daemon DEBUG: Spawned worker mock-pid with {'mock_options': True} my-daemon NOTICE: Worker mock-pid died my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 my-daemon DEBUG: Spawned worker mock-pid-1 with {'mock_options': True} my-daemon DEBUG: Spawned worker mock-pid-2 with {'mock_options': True} my-daemon DEBUG: Cleaned up worker mock-pid-1 my-daemon DEBUG: Cleaned up worker mock-pid-2 test ERROR: Problem finding shard upper in '/tmp/tmpa19xhqb2/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpa19xhqb2/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmps4edbbp4/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmps4edbbp4/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeReportedColumn.test_sync_merging ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_bad_value_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_in_worker ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_dump_recon_run_once_inline ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_final_recon_dump ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_detects_ring_change ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_is_healthy_rcache_update_waits_for_next_update ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_next_rcache_update_configured_with_stats_interval ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_dump_recon_run_once ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_no_workers_by_default ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_at_end_of_run_once ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_races_with_final_recon_dump ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_removes_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_recon_aggregation_waits_for_all_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_run_forever_recon_no_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_worker_logging ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_no_valid_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_devices_and_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_devices_and_overrides ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_lots_of_workers_and_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_no_devices ... ok test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices test.unit.obj.test_reconstructor.TestWorkerReconstructor.test_workers_with_some_workers_and_devices ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum test.unit.proxy.controllers.test_base.TestFuncs.test_base_have_quorum ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides test.unit.proxy.controllers.test_base.TestFuncs.test_best_response_overrides ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip test.unit.proxy.controllers.test_base.TestFuncs.test_bytes_to_skip ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_needs_req ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy test.unit.proxy.controllers.test_base.TestFuncs.test_container_info_preserves_storage_policy ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Setting object count to 1065 (txn: txe8e9a6e11a9a448fa0c96-00665ca439) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='onion') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=onion%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='rhubarb') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=rhubarb%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Setting object count to 1065 (txn: tx3a794991823143cbb21ff-00665ca43e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx79476ac874c04255a5f29-00665ca440) test DEBUG: GET listing from 0 shards for: /v1/a/c?marker=bb&end_marker=aa&format=json&states=listing (txn: tx79476ac874c04255a5f29-00665ca440) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.2:1002/sdc (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.0:1000/sda (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.1:1001/sdb (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.3:1003/sda (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.4:1004/sda (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: ERROR 503 b'""' Trying to GET /.shards_a/c_ From Container Server 10.0.0.5:1005/sda (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: Container GET returning 503 for [503, 503, 503, 503, 503, 503] (txn: txd4d7935c06044a57ad170-00665ca443) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txd4d7935c06044a57ad170-00665ca443) test ERROR: Aborting listing from shards due to bad response: [200, 200, 503] (txn: txd4d7935c06044a57ad170-00665ca443) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx953af4161f4d450d9f213-00665ca443) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx953af4161f4d450d9f213-00665ca443) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_mixed_policies_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_with_read_permission ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_GET_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:20 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_owner_permission ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 237, in HEAD return self._handle_acl(app, 'HEAD') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 136, in _handle_acl resp = self.req.get_acl_response(app, 'HEAD', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_POST_error ... ok Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 433, in set raise MemcacheConnectionError('failed set: %s' % msg) swift.common.memcached.MemcacheConnectionError: failed set: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/queue.py", line 116, in switch self.greenlet.switch(value) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1551, in _timing_stats result = func(cache, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 437, in set self._exception_occurred(server, e, cmd, conn_start_time, File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 283, in _exception_occurred elif isinstance(e, (socket.error, MemcacheConnectionError)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_bucket ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_GET_without_segment_bucket ... ok test INFO: - - - [02/Jun/2024:16:55:44 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0051 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0293 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:47 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0041 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:47 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:47 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0316 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:47 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0056 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.3964 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0188 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0188 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0074 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0367 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0182 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0380 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0168 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0208 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:54 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:54 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0162 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0190 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0104 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0200 "-" 3656251 - test ERROR: ERROR Account update failed with 127.0.0.1:46089/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0137 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0236 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0023 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0022 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0025 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0025 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3656251 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0006 "-" 3656251 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0153 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0043 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0005 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0178 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0122 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0111 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0048 "-" 3656251 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0189 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0038 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0262 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:55:55 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0046 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0195 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0038 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0044 "-" 3656251 - test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_obj_not_found ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347342.85440', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy ... ok proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECDuplicationObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it test.unit.proxy.controllers.test_obj.TestNumContainerUpdates.test_it ... ok test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptchu0fmc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2857f672c82c5f520d1c086081b33e6d', '44266215-e18b-4e9a-addb-f07ff529d844-sda', '1717347352.93294', '1717347356', '1717347355', '', '1717347356', 1, 1) test DEBUG: /tmp/tmptchu0fmc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2857f672c82c5f520d1c086081b33e6d', '44266215-e18b-4e9a-addb-f07ff529d844-sda', '1717347352.93294', '1717347356', '1717347355', '', '1717347356', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('44266215-e18b-4e9a-addb-f07ff529d844-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15802 seconds (6.32824/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpttmyxm6w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e3ba85b3-6006-416e-9025-083f9de149c2-sda', '1717347356.89942', '1717347361', '1717347360', '', '1717347361', 0, 0) test DEBUG: /tmp/tmpttmyxm6w/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e3ba85b3-6006-416e-9025-083f9de149c2-sda', '1717347356.89942', '1717347361', '1717347360', '', '1717347361', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e3ba85b3-6006-416e-9025-083f9de149c2-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08743 seconds (11.43818/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1m4v3yx8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7243d8b8034b7ea64129d4f7e8064c23', '87901daf-2e8e-4de5-ae8c-3bda93f68b3e-sda', '1717347357.07409', '1717347362', '1717347361', '', '1717347362', 1, 1) test DEBUG: /tmp/tmp1m4v3yx8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347363.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '87901daf-2e8e-4de5-ae8c-3bda93f68b3e-sda') test DEBUG: /tmp/tmp1m4v3yx8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0138528s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7243d8b8034b7ea64129d4f7e8064c23', '87901daf-2e8e-4de5-ae8c-3bda93f68b3e-sda', '1717347357.07409', '1717347362', '1717347361', '', '1717347362', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('87901daf-2e8e-4de5-ae8c-3bda93f68b3e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.40498 seconds (2.46926/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp66wb_nan/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '47f2dbb0-bc3e-461c-a741-7092402c09cb-sda', '1717347357.61302', '1717347362', '1717347361', '', '1717347362', 0, 0) test DEBUG: /tmp/tmp66wb_nan/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '47f2dbb0-bc3e-461c-a741-7092402c09cb-sda', '1717347357.61302', '1717347362', '1717347361', '', '1717347362', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('47f2dbb0-bc3e-461c-a741-7092402c09cb-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09988 seconds (10.01177/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpvz_ksyn3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7243d8b8034b7ea64129d4f7e8064c23', 'e616c20b-5880-44ea-b6d4-5eb486574ce2-sda', '1717347357.80082', '1717347362', '1717347361', '', '1717347362', 1, 1) test DEBUG: /tmp/tmpvz_ksyn3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '7243d8b8034b7ea64129d4f7e8064c23', 'e616c20b-5880-44ea-b6d4-5eb486574ce2-sda', '1717347357.80082', '1717347362', '1717347361', '', '1717347362', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e616c20b-5880-44ea-b6d4-5eb486574ce2-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.13954 seconds (7.16644/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b16e93e0-c0cd-4244-ba94-e51c46ad9eb8-sda', '1717347358.01696', '1717347358.00000', '0', '', '1717347358.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b16e93e0-c0cd-4244-ba94-e51c46ad9eb8-sda', '1717347358.01696', '1717347359.00000', '0', '', '1717347358.00000', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '307afe872cff9357ce20b13ce4a61a8f', '916653d7-9733-4700-921e-22bc6e675c1d-sda', '1717347358.08037', '1717347358.07366', '0', '', '1717347358.07366', 11, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_can_not_keep_up ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: txb2a935b6f5574cc28757c-00665ca416) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Expect: 100-continue on /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: Connection refused: Please insert ten dollars (txn: tx03f9651faf264fe9918a4-00665ca41a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.0:1000/sda (txn: tx551562719e4940cd8e256-00665ca41a) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.1:1001/sdb (txn: tx0d14c687146449d28ea2b-00665ca41a) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o.jpg From Object Server 10.0.0.2:1002/sdc (txn: tx52f883175674479586646-00665ca41a) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Expect: 100-continue on /a/c/o.jpg: Timeout (Nones) (txn: tx220bd5bf99ef4102a2508-00665ca41a) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Expect: 100-continue on /a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 981, in _make_putter putter = Putter.connect( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1850, in connect conn, expect_resp, final_resp, connect_duration = cls._make_connection( ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1814, in _make_connection conn = http_connect(ip, port, node['device'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx7d1a702865244acdbb07d-00665ca41a) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.2:1002/sdc (txn: tx18b6405818c94beeaa0be-00665ca41a) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx7593e5b33cbf466497ddf-00665ca41a) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to get final status of PUT to /v1/a/c/o.jpg: Timeout (Nones) (txn: tx5d085b32888848aa8e538-00665ca41a) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to get final status of PUT to /v1/a/c/o.jpg: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception (txn: tx3442627f871042f3811f4-00665ca41a) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: txb94207ddd3a1490e80c10-00665ca41a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_empty_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_error_with_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_final_response_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_get_expect_errors_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_denied ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_if_none_match_not_star ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server ERROR: Object returning 503 for [] (txn: tx76a539573abd4ade9b068-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.9:1009/sdj (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.3:1003/sdd (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.13:1013/sdn (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.5:1005/sdf (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: txa0ff57ee2c9b4058a0232-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 Trying to GET /a/c/o From Object Server 10.0.0.6:1006/sdg (txn: tx1ae3636c665643f2aaef0-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx8cbc8940a2b04d92948a2-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Ignoring 412 from primary (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_match ... ok test ERROR: Failed to unlink '/tmp/tmpvoi_7jpb/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpvoi_7jpb/containers/part/suffix/hash/container_1717347405.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpvoi_7jpb/containers/part/suffix/hash/container_1717347405.00000.db': ['/tmp/tmpvoi_7jpb/containers/part/suffix/hash/container.db', '/tmp/tmpvoi_7jpb/containers/part/suffix/hash/container_1717347406.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpvoi_7jpb/containers/part/suffix/hash/container_1717347406.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (10.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: Running object reconstructor in script mode. test INFO: Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/1 pid=3656202] Running object reconstructor in script mode. test INFO: [worker 1/1 pid=3656202] Object reconstruction complete (once). (5.00 minutes) test INFO: [worker 1/2 pid=pid-1] Starting object reconstructor in daemon mode. test INFO: [worker 1/2 pid=pid-1] Starting object reconstruction pass. test INFO: [worker 1/2 pid=pid-1] Object reconstruction complete. (0.10 minutes) test DEBUG: [worker 1/2 pid=pid-1] reconstruction sleeping for 30.0 seconds. test INFO: [worker 2/4 pid=20641] Running object reconstructor in script mode. test DEBUG: [worker 2/4 pid=20641] debug message test INFO: [worker 2/4 pid=20641] info message test WARNING: [worker 2/4 pid=20641] warning message test ERROR: [worker 2/4 pid=20641] error message test INFO: [worker 2/4 pid=20641] Object reconstruction complete (once). (0.00 minutes) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Base DELETE returning 503 for (204, 204, 404) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5db7b7f6e8a345efbe88a-00665ca44e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5db7b7f6e8a345efbe88a-00665ca44e) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5db7b7f6e8a345efbe88a-00665ca44e) test ERROR: Container HEAD returning 503 for [] (txn: tx5db7b7f6e8a345efbe88a-00665ca44e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_change_backend_user_agent ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_no_orig_req ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_generate_request_headers_with_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_infocache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_owner ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_account_info_uses_logging_app ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key test.unit.proxy.controllers.test_base.TestFuncs.test_get_cache_key ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_bad_path ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_in_pipeline ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_auto_account ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_cache ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_no_container_gets_cached ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_only_lookup_cache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:32 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:read_acp:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_with_read_acp_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_GET_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:43 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_bucket_already_owned_by_you ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347352.56203', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy ... ok test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: txdacf524bea784b76b9c7a-00665ca42a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: tx96a3c6aa5a834fb0b2040-00665ca42d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx58379418ad444a039cd0c-00665ca431) test DEBUG: Setting object count to 3120 (txn: tx58379418ad444a039cd0c-00665ca431) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txc37ef01def7e4abbb3c46-00665ca431) test DEBUG: Setting object count to 3120 (txn: txc37ef01def7e4abbb3c46-00665ca431) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test DEBUG: Setting object count to 3120 (txn: tx8fbaa1b8211f42a5a6eb5-00665ca431) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txf9b4c650d7a24682a600a-00665ca431) test DEBUG: Setting object count to 3120 (txn: txf9b4c650d7a24682a600a-00665ca431) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: tx44a5843c1b01482f9dc1c-00665ca431) test DEBUG: Setting object count to 3120 (txn: tx44a5843c1b01482f9dc1c-00665ca431) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9ab98928edea4387a0d34-00665ca432) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx9ab98928edea4387a0d34-00665ca432) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txd843fe1d08594b3088909-00665ca432) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txd843fe1d08594b3088909-00665ca432) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: txa6ea1c043b3d4e81b0df0-00665ca432) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx3188c20ce607445798884-00665ca432) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx3188c20ce607445798884-00665ca432) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx3188c20ce607445798884-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx3188c20ce607445798884-00665ca432) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx3188c20ce607445798884-00665ca432) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txea7c8b2851c14e3993edd-00665ca432) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: txea7c8b2851c14e3993edd-00665ca432) test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainer.test_GET_sharding_container_gap_in_shards_with_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_abort ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_complete ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmprmyxqn10/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmprmyxqn10/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmprmyxqn10/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmprmyxqn10/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:08 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0152 "-" 3656208 - test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_mem_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_mem_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_mem_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_mem_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_mem_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_mem_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_mem_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test INFO: - - - [02/Jun/2024:16:55:55 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0037 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0223 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0219 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0122 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0013 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0073 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:09 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0048 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0052 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0017 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0107 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0271 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0181 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0136 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0017 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0017 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0097 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0147 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0028 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_autocreates_shard_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_json_in_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347355.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347355.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347355.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347355.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_error ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'o_0', 'created_at': '1717347358.10371', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'o_1', 'created_at': '1717347358.10514', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'o_2', 'created_at': '1717347358.10560', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'o_3', 'created_at': '1717347358.10601', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'o_4', 'created_at': '1717347358.10641', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': 'o_5', 'created_at': '1717347358.10690', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': 'o_6', 'created_at': '1717347358.10732', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': 'o_7', 'created_at': '1717347358.10772', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': 'o_8', 'created_at': '1717347358.10812', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': 'o_9', 'created_at': '1717347358.10852', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing_0', 'created_at': '1717347358.10901', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 12, 'name': '/a/c/o_missing_1', 'created_at': '1717347358.17153', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 13, 'name': '/a/c/o_missing_2', 'created_at': '1717347358.20356', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 14, 'name': '/a/c/o_missing_3', 'created_at': '1717347358.23954', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 15, 'name': '/a/c/o_missing_4', 'created_at': '1717347358.36366', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 16, 'name': '/a/c/o_missing_5', 'created_at': '1717347358.40363', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 17, 'name': '/a/c/o_missing_6', 'created_at': '1717347358.44392', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 18, 'name': '/a/c/o_missing_7', 'created_at': '1717347358.47351', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 19, 'name': '/a/c/o_missing_8', 'created_at': '1717347358.53948', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 20, 'name': '/a/c/o_missing_9', 'created_at': '1717347358.59130', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (21, 'c2ce4416c4a9443a8a2bcf46cfd450d0', '916653d7-9733-4700-921e-22bc6e675c1d-sda', '1717347358.08037', '1717347358.07366', '0', '', '1717347358.07366', 21, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 21, 'name': '/a/c/o_missing_10', 'created_at': '1717347358.63965', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '916653d7-9733-4700-921e-22bc6e675c1d-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp0ogfxxkd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a7afc013-bbc8-4223-a96e-63dc06a8146f-sda', '1717347366.01560', '1717347366', '0', '', '1717347366', 0, 0) test DEBUG: /tmp/tmp0ogfxxkd/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a7afc013-bbc8-4223-a96e-63dc06a8146f-sda', '1717347366.01560', '1717347366', '0', '', '1717347366', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a7afc013-bbc8-4223-a96e-63dc06a8146f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.06978 seconds (14.33081/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5rludwoz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '23990561e0e339033e9d939d2a99b6b6', '06c6db0d-54c1-4943-ac5c-a3ce3683c229-sda', '1717347366.12386', '1717347367', '0', '', '1717347367', 1, 1) test DEBUG: /tmp/tmp5rludwoz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347368.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '06c6db0d-54c1-4943-ac5c-a3ce3683c229-sda') test DEBUG: /tmp/tmp5rludwoz/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0176678s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '23990561e0e339033e9d939d2a99b6b6', '06c6db0d-54c1-4943-ac5c-a3ce3683c229-sda', '1717347366.12386', '1717347367', '0', '', '1717347367', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('06c6db0d-54c1-4943-ac5c-a3ce3683c229-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp5rludwoz/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14290 seconds (6.99804/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_create ... ok test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx953af4161f4d450d9f213-00665ca443) test ERROR: Aborting listing from shards due to bad shard policy index: 0 (expected 1) (txn: tx953af4161f4d450d9f213-00665ca443) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx8504e09f780346bea20d6-00665ca443) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx8504e09f780346bea20d6-00665ca443) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx8504e09f780346bea20d6-00665ca443) test ERROR: Aborting listing from shards due to bad shard policy index: None (expected 1) (txn: tx8504e09f780346bea20d6-00665ca443) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: Setting object count to 3120 (txn: txdfec1db525394693949a2-00665ca446) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: Setting object count to 3120 (txn: tx1fa976b17b1d49f9a23d9-00665ca446) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: Setting object count to 3120 (txn: tx19d0b9994de6473b94d8b-00665ca446) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: Setting object count to 3120 (txn: txdee968f6bd11447085309-00665ca446) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: Setting object count to 3120 (txn: txe895f0e7d9cd4123abaef-00665ca446) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx649256a8871348d4ae9dd-00665ca446) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txc5f2f5181eba46c4b3534-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc5f2f5181eba46c4b3534-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txc5f2f5181eba46c4b3534-00665ca446) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txc5f2f5181eba46c4b3534-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txc5f2f5181eba46c4b3534-00665ca446) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_overlapping_shards_no_memcache ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app test.unit.proxy.controllers.test_base.TestFuncs.test_get_container_info_uses_logging_app ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info test.unit.proxy.controllers.test_base.TestFuncs.test_get_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck test.unit.proxy.controllers.test_base.TestFuncs.test_get_info_zero_recheck ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_disabled ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_hit ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_infocache_hit ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_miss ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips test.unit.proxy.controllers.test_base.TestFuncs.test_get_namespaces_from_cache_skips ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_no_env ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source test.unit.proxy.controllers.test_base.TestFuncs.test_get_object_info_swift_source ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info test.unit.proxy.controllers.test_base.TestFuncs.test_headers_from_container_info ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_storage_policies ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_account_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_missing ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: tx34b22cbcb7054135989a1-00665ca429) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txda27a3e8695d408b9198f-00665ca429) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txda27a3e8695d408b9198f-00665ca429) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txda27a3e8695d408b9198f-00665ca429) proxy-server ERROR: Object PUT returning 503 for [503, 503, 503] (txn: txda27a3e8695d408b9198f-00665ca429) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1020, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1662, in read raise Exception('exception message') Exception: exception message (txn: tx045547dc00da4d5e93b64-00665ca429) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1752, in await_response self.resp = self.conn.getresponse() ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 848, in getresponse self.status = self._status.get_response_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 748, in get_response_status raise self.status Exception: boom (txn: tx36fa59e6929b44d797b96-00665ca42a) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to get final status of PUT to /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89: Timeout (Nones) (txn: tx85b5f6fcb3f446b085352-00665ca42a) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx228d9971856b474b9ce7f-00665ca42a) proxy-server DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txc1b6f0edade34b1fa0a56-00665ca42a) proxy-server ERROR: ERROR 500 b'' Trying to PUT /v1/AUTH_kilroy/%ED%88%8E/%E9%90%89 From Object Server 10.0.0.2:1002/sdc (txn: txc1b6f0edade34b1fa0a56-00665ca42a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: txcbcfdfab9a2943e492aaa-00665ca42a) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx89b804b7a3d04f16aa28c-00665ca42a) proxy-server ERROR: ERROR 503 Expect: 100-continue From Object Server 10.0.0.0:1000/sda (txn: tx89b804b7a3d04f16aa28c-00665ca42a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [412] (txn: tx0d71904969dd4191b0a00-00665ca42a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_insufficient_data_from_client ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_send_exception_with_unicode_path ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_timeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_chunked_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_PUT_with_no_body_and_no_footers ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_newer ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:08 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0216 "-" 3656208 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:19 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.4554 "-" 3656208 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:20 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.8400 "-" 3656208 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:20 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa57031bf6c90470883a78-00665ca434" "proxy-server 3656208" 0.0030 "-" 3656208 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:59387/sdb1/1/a/c" "txa57031bf6c90470883a78-00665ca434" "container-server 3656208" 0.0037 "-" 3656208 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:46367/sda1/1/a/c" "txa57031bf6c90470883a78-00665ca434" "container-server 3656208" 0.0062 "-" 3656208 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa57031bf6c90470883a78-00665ca434" "proxy-server 3656208" 0.3384 "-" 3656208 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txa57031bf6c90470883a78-00665ca434" "proxy-server 3656208" 1.0316 "-" 3656208 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a/c HTTP/1.0 201 - - t - - - txa57031bf6c90470883a78-00665ca434 - 1.0690 - - 1717347380.588944674 1717347381.657947540 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx7d0d2c2de90c4542ac32b-00665ca435" "proxy-server 3656208" 0.0029 "-" 3656208 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:46367/sda1/1/a1/c1" "tx7d0d2c2de90c4542ac32b-00665ca435" "container-server 3656208" 0.0048 "-" 3656208 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:59387/sdb1/1/a1/c1" "tx7d0d2c2de90c4542ac32b-00665ca435" "container-server 3656208" 0.0078 "-" 3656208 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx7d0d2c2de90c4542ac32b-00665ca435" "proxy-server 3656208" 0.0424 "-" 3656208 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx7d0d2c2de90c4542ac32b-00665ca435" "proxy-server 3656208" 0.0626 "-" 3656208 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx7d0d2c2de90c4542ac32b-00665ca435 - 0.0799 - - 1717347381.661465645 1717347381.741335392 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx00b6d503d71d4203b7b80-00665ca435" "proxy-server 3656208" 0.0154 "-" 3656208 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59387/sdb1/2/a/c1" "tx00b6d503d71d4203b7b80-00665ca435" "container-server 3656208" 0.0028 "-" 3656208 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:46367/sda1/2/a/c1" "tx00b6d503d71d4203b7b80-00665ca435" "container-server 3656208" 0.0166 "-" 3656208 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx00b6d503d71d4203b7b80-00665ca435" "proxy-server 3656208" 0.0651 "-" 3656208 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx00b6d503d71d4203b7b80-00665ca435" "proxy-server 3656208" 0.0458 "-" 3656208 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx00b6d503d71d4203b7b80-00665ca435 - 0.0974 - - 1717347381.744962215 1717347381.842407703 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1034f632bed2492abc67b-00665ca435" "proxy-server 3656208" 0.0169 "-" 3656208 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:59387/sdb1/1/a/c2" "tx1034f632bed2492abc67b-00665ca435" "container-server 3656208" 0.0025 "-" 3656208 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:46367/sda1/1/a/c2" "tx1034f632bed2492abc67b-00665ca435" "container-server 3656208" 0.0197 "-" 3656208 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx1034f632bed2492abc67b-00665ca435" "proxy-server 3656208" 0.0849 "-" 3656208 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx1034f632bed2492abc67b-00665ca435" "proxy-server 3656208" 0.0600 "-" 3656208 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/21 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx1034f632bed2492abc67b-00665ca435 - 0.1186 - - 1717347381.845732689 1717347381.964297771 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_owner_permission ... ok test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347367.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347367.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source?versionId=null', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347367.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347367.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_empty_src_path ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_multipart_uploads_initiate ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_bucket_upload_part_missing_key ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_with_read_permission ... ok test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txc5f2f5181eba46c4b3534-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txc5f2f5181eba46c4b3534-00665ca446) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: txc25ddd14e67f4cc4a278f-00665ca446) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx7711f555bfc44e1ab9936-00665ca446) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7711f555bfc44e1ab9936-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7711f555bfc44e1ab9936-00665ca446) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx7711f555bfc44e1ab9936-00665ca446) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx98158a7fb98948539c38b-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx98158a7fb98948539c38b-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx98158a7fb98948539c38b-00665ca446) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx98158a7fb98948539c38b-00665ca446) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: txc543d37c1f5e4fa693913-00665ca446) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txc543d37c1f5e4fa693913-00665ca446) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: txc543d37c1f5e4fa693913-00665ca446) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txc543d37c1f5e4fa693913-00665ca446) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Found 1020 objects in shard (state=unsharded), total = 1068 (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: Setting object count to 1068 (txn: tx7498cdaba1cb40c0a1258-00665ca449) test DEBUG: GET listing from 3 shards for: /v1/a/c?reverse=true&format=json&states=listing (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='lemon', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&format=json&limit=10000&marker=&end_marker=lemon {'X-Backend-Record-Type': 'auto'} (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&format=json&limit=8976&marker=m&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Found 44 objects in shard (state=unsharded), total = 1068 (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&format=json&limit=8932&marker=A&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1068 (txn: tx6ae64275dc934d5d80f1e-00665ca449) test DEBUG: Setting object count to 1068 (txn: tx6ae64275dc934d5d80f1e-00665ca449) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_errors_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_between_shards ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:55:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347363.98260', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test DEBUG: Replicating db /tmp/tmp5rludwoz/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '612c72c15e6582f12e2133f110d8a23f', '83718a37-0869-4526-8fe6-db6e96cf62ab-sda', '1717347366.24912', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('83718a37-0869-4526-8fe6-db6e96cf62ab-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '612c72c15e6582f12e2133f110d8a23f', '83718a37-0869-4526-8fe6-db6e96cf62ab-sda', '1717347366.24912', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('83718a37-0869-4526-8fe6-db6e96cf62ab-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpdgyc99y6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b7eaefb5-d385-4eeb-9e14-169b4605a553-sda', '1717347372.80479', '1717347373', '0', '', '1717347373', 0, 0) test DEBUG: /tmp/tmpdgyc99y6/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b7eaefb5-d385-4eeb-9e14-169b4605a553-sda', '1717347372.80479', '1717347373', '0', '', '1717347373', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b7eaefb5-d385-4eeb-9e14-169b4605a553-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09052 seconds (11.04746/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6awk6ong/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a4ce1235b73f8159921d0a0cd55d5195', '322ae24a-dbed-40de-8889-21860fe376a3-sda', '1717347372.93831', '1717347373', '0', '', '1717347373', 1, 0) test DEBUG: /tmp/tmp6awk6ong/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'a4ce1235b73f8159921d0a0cd55d5195', '322ae24a-dbed-40de-8889-21860fe376a3-sda', '1717347372.93831', '1717347373', '0', '', '1717347373', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('322ae24a-dbed-40de-8889-21860fe376a3-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp6awk6ong/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.14351 seconds (6.96798/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp6awk6ong/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '53d5a71d26eaadecbb12520902453230', '5ce2aa72-6798-447a-8de6-f599d1f0e407-sda', '1717347373.06398', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('5ce2aa72-6798-447a-8de6-f599d1f0e407-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '53d5a71d26eaadecbb12520902453230', '5ce2aa72-6798-447a-8de6-f599d1f0e407-sda', '1717347373.06398', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('5ce2aa72-6798-447a-8de6-f599d1f0e407-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1h1qiwjb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '800e6382-e4a9-4d68-94be-9a4fbe31104e-sda', '1717347378.28811', '1717347378', '1717347379', '', '1717347379', 0, 1) test DEBUG: /tmp/tmp1h1qiwjb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '800e6382-e4a9-4d68-94be-9a4fbe31104e-sda', '1717347378.28811', '1717347378', '1717347379', '', '1717347379', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('800e6382-e4a9-4d68-94be-9a4fbe31104e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 1.92519 seconds (0.51943/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp3pw2qriq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '531e15b7b38c88827005fd81837d4c23', '1ba4cac0-964a-42bf-8e27-0e80e2e5302d-sda', '1717347381.71712', '1717347382', '1717347383', '', '1717347383', 1, 0) test DEBUG: /tmp/tmp3pw2qriq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347384.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '1ba4cac0-964a-42bf-8e27-0e80e2e5302d-sda') test DEBUG: /tmp/tmp3pw2qriq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0149221s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '531e15b7b38c88827005fd81837d4c23', '1ba4cac0-964a-42bf-8e27-0e80e2e5302d-sda', '1717347381.71712', '1717347382', '1717347383', '', '1717347383', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('1ba4cac0-964a-42bf-8e27-0e80e2e5302d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp3pw2qriq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.22687 seconds (4.40786/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp3pw2qriq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8274c939ad4970bfe2120e164605a813', 'bf8214ac-a478-4efe-8444-7ae286d71b67-sda', '1717347381.92144', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('bf8214ac-a478-4efe-8444-7ae286d71b67-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_newer_local_delete ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_create_policy_over_older_local_delete ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txea7c8b2851c14e3993edd-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: txea7c8b2851c14e3993edd-00665ca432) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txea7c8b2851c14e3993edd-00665ca432) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txb4b38118a0c642b5bb92a-00665ca432) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: txb4b38118a0c642b5bb92a-00665ca432) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txb4b38118a0c642b5bb92a-00665ca432) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: txb4b38118a0c642b5bb92a-00665ca432) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txb4b38118a0c642b5bb92a-00665ca432) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx39199eb5b57f4ef5beae5-00665ca435) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx39199eb5b57f4ef5beae5-00665ca435) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: tx39199eb5b57f4ef5beae5-00665ca435) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx302ff9fd295d45db89fd5-00665ca437) test DEBUG: Setting object count to 1065 (txn: tx302ff9fd295d45db89fd5-00665ca437) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpteapcji7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpteapcji7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpteapcji7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpteapcji7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:23 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0188 "-" 3656256 - HTTP PERF: 0.02337 seconds to PUT 127.0.0.1:40251 /sda1/0/a) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:23 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0124 "-" 3656256 - HTTP PERF: 0.01470 seconds to PUT 127.0.0.1:60569 /sdb1/0/a) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0233 "-" 3656256 - HTTP PERF: 0.02610 seconds to PUT 127.0.0.1:40251 /sda1/2/a1) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0157 "-" 3656256 - HTTP PERF: 0.01807 seconds to PUT 127.0.0.1:60569 /sdb1/2/a1) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdfdc6a753530424eb863d-00665ca43a" "proxy-server 3656256" 0.0023 "-" 3656256 - HTTP PERF: 0.00436 seconds to HEAD 127.0.0.1:40251 /sda1/0/a?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:43681/sdb1/3/a/c" "txdfdc6a753530424eb863d-00665ca43a" "container-server 3656256" 0.0017 "-" 3656256 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:60075/sda1/3/a/c" "txdfdc6a753530424eb863d-00665ca43a" "container-server 3656256" 0.0030 "-" 3656256 0 HTTP PERF: 0.00841 seconds to PUT 127.0.0.1:60569 /sdb1/0/a/c) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txdfdc6a753530424eb863d-00665ca43a" "proxy-server 3656256" 0.0245 "-" 3656256 0 HTTP PERF: 0.02636 seconds to PUT 127.0.0.1:40251 /sda1/0/a/c) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "PUT /sdb1/3/a/c" 201 - "PUT http://localhost/v1/a/c" "txdfdc6a753530424eb863d-00665ca43a" "proxy-server 3656256" 0.0434 "-" 3656256 0 HTTP PERF: 0.04561 seconds to PUT 127.0.0.1:60075 /sda1/3/a/c) HTTP PERF: 0.04700 seconds to PUT 127.0.0.1:43681 /sdb1/3/a/c) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/26 PUT /v1/a/c HTTP/1.0 201 - - t - - - txdfdc6a753530424eb863d-00665ca43a - 0.0561 - - 1717347386.898092508 1717347386.954211235 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx5eca45cf922741308f482-00665ca43a" "proxy-server 3656256" 0.0017 "-" 3656256 - HTTP PERF: 0.00384 seconds to HEAD 127.0.0.1:40251 /sda1/2/a1?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:43681/sdb1/0/a1/c1" "tx5eca45cf922741308f482-00665ca43a" "container-server 3656256" 0.0019 "-" 3656256 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:60075/sda1/0/a1/c1" "tx5eca45cf922741308f482-00665ca43a" "container-server 3656256" 0.0046 "-" 3656256 0 HTTP PERF: 0.03105 seconds to PUT 127.0.0.1:40251 /sda1/2/a1/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx5eca45cf922741308f482-00665ca43a" "proxy-server 3656256" 0.0460 "-" 3656256 0 HTTP PERF: 0.01159 seconds to PUT 127.0.0.1:60569 /sdb1/2/a1/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx5eca45cf922741308f482-00665ca43a" "proxy-server 3656256" 0.0317 "-" 3656256 0 HTTP PERF: 0.04930 seconds to PUT 127.0.0.1:60075 /sda1/0/a1/c1) HTTP PERF: 0.05014 seconds to PUT 127.0.0.1:43681 /sdb1/0/a1/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/27 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx5eca45cf922741308f482-00665ca43a - 0.0583 - - 1717347386.956326962 1717347387.014590263 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1ffe70b29814444b99705-00665ca43b" "proxy-server 3656256" 0.0115 "-" 3656256 - HTTP PERF: 0.01397 seconds to HEAD 127.0.0.1:40251 /sda1/0/a?format=json) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:43681/sdb1/3/a/c1" "tx1ffe70b29814444b99705-00665ca43b" "container-server 3656256" 0.0143 "-" 3656256 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:60075/sda1/3/a/c1" "tx1ffe70b29814444b99705-00665ca43b" "container-server 3656256" 0.0016 "-" 3656256 1 HTTP PERF: 0.02186 seconds to PUT 127.0.0.1:40251 /sda1/0/a/c1) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sda1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1ffe70b29814444b99705-00665ca43b" "proxy-server 3656256" 0.0377 "-" 3656256 1 HTTP PERF: 0.03943 seconds to PUT 127.0.0.1:60569 /sdb1/0/a/c1) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sdb1/3/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx1ffe70b29814444b99705-00665ca43b" "proxy-server 3656256" 0.0546 "-" 3656256 1 HTTP PERF: 0.05727 seconds to PUT 127.0.0.1:60075 /sda1/3/a/c1) HTTP PERF: 0.05715 seconds to PUT 127.0.0.1:43681 /sdb1/3/a/c1) proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/27 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx1ffe70b29814444b99705-00665ca43b - 0.0763 - - 1717347387.018818855 1717347387.095138788 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe82b313e062c42ed804ba-00665ca43b" "proxy-server 3656256" 0.0096 "-" 3656256 - HTTP PERF: 0.01172 seconds to HEAD 127.0.0.1:40251 /sda1/0/a?format=json) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:43681/sdb1/0/a/c2" "txe82b313e062c42ed804ba-00665ca43b" "container-server 3656256" 0.0016 "-" 3656256 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:60075/sda1/0/a/c2" "txe82b313e062c42ed804ba-00665ca43b" "container-server 3656256" 0.0089 "-" 3656256 2 HTTP PERF: 0.01410 seconds to PUT 127.0.0.1:40251 /sda1/0/a/c2) cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txe82b313e062c42ed804ba-00665ca43b" "proxy-server 3656256" 0.0293 "-" 3656256 2 HTTP PERF: 0.03110 seconds to PUT 127.0.0.1:60569 /sdb1/0/a/c2) cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:27 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txe82b313e062c42ed804ba-00665ca43b" "proxy-server 3656256" 0.0460 "-" 3656256 2 HTTP PERF: 0.04827 seconds to PUT 127.0.0.1:60075 /sda1/0/a/c2) HTTP PERF: 0.04914 seconds to PUT 127.0.0.1:43681 /sdb1/0/a/c2) test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_ec_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_chunked_transfer_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up test.unit.proxy.test_server.TestObjectDisconnectCleanup.test_repl_disconnect_cleans_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.test_locale.test_locale.TestTranslations.test_translations test.unit.test_locale.test_locale.TestTranslations.test_translations ... ok proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/27 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txe82b313e062c42ed804ba-00665ca43b - 0.0650 - - 1717347387.097091436 1717347387.162141085 2 test INFO: - - - [02/Jun/2024:16:56:10 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0092 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:15 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:15 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:17 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.6165 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:17 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:18 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:19 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.8843 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:19 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:20 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:20 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0054 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:20 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0015 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmpe7rg_7qf/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.4386 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0019 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0017 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0160 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0037 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0119 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0056 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0012 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0114 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0031 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0083 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0129 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0028 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0138 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:21 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0023 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0160 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0129 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0056 "-" 3656251 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0354 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0053 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0117 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0010 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0005 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0449 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0116 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0117 "-" 3656251 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0009 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0190 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0140 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0211 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0127 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0005 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0004 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0004 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0140 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0141 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0190 "-" 3656251 2 test DEBUG: Unlinked retiring db '/tmp/tmpqf7x5zc0/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:56:22 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0195 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:22 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0028 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_PUT_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test__call__raise_timeout ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_account_update_mismatched_host_device ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_acl_container ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_container_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_missing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_sys_meta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_transient_sysmeta ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values test.unit.proxy.controllers.test_base.TestFuncs.test_headers_to_object_info_values ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing test.unit.proxy.controllers.test_base.TestFuncs.test_info_clearing ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options test.unit.proxy.controllers.test_base.TestFuncs.test_options ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized test.unit.proxy.controllers.test_base.TestFuncs.test_options_unauthorized ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin test.unit.proxy.controllers.test_base.TestFuncs.test_options_with_null_allow_origin ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics test.unit.proxy.controllers.test_base.TestFuncs.test_record_cache_op_metrics ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_disabled ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending enough data (txn: txd4c8a5657e3c46aa8122b-00665ca432) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1795, in _send_chunk self.conn.send(to_send) File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 926, in send self.give_send(self, data) File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 1426, in capture_send raise exc Exception: Exception while sending data on connection (txn: tx27704306227a4fbca2dd1-00665ca435) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to write to /AUTH_kilroy/%ED%88%8E/%E9%90%89: ChunkWriteTimeout (Nones) (txn: tx2240392525bf48d098bcc-00665ca435) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: tx8af39b4eab6c413fb37dc-00665ca435) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347381.74484 <= '409 (1717347381.74484), 409 (1717347381.74484), 409 (1717347381.74484)' (txn: tx338a8552a3b64378a6c3e-00665ca435) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347381.74976 <= '409 (1717347381.74976), 409 (1717347381.74976), 409 (1717347381.74976)' (txn: txe498beab031449659e1d2-00665ca435) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347381.75456 <= '409 (1717347381.75456), 409 (1717347381.75456), 409 (1717347381.75456)' (txn: tx9da1c77c994d4874ac6ab-00665ca435) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_older ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_sync_put_x_timestamp_unsynced_race ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_primary_returns_some_nonsense_timestamp ... ok test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:full_control FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:09 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:tester FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Grant test:write_acp FULL_CONTROL permission on the object /bucket/object test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_with_write_acp_permission ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/bucket.py", line 394, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1630, in get_response resp = self.acl_handler.handle_acl( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 94, in handle_acl return getattr(ah, method)(app) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/acl_handlers.py", line 220, in PUT self.req.get_acl_response(app, 'PUT', self.container) File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_error_non_swift_owner ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_first_put_fail ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347373.73670', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_headers_with_not_match_and_s3acl ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_multipart_uploads_acl_without_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx9539fd5997964c6cb4772-00665ca44c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9539fd5997964c6cb4772-00665ca44c) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9539fd5997964c6cb4772-00665ca44c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9539fd5997964c6cb4772-00665ca44c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9539fd5997964c6cb4772-00665ca44c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9539fd5997964c6cb4772-00665ca44c) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx9539fd5997964c6cb4772-00665ca44c) test ERROR: Aborting listing from shards due to bad response: [200, 404] (txn: tx9539fd5997964c6cb4772-00665ca44c) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txf990ab714f1b44a383091-00665ca44e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txf990ab714f1b44a383091-00665ca44e) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txf990ab714f1b44a383091-00665ca44e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txf990ab714f1b44a383091-00665ca44e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txf990ab714f1b44a383091-00665ca44e) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.2:1002/sdc (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.1:1001/sdb (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.0:1000/sda (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.3:1003/sda (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.4:1004/sda (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: ERROR 500 b'[]' Trying to GET /.shards_a/c_pie From Container Server 10.0.0.5:1005/sda (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: txf990ab714f1b44a383091-00665ca44e) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 503 (txn: txf990ab714f1b44a383091-00665ca44e) test ERROR: Aborting listing from shards due to bad response: [200, 503] (txn: txf990ab714f1b44a383091-00665ca44e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_a', lower='', upper='a') .shards_a/c_a with {'X-Backend-Record-Type': 'auto'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Get listing from /v1/.shards_a/c_a?format=json&limit=10000&marker=&end_marker=a%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Found 33 objects in shard (state=unsharded), total = 33 (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=9967&marker=a&end_marker=b%00&states=listing (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'object'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=9967&marker=a&end_marker=b%00 {'X-Backend-Record-Type': 'object'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Found 1 objects in shard (state=sharded), total = 1 (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Found 1 objects in shard (state=sharded), total = 34 (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='b', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'object'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9966&marker=b&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1058 (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Setting object count to 1058 (txn: tx2e521f0084944b5db8093-00665ca451) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 1 shards for: /v1/a/c?format=json&states=listing (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: GET listing from 1 shards for: /v1/.shards_a/c_?format=json&limit=10000&marker=&end_marker=&states=listing (txn: tx86d1023c29104578958c0-00665ca454) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_shard_redirects_to_root_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharded_shard_with_memcache ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_with_write_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx835f12cf2645424aa864d-00665ca44e) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx835f12cf2645424aa864d-00665ca44e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx835f12cf2645424aa864d-00665ca44e) test ERROR: Container HEAD returning 503 for [] (txn: tx835f12cf2645424aa864d-00665ca44e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb0030937028a40bd91911-00665ca44e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb0030937028a40bd91911-00665ca44e) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb0030937028a40bd91911-00665ca44e) test ERROR: Container HEAD returning 503 for [] (txn: txb0030937028a40bd91911-00665ca44e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_error ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_infocache_exists ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok test.unit.proxy.controllers.test_base.TestFuncs.test_set_namespaces_in_cache_ok ... ok test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta test.unit.proxy.controllers.test_base.TestFuncs.test_transfer_headers_with_sysmeta ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_exception ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_commit_timeout ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347379.41031', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_copy ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_chunkreadtimeout_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_error_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_exception_during_transfer_data ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_ec_timeout_during_transfer_data ... ok test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0179 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0083 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:22 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0042 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0239 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0025 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0163 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0161 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:56:26 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:56:26 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:56:26 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0005 "-" 3656251 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:56:26 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0055 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:56:26 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0005 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0004 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0034 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0172 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0039 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0007 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0306 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0098 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0158 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0208 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0235 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0202 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0244 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0020 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0189 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0143 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0172 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0191 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0121 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0187 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:26 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0110 "-" 3656251 - test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_correct_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_creation ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delete_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_deleted_headers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_delimiter_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_list_allowed_methods ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:15 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE_ACP test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:29 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_without_permission ... ok test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error test.unit.common.middleware.s3api.test_s3_acl.TestS3ApiS3Acl.test_object_acl_PUT_xml_error ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_isoformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_from_s3xmlformat ... ok test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat test.unit.common.middleware.s3api.test_utils.TestS3Timestamp.test_s3xmlformat ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults test.unit.common.middleware.test_cname_lookup.TestSwiftInfo.test_registered_nondefaults ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme test.unit.common.middleware.test_formpost.TestFormPost.test_auth_scheme ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_chrome test.unit.common.middleware.test_formpost.TestFormPost.test_chrome ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires test.unit.common.middleware.test_formpost.TestFormPost.test_completely_non_int_expires ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode test.unit.common.middleware.test_formpost.TestFormPost.test_curl_with_unicode ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account test.unit.common.middleware.test_formpost.TestFormPost.test_empty_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_empty_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_explorer test.unit.common.middleware.test_formpost.TestFormPost.test_explorer ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_firefox test.unit.common.middleware.test_formpost.TestFormPost.test_firefox ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_container_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_multiple_keys ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_with_origin ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent test.unit.common.middleware.test_formpost.TestFormPost.test_formpost_without_useragent ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_global_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_count_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded test.unit.common.middleware.test_formpost.TestFormPost.test_max_file_size_exceeded ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start test.unit.common.middleware.test_formpost.TestFormPost.test_messed_up_start ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_multiple_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_container test.unit.common.middleware.test_formpost.TestFormPost.test_no_container ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process test.unit.common.middleware.test_formpost.TestFormPost.test_no_file_to_process ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_expired ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_invalid_sig ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error test.unit.common.middleware.test_formpost.TestFormPost.test_no_redirect_with_error ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 test.unit.common.middleware.test_formpost.TestFormPost.test_no_v1 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough test.unit.common.middleware.test_formpost.TestFormPost.test_passthrough ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_good ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported test.unit.common.middleware.test_formpost.TestFormPost.test_prefixed_and_not_prefixed_sigs_unsupported ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect test.unit.common.middleware.test_formpost.TestFormPost.test_redirect ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_allowed_deprecated_and_unsupported_digests ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query test.unit.common.middleware.test_formpost.TestFormPost.test_redirect_with_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_safari test.unit.common.middleware.test_formpost.TestFormPost.test_safari ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding test.unit.common.middleware.test_formpost.TestFormPost.test_single_content_type_encoding ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_does_not_pass_query ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_no_redirect_503 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 test.unit.common.middleware.test_formpost.TestFormPost.test_subrequest_fails_redirect_404 ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value test.unit.common.middleware.test_formpost.TestFormPost.test_truncated_attr_value ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account test.unit.common.middleware.test_formpost.TestFormPost.test_wrong_account ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_after_not_int ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at ... ok test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int test.unit.common.middleware.test_formpost.TestFormPost.test_x_delete_at_not_int ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_defaults ... ok test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple test.unit.common.middleware.test_keystoneauth.ResellerInInfo.test_multiple ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_big_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_fails ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_finds_old ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_small_non_slo ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_beyond_manifest_refetch_tombstone ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_bogus_content_range ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_manifest ... ok test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest test.unit.common.middleware.test_slo.TestOldSwiftWithRangesOldManifests.test_old_swift_range_get_includes_whole_range_manifest ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_first_version ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_new_version_different_account ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true test.unit.common.middleware.test_versioned_writes.VersionedWritesCopyingTestCase.test_copy_object_no_versioning_with_container_config_true ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear test.unit.common.middleware.test_xprofile.Test_profile_log.test_clear ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile test.unit.common.middleware.test_xprofile.Test_profile_log.test_dump_profile ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_all_pids ... ok test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles test.unit.common.middleware.test_xprofile.Test_profile_log.test_get_logfiles ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_component_different_builder_order ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_different_number_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_duplicate_builder_ids ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_insufficient_builders ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_missing_builder_id ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_older_builder ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_replica_count_changed ... ok test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders test.unit.common.ring.test_composite_builder.TestComposeLoadComponents.test_load_components_unchanged_builders ... ok test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts test.unit.common.ring.test_utils.TestUtils.test_build_dev_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree test.unit.common.ring.test_utils.TestUtils.test_build_tier_tree ... ok test.unit.common.ring.test_utils.TestUtils.test_dispersion_report test.unit.common.ring.test_utils.TestUtils.test_dispersion_report ... ok test.unit.common.ring.test_utils.TestUtils.test_is_local_device test.unit.common.ring.test_utils.TestUtils.test_is_local_device ... ok test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname test.unit.common.ring.test_utils.TestUtils.test_is_valid_hostname ... ok test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names test.unit.common.ring.test_utils.TestUtils.test_normalized_device_tier_names ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format test.unit.common.ring.test_utils.TestUtils.test_parse_address_old_format ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_args test.unit.common.ring.test_utils.TestUtils.test_parse_args ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args test.unit.common.ring.test_utils.TestUtils.test_parse_builder_ring_filename_args ... ok REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8274c939ad4970bfe2120e164605a813', 'bf8214ac-a478-4efe-8444-7ae286d71b67-sda', '1717347381.92144', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('bf8214ac-a478-4efe-8444-7ae286d71b67-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpc_wn91gx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8e17300f-c49e-4a4f-b0f6-ced8c828521e-sda', '1717347382.09208', '1717347383', '1717347384', '', '1717347384', 0, 0) test DEBUG: /tmp/tmpc_wn91gx/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '8e17300f-c49e-4a4f-b0f6-ced8c828521e-sda', '1717347382.09208', '1717347383', '1717347384', '', '1717347384', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('8e17300f-c49e-4a4f-b0f6-ced8c828521e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07066 seconds (14.15133/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpx394c4sl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '22a3c592d3032d2810629c30ec66e418', 'eacd8e57-5ea0-4117-bace-4be3c23775e9-sda', '1717347382.21654', '1717347383', '1717347384', '', '1717347384', 1, 1) test DEBUG: /tmp/tmpx394c4sl/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '22a3c592d3032d2810629c30ec66e418', 'eacd8e57-5ea0-4117-bace-4be3c23775e9-sda', '1717347382.21654', '1717347383', '1717347384', '', '1717347384', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('eacd8e57-5ea0-4117-bace-4be3c23775e9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpx394c4sl/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.15874 seconds (6.29974/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpx394c4sl/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e5d06aaeb41e5df2d79ea5ea98c3353d', '68054f41-9121-4662-b40d-9841ba7779cb-sda', '1717347382.36817', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('68054f41-9121-4662-b40d-9841ba7779cb-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'e5d06aaeb41e5df2d79ea5ea98c3353d', '68054f41-9121-4662-b40d-9841ba7779cb-sda', '1717347382.36817', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('68054f41-9121-4662-b40d-9841ba7779cb-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6npbwawj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aaa6d83c-3162-47c3-a2b1-689b6cdca2f9-sda', '1717347385.37211', '1717347385', '1717347386', '', '1717347386', 0, 0) test DEBUG: /tmp/tmp6npbwawj/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aaa6d83c-3162-47c3-a2b1-689b6cdca2f9-sda', '1717347385.37211', '1717347385', '1717347386', '', '1717347386', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('aaa6d83c-3162-47c3-a2b1-689b6cdca2f9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08921 seconds (11.20913/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp963s0t1o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '05884e93-3d48-4b81-b8d3-5aa16da456d1-sda', '1717347385.51988', '1717347385', '1717347386', '', '1717347386', 0, 1) test DEBUG: /tmp/tmp963s0t1o/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '05884e93-3d48-4b81-b8d3-5aa16da456d1-sda', '1717347385.51988', '1717347385', '1717347386', '', '1717347386', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('05884e93-3d48-4b81-b8d3-5aa16da456d1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10322 seconds (9.68784/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1nm5f9dc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '730b94b3-1f17-40b7-bfdb-aa20d6965936-sda', '1717347385.72883', '1717347387', '0', '', '1717347387', 0, 1) test DEBUG: /tmp/tmp1nm5f9dc/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '730b94b3-1f17-40b7-bfdb-aa20d6965936-sda', '1717347385.72883', '1717347387', '0', '', '1717347387', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('730b94b3-1f17-40b7-bfdb-aa20d6965936-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08740 seconds (11.44176/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpv9pyz_8k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2da8fe73cf5cabfbee6d71150843c60d', 'aa07db42-3f17-4530-a2d1-aa59da7b0471-sda', '1717347385.89709', '1717347387', '0', '', '1717347387', 1, 0) test DEBUG: /tmp/tmpv9pyz_8k/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_half_delete_policy_over_newer_local_create ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_in_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_most_rows ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_missing_one_rows ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_change_values_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_value test.unit.common.ring.test_utils.TestUtils.test_parse_search_value ... ok test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts test.unit.common.ring.test_utils.TestUtils.test_parse_search_values_from_opts ... ok test.unit.common.ring.test_utils.TestUtils.test_replication_defaults test.unit.common.ring.test_utils.TestUtils.test_replication_defaults ... ok test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev test.unit.common.ring.test_utils.TestUtils.test_tiers_for_dev ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_address ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip test.unit.common.ring.test_utils.TestUtils.test_validate_and_normalize_ip ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args test.unit.common.ring.test_utils.TestUtils.test_validate_args ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format test.unit.common.ring.test_utils.TestUtils.test_validate_args_new_cmd_format ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_close ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_errors ... ok test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact test.unit.common.ring.test_utils.TestUtils.test_validate_replicas_by_tier_exact ... ok test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout test.unit.common.test_memcached.TestMemcached.test_connection_pool_timeout ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347392.00000 <= '409 (1717347393.00000), 409 (1717347393.00000), 409 (1717347393.00000)' (txn: txecee9897334a4096a3970-00665ca440) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347394.00000 <= '409 (1717347395.00000), 409 (1717347395.00000), 409 (1717347395.00000)' (txn: tx0b891c72a3f04a4a91493-00665ca440) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347396.00000 <= '409 (1717347397.00000), 409 (1717347397.00000), 409 (1717347397.00000)' (txn: txcb06f6e92ef34d1091efb-00665ca440) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347392.00000 <= '409 (1717347392.00000), 409 (1717347392.00000), 409 (1717347392.00000)' (txn: txdb6672d8585e41288c535-00665ca440) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347393.00000 <= '409 (1717347393.00000), 409 (1717347393.00000), 409 (1717347393.00000)' (txn: tx71de5f6d8a754e8c9d2c1-00665ca440) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347394.00000 <= '409 (1717347394.00000), 409 (1717347394.00000), 409 (1717347394.00000)' (txn: tx48fd543ab95e4209a38fb-00665ca440) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347392.00000 <= '409 (1717347392.00000)' (txn: tx728fc7281699456393440-00665ca440) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347393.00000 <= '409 (1717347393.00000)' (txn: txee7e3716872b48198229c-00665ca440) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347394.00000 <= '409 (1717347394.00000)' (txn: tx21add12f1f364d58bae02-00665ca440) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Client disconnected on read of '/a/c/o' (txn: tx029712cb1a15454ba1401-00665ca441) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_missing_backend_timestamp ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_put_x_timestamp_conflict_with_other_weird_success_response ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_txn_id_logging_on_PUT ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='a/c', lower='', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Get listing from /v1/a/c?format=json&limit=10000&marker=&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Found 1024 objects in shard (state=sharded), total = 1024 (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Setting object count to 1024 (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Found 1024 objects in shard (state=sharding), total = 1024 (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Setting object count to 1024 (txn: tx86d1023c29104578958c0-00665ca454) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test INFO: Caching listing namespaces for shard-listing-v2/.shards_a/c_pie (3 namespaces) (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c_sub_pie', lower='lemon', upper='') a/c_sub_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Get listing from /v1/a/c_sub_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Found 4 objects in shard (state=unsharded), total = 8 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Found 8 objects in shard (state=sharded), total = 48 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Setting object count to 1072 (txn: tx16279e7b0632442198292-00665ca456) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txe6cbe5782eb0410eb8065-00665ca459) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_mixed_policies ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_sharding_shard_with_memcache ... ok test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_with_read_permission ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to get commit status of PUT to /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 523, in _get_conn_response resp = putter.await_response( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1750, in await_response self.resp = self.conn.getexpect() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 854, in getexpect expect_status = self._status.get_expect_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 757, in get_expect_status raise expect_status Exception: kaboom! (txn: tx7a4297d1ace4483b957d3-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to get commit status of PUT to /v1/a/c/o: Timeout (Nones) (txn: txe3d21802b12644ed8d4aa-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: ERROR Client read timeout (Nones) (txn: txedf2ebedcb9645d7bc084-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected without sending last chunk (txn: tx2a22a9fe7711459e97ff6-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception transferring data to object servers {'path': '/v1/a/c/o.jpg'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3173, in _transfer_data chunk = next(data_source) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 882, in reader return req.environ['wsgi.input'].read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/proxy/controllers/test_obj.py", line 6776, in read raise Exception('exception message') Exception: exception message (txn: txf6fe9e16d3fb4571817b6-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR Exception causing client disconnect: Timeout (Nones) (txn: tx84a03c937e9741c5b39b3-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx731397e6046846138997d-00665ca44e) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_error_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_mostly_success_commit ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_old_obj_server ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_body_and_bad_etag ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_both_body ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_chunked_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_complete_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_fullcontrol_permission ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:21 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347387.25248', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:18 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_ami_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_canned_s3acl ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347379.43411', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347384.88097', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:24 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'X-Copy-From': '/bucket/object', 'Content-Length': 0} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test WARNING: The following digest algorithms are configured but deprecated: sha1. Support will be removed in a future release. test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10039830207824707 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10215044021606445 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10253691673278809 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10286140441894531 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10314702987670898 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10341835021972656 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10370993614196777 test ERROR: Timeout getting a connection to memcached: 1.2.3.5:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10398650169372559 test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.001436710357666 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0019257068634033 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0021872520446777 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.0024456977844238 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 1.002694845199585 test.unit.common.test_memcached.TestMemcached.test_connection_pooling test.unit.common.test_memcached.TestMemcached.test_connection_pooling ... ok test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect test.unit.common.test_memcached.TestMemcached.test_connection_slow_connect ... ok test.unit.common.test_memcached.TestMemcached.test_decr test.unit.common.test_memcached.TestMemcached.test_decr ... ok test.unit.common.test_memcached.TestMemcached.test_delete test.unit.common.test_memcached.TestMemcached.test_delete ... ok test.unit.common.test_memcached.TestMemcached.test_error_disabled test.unit.common.test_memcached.TestMemcached.test_error_disabled ... ok test INFO: - - - [02/Jun/2024:16:56:26 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0154 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:56:32 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656251 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0202 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0117 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0113 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0018 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0024 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0022 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0146 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0134 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0149 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0133 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0129 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0116 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0142 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0097 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:36 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0121 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0127 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0040 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0144 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0022 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0003 "-" 3656251 nada test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0003 "-" 3656251 999 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_legacy" 201 - "-" "-" "-" 0.0120 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/c_legacy" 204 - "-" "-" "-" 0.0021 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_one" 201 - "-" "-" "-" 0.0117 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/c_one" 204 - "-" "-" "-" 0.0017 "-" 3656251 1 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_two" 201 - "-" "-" "-" 0.0177 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/c_two" 204 - "-" "-" "-" 0.0023 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_three" 201 - "-" "-" "-" 0.0153 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "HEAD /sda1/p/a/c_three" 204 - "-" "-" "-" 0.0022 "-" 3656251 3 test INFO: - - - [02/Jun/2024:16:56:36 +0000] "PUT /sda1/p/a/c_four" 201 - "-" "-" "-" 0.0122 "-" 3656251 4 test INFO: - - - [02/Jun/2024:16:56:37 +0000] "HEAD /sda1/p/a/c_four" 204 - "-" "-" "-" 0.0026 "-" 3656251 4 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:37 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:37 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:37 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0144 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:37 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_listing_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_log_line_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_markers_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_not_allowed_method ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_multiple_timestamps ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_object_update_with_offset ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_format ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_params_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_and_delim_with_reserved ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347396.00000 <= '409 (1717347397.00000)' (txn: txad29302c40944bfc9afe4-00665ca444) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 412, [409, 412, 412] (txn: txa0abd3937a164475bbcc3-00665ca444) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347400.00000 <= '409 (unknown), 409 (unknown), 409 (unknown)' (txn: txfcad4890ddd74e2c8adaf-00665ca448) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Object PUT returning 202 for 409: 1717347400.00000 <= '409 (1717347401.00000), 201 (unknown), 201 (unknown)' (txn: tx2325a91d9cf14c20a0ed3-00665ca448) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_write_affinity_proxy_server_config ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden test.unit.proxy.controllers.test_obj.TestReplicatedObjController.test_x_timestamp_not_overridden ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_headers_and_body ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_half_not_found_statuses ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_handoff ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_limits_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_missing_one ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_mostly_not_found ... ok test DEBUG: Setting object count to 1072 (txn: txe6cbe5782eb0410eb8065-00665ca459) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Setting object count to 1072 (txn: tx9f987a20d64843afa77ca-00665ca45c) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: GET listing from 3 shards for: /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00&states=listing (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Getting listing part 0 from shard Namespace(name='a/c_sub_juice', lower='ham', upper='juice') a/c_sub_juice with {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Get listing from /v1/a/c_sub_juice?format=json&limit=9960&marker=h&end_marker=juice%00 {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Getting listing part 1 from shard Namespace(name='a/c_sub_lemon', lower='juice', upper='lemon') a/c_sub_lemon with {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Get listing from /v1/a/c_sub_lemon?format=json&limit=9958&marker=j&end_marker=lemon%00 {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Found 2 objects in shard (state=unsharded), total = 4 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_pie', lower='lemon', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'object'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9956&marker=l&end_marker=pie%00 {'X-Backend-Record-Type': 'object'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Found 4 objects in shard (state=sharding), total = 8 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Found 8 objects in shard (state=sharding), total = 48 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Setting object count to 1072 (txn: tx14ac7d5db18e488cbb26a-00665ca45e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: txa034146c4e68439f80139-00665ca461) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_no_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_delimiter_reverse_no_memcache ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '2da8fe73cf5cabfbee6d71150843c60d', 'aa07db42-3f17-4530-a2d1-aa59da7b0471-sda', '1717347385.89709', '1717347387', '0', '', '1717347387', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('aa07db42-3f17-4530-a2d1-aa59da7b0471-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpv9pyz_8k/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 4.90311 seconds (0.20395/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpv9pyz_8k/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '4c462415d8b2cab310d5377dff01f36a', 'ba555cf2-7903-4c03-b0ca-3d5efc8c2879-sda', '1717347390.77941', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('ba555cf2-7903-4c03-b0ca-3d5efc8c2879-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '4c462415d8b2cab310d5377dff01f36a', 'ba555cf2-7903-4c03-b0ca-3d5efc8c2879-sda', '1717347390.77941', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('ba555cf2-7903-4c03-b0ca-3d5efc8c2879-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '81a63d5d-4357-4918-97f0-057214a71483-sda', '1717347395.72907', '1717347395.72267', '0', '', '1717347395.72267', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp63rcjghh/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '56d28fff-5844-458c-a306-cd9c506c5ede-sda', '1717347395.75188', '1717347395.74741', '0', '', '1717347395.74741', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('56d28fff-5844-458c-a306-cd9c506c5ede-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '56d28fff-5844-458c-a306-cd9c506c5ede-sda', '1717347395.75188', '1717347395.74741', '0', '', '1717347395.74741', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('56d28fff-5844-458c-a306-cd9c506c5ede-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05907 seconds (16.93017/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '0466f94a894e0060495de94cafcdd3c7', '14d3ade5-997b-44e1-abdf-158e349e390d-sda', '1717347395.83465', '1717347395.8296', '0', '', '1717347395.8296', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, rsync_then_merge, ('14d3ade5-997b-44e1-abdf-158e349e390d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (11, '13aed4abcb4530e7b742fa5e40ea94eb', '0556dda3-3548-4008-9352-f4b7bca4d584-sda', '1717347395.91877', '1717347395.91251', '0', '', '1717347395.91251', 11, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o_0', 'created_at': '1717347395.94161', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': '/a/c/o_1', 'created_at': '1717347395.94289', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': '/a/c/o_2', 'created_at': '1717347395.94323', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': '/a/c/o_3', 'created_at': '1717347395.94354', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': '/a/c/o_4', 'created_at': '1717347395.94390', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 6, 'name': '/a/c/o_5', 'created_at': '1717347395.94422', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 7, 'name': '/a/c/o_6', 'created_at': '1717347395.94452', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 8, 'name': '/a/c/o_7', 'created_at': '1717347395.94482', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 9, 'name': '/a/c/o_8', 'created_at': '1717347395.94517', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 10, 'name': '/a/c/o_9', 'created_at': '1717347395.94554', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 11, 'name': '/a/c/o_missing', 'created_at': '1717347395.94587', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '0556dda3-3548-4008-9352-f4b7bca4d584-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp6gqm1eri/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bb44c9e5-acd6-47de-95af-d7e9e5717298-sda', '1717347396.05323', '1717347399', '0', '', '1717347399', 0, 1) test DEBUG: /tmp/tmp6gqm1eri/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bb44c9e5-acd6-47de-95af-d7e9e5717298-sda', '1717347396.05323', '1717347399', '0', '', '1717347399', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('bb44c9e5-acd6-47de-95af-d7e9e5717298-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05758 seconds (17.36704/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpd12227ls/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '801ef94f02a8e3d290694709efc32317', '9d4512f9-1627-4172-8831-2a6ee368b42e-sda', '1717347396.16797', '1717347399', '0', '', '1717347399', 1, 0) test DEBUG: /tmp/tmpd12227ls/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347400.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '9d4512f9-1627-4172-8831-2a6ee368b42e-sda') test DEBUG: /tmp/tmpd12227ls/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0157022s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_create ... ok proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.13:1013/sdn (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx731397e6046846138997d-00665ca44e) proxy-server ERROR: Not enough object servers ack'ed (got 0) (txn: tx731397e6046846138997d-00665ca44e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.2:1002/sdc (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.3:1003/sdd (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.6:1006/sdg (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server ERROR: Not enough object servers ack'ed (got 3) (txn: txd5f842e648144ff0a6bb5-00665ca450) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.4:1004/sde (txn: tx33a7e74bb1384682b7785-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.7:1007/sdh (txn: tx33a7e74bb1384682b7785-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: tx33a7e74bb1384682b7785-00665ca450) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.12:1012/sdm (txn: txe63448c2bdaa4009a9d86-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: txe63448c2bdaa4009a9d86-00665ca450) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txe63448c2bdaa4009a9d86-00665ca450) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Expect: 100-continue on /a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 724, in _connect_put_node putter = self._make_putter(node, part, req, headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 3038, in _make_putter return MIMEPutter.connect( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2001, in connect raise FooterNotSupported() swift.common.exceptions.FooterNotSupported (txn: tx05eddc39defd47029ca08-00665ca450) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_explicit_commit_status ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_footers ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_just_enough_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_less_durable_responses ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_PUT_with_slow_commits ... ok test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT test.unit.proxy.controllers.test_obj.TestECObjControllerMimePutter.test_txn_id_logging_ECPUT ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'X-Amz-Metadata-Directive': 'COPY', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347387.27333', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: some source HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/some/source', 'Date': 'Sun, 02 Jun 2024 16:56:31 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'X-Amz-Meta-Something': 'oh hai', 'X-Amz-Meta-Unreadable-Prefix': '\x04w', 'X-Amz-Meta-Unreadable-Suffix': 'h\x04', 'X-Amz-Meta-Lots-Of-Unprintable': '\x04\x04\x04\x04\x04', 'Cache-Control': 'hello', 'Content-Disposition': 'how are you', 'Content-Encoding': 'good and you', 'Content-Language': 'great', 'Expires': 'yeah', 'X-Robots-Tag': 'bye', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347391.58504', 'Content-Type': 'so'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_copy ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_error ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (nulo): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.0:1000/sda, error: ERROR Insufficient Storage (txn: tx3ddc6e2d845d4260924a6-00665ca44f) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx3ddc6e2d845d4260924a6-00665ca44f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_not_found ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_with_write_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_after_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_DELETE_write_affinity_before_replication ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_GET_all_primaries_error_limited ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_simple ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_some_missing ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_negative_delete_after ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347390.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 4.61503005027771 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 4.6156227588653564 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout getting a connection to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 1.0, time_spent 4.61589503288269 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Timeout connecting to memcached: 1.2.3.4:11211: with key_prefix key, method set, config_timeout 0.1, time_spent 0.10370230674743652 test ERROR: Error connecting to memcached: ALL: with key_prefix key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 0.00012087821960449219: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.4:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test.unit.common.test_memcached.TestMemcached.test_error_limiting test.unit.common.test_memcached.TestMemcached.test_error_limiting ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '801ef94f02a8e3d290694709efc32317', '9d4512f9-1627-4172-8831-2a6ee368b42e-sda', '1717347396.16797', '1717347399', '0', '', '1717347399', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('9d4512f9-1627-4172-8831-2a6ee368b42e-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpd12227ls/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 3.14939 seconds (0.31752/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpd12227ls/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'a0d43956dd1c57fe1c6a0ee9b2924c3b', '627d7ea9-c44c-420a-a09c-1f2fc7a1068b-sda', '1717347399.30971', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('627d7ea9-c44c-420a-a09c-1f2fc7a1068b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'a0d43956dd1c57fe1c6a0ee9b2924c3b', '627d7ea9-c44c-420a-a09c-1f2fc7a1068b-sda', '1717347399.30971', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('627d7ea9-c44c-420a-a09c-1f2fc7a1068b-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp3a8pszma/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2745340d-0525-4855-bbcf-433099c93af1-sda', '1717347399.41506', '1717347402', '0', '', '1717347402', 0, 1) test DEBUG: /tmp/tmp3a8pszma/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '2745340d-0525-4855-bbcf-433099c93af1-sda', '1717347399.41506', '1717347402', '0', '', '1717347402', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('2745340d-0525-4855-bbcf-433099c93af1-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05705 seconds (17.52743/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpawkzh4i8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '226e273d3cc2649f388ab831949e8cf0', 'd9742244-5823-4a55-86b2-9fff21a2a4ab-sda', '1717347399.52718', '1717347402', '0', '', '1717347402', 1, 1) test DEBUG: /tmp/tmpawkzh4i8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '226e273d3cc2649f388ab831949e8cf0', 'd9742244-5823-4a55-86b2-9fff21a2a4ab-sda', '1717347399.52718', '1717347402', '0', '', '1717347402', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d9742244-5823-4a55-86b2-9fff21a2a4ab-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpawkzh4i8/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07914 seconds (12.63636/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpawkzh4i8/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '9f883de80b1b15dcec03fda75e0f3f9e', '60847fb7-313d-45ba-a7aa-b5373b837b04-sda', '1717347399.60781', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('60847fb7-313d-45ba-a7aa-b5373b837b04-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '9f883de80b1b15dcec03fda75e0f3f9e', '60847fb7-313d-45ba-a7aa-b5373b837b04-sda', '1717347399.60781', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('60847fb7-313d-45ba-a7aa-b5373b837b04-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpsz13lgwy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '708411a7-c4d6-4a59-b58b-442c01c91be8-sda', '1717347401.74133', '1717347401', '1717347405', '', '1717347405', 0, 0) test DEBUG: /tmp/tmpsz13lgwy/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '708411a7-c4d6-4a59-b58b-442c01c91be8-sda', '1717347401.74133', '1717347401', '1717347405', '', '1717347405', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('708411a7-c4d6-4a59-b58b-442c01c91be8-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05242 seconds (19.07669/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpj3z32vf5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'bc13c05748cbf6a5e7f1a58ae0b8765e', '53b45fc9-5b66-437e-921f-913e3d4cf9a2-sda', '1717347401.85049', '1717347401', '1717347405', '', '1717347405', 1, 1) test DEBUG: /tmp/tmpj3z32vf5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347406.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], '53b45fc9-5b66-437e-921f-913e3d4cf9a2-sda') test DEBUG: /tmp/tmpj3z32vf5/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0105171s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'bc13c05748cbf6a5e7f1a58ae0b8765e', '53b45fc9-5b66-437e-921f-913e3d4cf9a2-sda', '1717347401.85049', '1717347401', '1717347405', '', '1717347405', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('53b45fc9-5b66-437e-921f-913e3d4cf9a2-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpj3z32vf5/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09815 seconds (10.18830/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_newer_local_delete ... ok test INFO: - - - [02/Jun/2024:16:56:37 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0139 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:40 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0042 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0169 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0016 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0013 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0133 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0040 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0035 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0040 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0171 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0145 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0205 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0168 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0072 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0126 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0182 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0265 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0015 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0174 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0212 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0169 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0159 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0160 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0143 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0193 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0189 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0049 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0042 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0005 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0176 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0045 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0049 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0044 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0046 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0047 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0056 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0219 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0015 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0014 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0008 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0011 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0150 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:43 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0059 "-" 3656251 2 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_prefix_with_reserved ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_put_auto_create ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_serv_reserv ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types test.unit.container.test_server.TestNonLegacyDefaultStoragePolicy.test_weird_content_types ... ok test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable test.unit.obj.test_diskfile.TestSuffixHashes.test_auditor_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_drops_old_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_drops_isolated_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_hsh_path_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_ignore_orphaned_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_isolated_meta_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_meta_data_purge_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_one_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keep_single_old_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_keeps_one_datafile_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_multi_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_older_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_only_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_data_newer_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_expired_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_data_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_old_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_older_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purge_ts_newer_data_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_purges_single_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error test.unit.obj.test_diskfile.TestSuffixHashes.test_cleanup_ondisk_files_reclaim_tombstone_remove_file_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception test.unit.obj.test_diskfile.TestSuffixHashes.test_consolidate_hashes_raises_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_bad_dev ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_consolidates_suffix_rehash_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_creates_pkl ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_create_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_does_not_rehash_known_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_hash_suffix_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_modified_recursive_retry ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_multi_file_multi_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_finds_new_suffix_dirs_unicode ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_lying_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_new_pkl_missing_invalid_finds_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_old_pickle_does_not_find_new_suffix_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_existing_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_invalid_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_race_new_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_get_hashes_zero_bytes_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_existing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_invalidations_race_get_hashes_same_suffix_new ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enodata_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_enotdir_quarantined ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_cleanup_ondisk_files_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_listdir_enotdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multi_file_ends_in_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_multifile_ends_in_datafile_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_datafile ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_and_one_valid_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_reclaim_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_one_tombstone_and_one_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdir_and_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_removes_empty_hashdirs_in_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_hsh_path_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_rmdir_suffix_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_invalidate_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_ts_cleanup_after_recalc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_obsolete_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_older_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_newer_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta test.unit.obj.test_diskfile.TestSuffixHashes.test_hash_suffix_with_same_age_content_type_in_older_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_bad_pickle ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_consolidation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_empty_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_file_not_truncated_when_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_invalidate_hash_racing_get_hashes_diff_suffix_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_existing_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part test.unit.obj.test_diskfile.TestSuffixHashes.test_unpickle_error_and_get_hashes_failure_new_part ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix test.unit.obj.test_diskfile.TestSuffixHashes.test_valid_suffix ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_creates_empty_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_mixed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_no_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones test.unit.obj.test_reconstructor.TestObjectReconstructor.test_build_jobs_revert_only_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_cleans_tmp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_creates_datadir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_mount_check ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_non_local_devs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_multi_device_skips_non_ring_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_invalid_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_ec_policy_and_device ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port test.unit.obj.test_reconstructor.TestObjectReconstructor.test_collect_parts_skips_non_local_devs_servers_per_port ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_local_devices_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffix_delta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_get_suffixes_to_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_first_enables_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_default ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_enabled ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_false_and_first_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_ignores_handoffs_first ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_none_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true test.unit.obj.test_reconstructor.TestObjectReconstructor.test_handoffs_only_true_and_first_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_down ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_not_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_primary_some_in_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_but_already_reclaimed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_cleanup_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_cleanup_meta_pure_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_also_primary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_does_not_cleanup_meta_new_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_revert_is_handoff_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_suffix_call_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_sync_partner_unmounted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff test.unit.obj.test_reconstructor.TestObjectReconstructor.test_process_job_will_not_revert_to_handoff ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_removes_non_partition_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env test.unit.obj.test_reconstructor.TestObjectReconstructor.test_reconstruct_with_no_ec_policy_env ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip test.unit.obj.test_reconstructor.TestObjectReconstructor.test_ring_ip_and_bind_ip ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region test.unit.obj.test_reconstructor.TestObjectReconstructor.test_sync_duplicates_to_remote_region ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies test.unit.obj.test_reconstructor.TestObjectReconstructor.test_two_ec_policies ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestConfigOptionHandling.test_default test.unit.obj.test_server.TestConfigOptionHandling.test_default ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_app ... ok test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_both ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.11:1011/sdl (txn: tx1b01501750194a61a9253-00665ca456) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.8:1008/sdi (txn: tx1b01501750194a61a9253-00665ca456) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.10:1010/sdk (txn: tx1b01501750194a61a9253-00665ca456) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.1:1001/sdb (txn: txede9966d2849485eb03f2-00665ca456) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.5:1005/sdf (txn: txede9966d2849485eb03f2-00665ca456) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.9:1009/sdj (txn: txede9966d2849485eb03f2-00665ca456) proxy-server ERROR: ERROR 503 b'' Trying to PUT /v1/a/c/o From Object Server 10.0.0.0:1000/sda (txn: txede9966d2849485eb03f2-00665ca456) proxy-server ERROR: Object PUT returning 503 for [201, 503, 201, 201, 503, 201, 201, 503, 503, 201, 201, 201, 201, 201] (txn: txede9966d2849485eb03f2-00665ca456) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpq3wlw0dd/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpq3wlw0dd/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpq3wlw0dd/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpq3wlw0dd/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0164 "-" 3656202 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0105 "-" 3656202 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0120 "-" 3656202 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0089 "-" 3656202 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3b47b976137845b9acbc1-00665ca456" "proxy-server 3656202" 0.0016 "-" 3656202 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:59755/sda1/1/a/c" "tx3b47b976137845b9acbc1-00665ca456" "container-server 3656202" 0.0017 "-" 3656202 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:55469/sdb1/1/a/c" "tx3b47b976137845b9acbc1-00665ca456" "container-server 3656202" 0.0015 "-" 3656202 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx3b47b976137845b9acbc1-00665ca456" "proxy-server 3656202" 0.0194 "-" 3656202 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx3b47b976137845b9acbc1-00665ca456" "proxy-server 3656202" 0.0333 "-" 3656202 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/54 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx3b47b976137845b9acbc1-00665ca456 - 0.0450 - - 1717347414.345449209 1717347414.390490532 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx2a092ddc1f5c4ff5b5a88-00665ca456" "proxy-server 3656202" 0.0018 "-" 3656202 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:59755/sda1/1/a1/c1" "tx2a092ddc1f5c4ff5b5a88-00665ca456" "container-server 3656202" 0.0017 "-" 3656202 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:55469/sdb1/1/a1/c1" "tx2a092ddc1f5c4ff5b5a88-00665ca456" "container-server 3656202" 0.0015 "-" 3656202 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2a092ddc1f5c4ff5b5a88-00665ca456" "proxy-server 3656202" 0.0348 "-" 3656202 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx2a092ddc1f5c4ff5b5a88-00665ca456" "proxy-server 3656202" 0.0231 "-" 3656202 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/54 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx2a092ddc1f5c4ff5b5a88-00665ca456 - 0.0471 - - 1717347414.392563105 1717347414.439687967 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa913755376494ccdb4abf-00665ca456" "proxy-server 3656202" 0.0094 "-" 3656202 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59755/sda1/2/a/c1" "txa913755376494ccdb4abf-00665ca456" "container-server 3656202" 0.0015 "-" 3656202 1 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default test.unit.obj.test_server.TestConfigOptionHandling.test_option_in_default ... ok test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECDuplicationObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECDuplicationObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECDuplicationObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECDuplicationObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECDuplicationObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_containers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object test.unit.proxy.test_server.TestSocketObjectVersions.test_version_manifest_utf8_version_container_utf_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_PUT_sysmeta_then_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_not_updated_by_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_replaced_by_PUT ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_sysmeta_updated_by_COPY_from ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST test.unit.proxy.test_sysmeta.TestObjectSysmeta.test_transient_sysmeta_replaced_by_PUT_or_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txfc9a4a83bb7b4883b3290-00665ca436) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txde8d54a59b4a45fbb39fc-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx86f26f0b2b7747a3bc289-00665ca436) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa54184a79e39483bb945f-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mismatched_fragment_archives ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_ranged_responses_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_mixed_success_with_range ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_ndata_equals_nparity_with_missing_and_errors ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:54 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:55469/sdb1/2/a/c1" "txa913755376494ccdb4abf-00665ca456" "container-server 3656202" 0.0087 "-" 3656202 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa913755376494ccdb4abf-00665ca456" "proxy-server 3656202" 1.7662 "-" 3656202 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa913755376494ccdb4abf-00665ca456" "proxy-server 3656202" 1.7932 "-" 3656202 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/56 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txa913755376494ccdb4abf-00665ca456 - 1.8144 - - 1717347414.441606998 1717347416.256050348 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd246d7a7c21e4c31919c9-00665ca458" "proxy-server 3656202" 0.0106 "-" 3656202 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:59755/sda1/1/a/c2" "txd246d7a7c21e4c31919c9-00665ca458" "container-server 3656202" 0.0078 "-" 3656202 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:55469/sdb1/1/a/c2" "txd246d7a7c21e4c31919c9-00665ca458" "container-server 3656202" 0.0014 "-" 3656202 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd246d7a7c21e4c31919c9-00665ca458" "proxy-server 3656202" 0.0255 "-" 3656202 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:56:56 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd246d7a7c21e4c31919c9-00665ca458" "proxy-server 3656202" 0.0398 "-" 3656202 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/56/56 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txd246d7a7c21e4c31919c9-00665ca458 - 0.0613 - - 1717347416.258029222 1717347416.319287300 2 test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/bucket/object', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'X-Amz-Metadata-Directive': 'REPLACE', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347396.00000', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_fullcontrol_permission ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: tx816188f2f1a2417f94ef9-00665ca455) proxy-server ERROR: Object GET returning 503 for [] (txn: tx816188f2f1a2417f94ef9-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx539e78bf8d954547b22a5-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx7f43c43cb4ff4f7cab98e-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3a71a136e9d74d299c7cc-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_object_DELETE_backend_update_container_ip_default ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:37 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:40 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_initiate_multipart_upload_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_with_read_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347401.22149', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347401.24091', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_location_invalid_xml ... ok test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location test.unit.common.middleware.s3api.test_bucket.TestS3ApiBucketAcl.test_bucket_PUT_with_strange_location ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_bad_hash ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_empty_bad_hash ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_good ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_long ... ok test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short test.unit.common.middleware.s3api.test_s3request.TestHashingInput.test_too_short ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_container test.unit.common.middleware.test_bulk.TestUntar.test_bad_container ... ok test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar test.unit.common.middleware.test_bulk.TestUntar.test_bad_tar ... ok test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required test.unit.common.middleware.test_bulk.TestUntar.test_content_length_required ... ok test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path test.unit.common.middleware.test_bulk.TestUntar.test_create_container_for_path ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_call test.unit.common.middleware.test_bulk.TestUntar.test_extract_call ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_compress_type ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_cont_401 ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_create_cont_value_err ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_cont ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_failed_extractions ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_max_file_size ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_401 ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_obj_name_len ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_fail_unicode ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_with_basefile ... ok test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works test.unit.common.middleware.test_bulk.TestUntar.test_extract_tar_works ... ok test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body test.unit.common.middleware.test_bulk.TestUntar.test_get_response_body ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_no_segments ... ok test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments test.unit.common.middleware.test_dlo.TestDloHeadManifest.test_head_large_object_too_many_segments ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACLWithConfiguredDomain.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_marker_over_versioned_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_delete_object_with_versioning_never_enabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success test.unit.common.middleware.test_object_versioning.ObjectVersioningTestDelete.test_put_delete_marker_no_object_success ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_DELETE_current_version ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_POST_error_versioning_disabled ... ok test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled test.unit.common.middleware.test_object_versioning.ObjectVersioningVersionAPIWhileDisabled.test_PUT_version_versioning_disbaled ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_account_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_non_swift_path ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_destination_account_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_off_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_off_src_acct_on_dest_acct_on_on_copy ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_account_off ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_allow_deletes ... ok test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy test.unit.common.middleware.test_read_only.TestReadOnly.test_global_read_only_on_destination_account_off_on_copy ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_fail ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_device_info_function ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail test.unit.common.middleware.test_recon.TestReconMiddleware.test_get_swift_conf_md5_fail ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_list_parts_error ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content test.unit.common.middleware.test_recon.TestReconMiddleware.test_no_content ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_async ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_auditor_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_driveaudit ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_expirer_object ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_load ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mem ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_quarantined ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_relink ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_all ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_replication_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_ringmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sharding ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_sockstat ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_swiftconfmd5 ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_time ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_unmounted_empty ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_invalid ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_updater_notype ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_get_version ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_invalid_path ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass test.unit.common.middleware.test_recon.TestReconMiddleware.test_recon_pass ... ok test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info test.unit.common.middleware.test_recon.TestReconMiddleware.test_reconstruction_info ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_get_with_if_modified_since ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_alternate_etag_non_slo_after_refetch ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_match_mismatches_manifest_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_modified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_matches_no_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_none_match_mismatches_json_md5 ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_ancient_date ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_last_modified ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_if_unmodified_since_now ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_alternate_etag_miss ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_match_mismatches_without_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_matches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_manifest_get_if_none_match_mismatches_alternate_etag ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_old_swift_if_match_matches_and_range ... ok test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download test.unit.common.middleware.test_slo.TestSloConditionalGetOldManifest.test_range_resume_download ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_nested ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_non_ascii_same_container ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_too_many_containers ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_turned_off ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_async_delete_whole_404 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_401 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_json ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_bad_utf8 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_client_content_type ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_man ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_401 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_404 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_500 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_nested_too_many_segments ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_non_ascii ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_not_a_manifest ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_segment_404 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_404 ... ok test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift test.unit.common.middleware.test_slo.TestSloDeleteManifest.test_handle_multipart_delete_whole_old_swift ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults test.unit.common.middleware.test_slo.TestSwiftInfo.test_registered_non_defaults ... ok test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler test.unit.common.middleware.test_xprofile.TestXProfile.test_get_profiler ... ok test.unit.common.test_internal_client.TestGetAuth.test_invalid_version test.unit.common.test_internal_client.TestGetAuth.test_invalid_version ... ok test.unit.common.test_internal_client.TestGetAuth.test_ok test.unit.common.test_internal_client.TestGetAuth.test_ok ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter ... ok test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges test.unit.common.test_request_helpers.TestSegmentedIterable.test_simple_segments_app_iter_ranges ... ok test.unit.common.test_utils.TestPunchHole.test_arg_bounds test.unit.common.test_utils.TestPunchHole.test_arg_bounds ... ok test.unit.common.test_utils.TestPunchHole.test_error test.unit.common.test_utils.TestPunchHole.test_error ... ok test.unit.common.test_utils.TestPunchHole.test_no_fallocate test.unit.common.test_utils.TestPunchHole.test_no_fallocate ... ok test.unit.common.test_utils.TestPunchHole.test_punch_hole test.unit.common.test_utils.TestPunchHole.test_punch_hole ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_insert_filter ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters test.unit.common.test_wsgi.TestPipelineWrapper.test_startswith_no_filters ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str test.unit.common.test_wsgi.TestPipelineWrapper.test_str ... ok test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter test.unit.common.test_wsgi.TestPipelineWrapper.test_str_unknown_filter ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add test.unit.obj.test_replicator.TestReplicatorStats.test_add ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats test.unit.obj.test_replicator.TestReplicatorStats.test_add_failure_stats ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields test.unit.obj.test_replicator.TestReplicatorStats.test_from_recon_skips_extra_fields ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip test.unit.obj.test_replicator.TestReplicatorStats.test_recon_roundtrip ... ok test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon test.unit.obj.test_replicator.TestReplicatorStats.test_to_recon ... ok test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync test.unit.obj.test_ssync.TestSsyncEC.test_fragment_sync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_only_missing_durable_state ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert test.unit.obj.test_ssync.TestSsyncEC.test_handoff_fragment_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment test.unit.obj.test_ssync.TestSsyncEC.test_handoff_non_durable_fragment ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable test.unit.obj.test_ssync.TestSsyncEC.test_revert_job_with_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index test.unit.obj.test_ssync.TestSsyncEC.test_send_invalid_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none test.unit.obj.test_ssync.TestSsyncEC.test_send_with_frag_index_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_container_info_got_cached ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:47 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0139 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:47 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0010 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:47 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:47 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:47 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0007 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:47 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0116 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:47 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0047 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0002 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0140 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0002 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0162 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0003 "-" 3656251 0 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0150 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0170 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0126 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3656251 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0004 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0001 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0192 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0147 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0161 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0135 "-" 3656251 - test INFO: - - - [02/Jun/2024:16:56:50 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0115 "-" 3656251 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0132 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:56:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0186 "-" 3656251 2 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:56:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:51 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0009 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:51 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3656251 2 test INFO: - - - [02/Jun/2024:16:56:51 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0100 "-" 3656251 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpdxpy1ixm/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpdxpy1ixm/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpdxpy1ixm/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpdxpy1ixm/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:19 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0160 "-" 3656251 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:19 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0129 "-" 3656251 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:19 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0128 "-" 3656251 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:19 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0146 "-" 3656251 - test.unit.proxy.test_server.TestController.test_account_info_200 test.unit.proxy.test_server.TestController.test_account_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_404 test.unit.proxy.test_server.TestController.test_account_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_container_count test.unit.proxy.test_server.TestController.test_account_info_container_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_account test.unit.proxy.test_server.TestController.test_account_info_no_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_account_info_no_cache test.unit.proxy.test_server.TestController.test_account_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_200 test.unit.proxy.test_server.TestController.test_container_info_200 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_404 test.unit.proxy.test_server.TestController.test_container_info_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_invalid_account test.unit.proxy.test_server.TestController.test_container_info_invalid_account ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_container_info_no_cache test.unit.proxy.test_server.TestController.test_container_info_no_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_account_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings test.unit.proxy.test_server.TestController.test_get_container_info_returns_values_as_strings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_make_requests test.unit.proxy.test_server.TestController.test_make_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestController.test_transfer_headers test.unit.proxy.test_server.TestController.test_transfer_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor test.unit.proxy.test_server.TestECMismatchedFA.test_mixing_different_objects_fragment_archives_with_dup_factor ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method get: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config test.unit.common.test_memcached.TestMemcached.test_error_limiting_custom_config ... ok test.unit.common.test_memcached.TestMemcached.test_error_raising test.unit.common.test_memcached.TestMemcached.test_error_raising ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns test.unit.common.test_memcached.TestMemcached.test_get_conns ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 test.unit.common.test_memcached.TestMemcached.test_get_conns_bad_v6 ... ok test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname test.unit.common.test_memcached.TestMemcached.test_get_conns_hostname ... ok test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_get_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr test.unit.common.test_memcached.TestMemcached.test_incr ... ok test.unit.common.test_memcached.TestMemcached.test_incr_add_expires test.unit.common.test_memcached.TestMemcached.test_incr_add_expires ... ok test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request test.unit.common.test_memcached.TestMemcached.test_incr_failed_connection_mid_request ... ok test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response test.unit.common.test_memcached.TestMemcached.test_incr_unexpected_response ... ok test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout test.unit.common.test_memcached.TestMemcached.test_incr_w_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold test.unit.common.test_memcached.TestMemcached.test_item_size_warning_threshold ... ok test.unit.common.test_memcached.TestMemcached.test_logger_kwarg test.unit.common.test_memcached.TestMemcached.test_logger_kwarg ... ok test.unit.common.test_memcached.TestMemcached.test_multi test.unit.common.test_memcached.TestMemcached.test_multi ... ok test.unit.common.test_memcached.TestMemcached.test_multi_delete test.unit.common.test_memcached.TestMemcached.test_multi_delete ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_error ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_get_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_incr_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_exception ... ok test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout test.unit.common.test_memcached.TestMemcached.test_operations_timing_stats_with_set_timeout ... ok test.unit.common.test_memcached.TestMemcached.test_retry test.unit.common.test_memcached.TestMemcached.test_retry ... ok test.unit.common.test_memcached.TestMemcached.test_serialization test.unit.common.test_memcached.TestMemcached.test_serialization ... ok test.unit.common.test_memcached.TestMemcached.test_set_error test.unit.common.test_memcached.TestMemcached.test_set_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error test.unit.common.test_memcached.TestMemcached.test_set_error_raise_on_error ... ok test.unit.common.test_memcached.TestMemcached.test_set_get_json test.unit.common.test_memcached.TestMemcached.test_set_get_json ... ok test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg test.unit.common.test_memcached.TestMemcached.test_tls_context_kwarg ... ok test.unit.common.test_swob.TestAccept.test_accept_invalid test.unit.common.test_swob.TestAccept.test_accept_invalid ... ok test.unit.common.test_swob.TestAccept.test_accept_json test.unit.common.test_swob.TestAccept.test_accept_json ... ok test.unit.common.test_swob.TestAccept.test_accept_plain test.unit.common.test_swob.TestAccept.test_accept_plain ... ok test.unit.common.test_swob.TestAccept.test_accept_xml test.unit.common.test_swob.TestAccept.test_accept_xml ... ok test.unit.common.test_swob.TestAccept.test_repr test.unit.common.test_swob.TestAccept.test_repr ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty test.unit.common.test_utils.TestAffinityLocalityPredicate.test_empty ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid test.unit.common.test_utils.TestAffinityLocalityPredicate.test_invalid ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple test.unit.common.test_utils.TestAffinityLocalityPredicate.test_multiple ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region test.unit.common.test_utils.TestAffinityLocalityPredicate.test_region ... ok test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone test.unit.common.test_utils.TestAffinityLocalityPredicate.test_zone ... ok test.unit.common.test_utils.TestClosingIterator.test_close test.unit.common.test_utils.TestClosingIterator.test_close ... ok test.unit.common.test_utils.TestClosingIterator.test_close_gen test.unit.common.test_utils.TestClosingIterator.test_close_gen ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_exception test.unit.common.test_utils.TestClosingIterator.test_close_on_exception ... ok test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration test.unit.common.test_utils.TestClosingIterator.test_close_on_stop_iteration ... ok test.unit.common.test_utils.TestClosingIterator.test_close_others test.unit.common.test_utils.TestClosingIterator.test_close_others ... ok test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter test.unit.common.test_utils.TestClosingIterator.test_close_wrapped_is_not_same_as_iter ... ok test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable test.unit.common.test_utils.TestClosingIterator.test_init_with_iterable ... ok test.unit.common.test_utils.TestClosingIterator.test_nested_iters test.unit.common.test_utils.TestClosingIterator.test_nested_iters ... ok test.unit.common.test_utils.TestShardRangeList.test_filter test.unit.common.test_utils.TestShardRangeList.test_filter ... ok test.unit.common.test_utils.TestShardRangeList.test_find_lower test.unit.common.test_utils.TestShardRangeList.test_find_lower ... ok test.unit.common.test_utils.TestShardRangeList.test_includes test.unit.common.test_utils.TestShardRangeList.test_includes ... ok test.unit.common.test_utils.TestShardRangeList.test_init test.unit.common.test_utils.TestShardRangeList.test_init ... ok test.unit.common.test_utils.TestShardRangeList.test_init_with_list test.unit.common.test_utils.TestShardRangeList.test_init_with_list ... ok test.unit.common.test_utils.TestShardRangeList.test_pop test.unit.common.test_utils.TestShardRangeList.test_pop ... ok test.unit.common.test_utils.TestShardRangeList.test_slice test.unit.common.test_utils.TestShardRangeList.test_slice ... ok test.unit.common.test_utils.TestShardRangeList.test_states test.unit.common.test_utils.TestShardRangeList.test_states ... ok test.unit.common.test_utils.TestShardRangeList.test_timestamps test.unit.common.test_utils.TestShardRangeList.test_timestamps ... ok test.unit.common.test_utils.TestSpliterator.test_big_input_string test.unit.common.test_utils.TestSpliterator.test_big_input_string ... ok test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries test.unit.common.test_utils.TestSpliterator.test_chunk_boundaries ... ok test.unit.common.test_utils.TestSpliterator.test_closing test.unit.common.test_utils.TestSpliterator.test_closing ... ok test.unit.common.test_utils.TestSpliterator.test_no_empty_strings test.unit.common.test_utils.TestSpliterator.test_no_empty_strings ... ok test.unit.common.test_utils.TestSpliterator.test_overlap test.unit.common.test_utils.TestSpliterator.test_overlap ... ok test.unit.common.test_utils.TestSpliterator.test_running_out test.unit.common.test_utils.TestSpliterator.test_running_out ... ok test.unit.common.test_utils.TestSpliterator.test_string test.unit.common.test_utils.TestSpliterator.test_string ... ok test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_check_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_non_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_ondisk_files_reclaim_with_data_files_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age test.unit.obj.test_diskfile.TestECDiskFileManager.test_cleanup_uses_configured_reclaim_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpj3z32vf5/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '1af477665d9e943a5fd7b4029837f63b', 'ed736dd7-ee44-4367-ae40-b4b9d1a5e007-sda', '1717347401.96793', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('ed736dd7-ee44-4367-ae40-b4b9d1a5e007-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '1af477665d9e943a5fd7b4029837f63b', 'ed736dd7-ee44-4367-ae40-b4b9d1a5e007-sda', '1717347401.96793', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('ed736dd7-ee44-4367-ae40-b4b9d1a5e007-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp4phe_7gb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b9505fc5-22ef-44bb-9aee-0a5a4ba56e7d-sda', '1717347408.66327', '1717347408', '1717347412', '', '1717347412', 0, 0) test DEBUG: /tmp/tmp4phe_7gb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b9505fc5-22ef-44bb-9aee-0a5a4ba56e7d-sda', '1717347408.66327', '1717347408', '1717347412', '', '1717347412', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b9505fc5-22ef-44bb-9aee-0a5a4ba56e7d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05128 seconds (19.49983/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpe83_lczb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '4f1bee0f3e31854870618e4a6c10d6e3', '352d04b7-86eb-4271-866d-76e94ac2ef59-sda', '1717347408.76549', '1717347408', '1717347412', '', '1717347412', 1, 1) test DEBUG: /tmp/tmpe83_lczb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '4f1bee0f3e31854870618e4a6c10d6e3', '352d04b7-86eb-4271-866d-76e94ac2ef59-sda', '1717347408.76549', '1717347408', '1717347412', '', '1717347412', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('352d04b7-86eb-4271-866d-76e94ac2ef59-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpe83_lczb/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.08142 seconds (12.28224/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpe83_lczb/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '63d9c064a4e1c678e33ba93841e7aa8d', 'bb0cbe48-1839-4913-ae88-78f7e0165b51-sda', '1717347408.86458', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('bb0cbe48-1839-4913-ae88-78f7e0165b51-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '63d9c064a4e1c678e33ba93841e7aa8d', 'bb0cbe48-1839-4913-ae88-78f7e0165b51-sda', '1717347408.86458', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('bb0cbe48-1839-4913-ae88-78f7e0165b51-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpn9uc4ezb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd027b806-edde-4571-89f3-1d4206995f47-sda', '1717347411.89445', '1717347411', '0', '', '1717347411', 0, 1) test DEBUG: /tmp/tmpn9uc4ezb/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd027b806-edde-4571-89f3-1d4206995f47-sda', '1717347411.89445', '1717347411', '0', '', '1717347411', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d027b806-edde-4571-89f3-1d4206995f47-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04496 seconds (22.24417/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp1mt3wp2p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '137fc8f83561f86c42cb26014209efcc', 'a14f24f0-7e72-4b86-ac70-75720a4a5395-sda', '1717347412.00546', '1717347412', '0', '', '1717347412', 1, 1) test DEBUG: /tmp/tmp1mt3wp2p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347416.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'a14f24f0-7e72-4b86-ac70-75720a4a5395-sda') test DEBUG: /tmp/tmp1mt3wp2p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0104523s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '137fc8f83561f86c42cb26014209efcc', 'a14f24f0-7e72-4b86-ac70-75720a4a5395-sda', '1717347412.00546', '1717347412', '0', '', '1717347412', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('a14f24f0-7e72-4b86-ac70-75720a4a5395-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp1mt3wp2p/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10095 seconds (9.90619/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp1mt3wp2p/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '4749ceef93bef254112d929e9e3360c9', '68e2dc82-e72c-4701-8751-2fe30e106da4-sda', '1717347412.12323', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('68e2dc82-e72c-4701-8751-2fe30e106da4-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_create ... ok test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path test.unit.obj.test_diskfile.TestECDiskFileManager.test_construct_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up test.unit.obj.test_diskfile.TestECDiskFileManager.test_focused_yield_hashes_does_not_clean_up ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_and_filenames_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_bad_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_frag_index_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_actual_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_no_meta_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_diskfile_from_hash_read_metadata_problem ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_no_rsync_temp_file_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_frag_index_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_and_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_ec_policy_some_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_empty_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_stray_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file test.unit.obj.test_diskfile.TestECDiskFileManager.test_get_ondisk_files_with_unexpected_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init test.unit.obj.test_diskfile.TestECDiskFileManager.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window test.unit.obj.test_diskfile.TestECDiskFileManager.test_init_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror test.unit.obj.test_diskfile.TestECDiskFileManager.test_listdir_other_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_for_meta_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_make_on_disk_filename_with_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning test.unit.obj.test_diskfile.TestECDiskFileManager.test_missing_splice_warning ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator test.unit.obj.test_diskfile.TestECDiskFileManager.test_object_audit_location_generator ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors test.unit.obj.test_diskfile.TestECDiskFileManager.test_parse_on_disk_filename_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_and_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name test.unit.obj.test_diskfile.TestECDiskFileManager.test_partition_lock_same_partition_different_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update test.unit.obj.test_diskfile.TestECDiskFileManager.test_pickle_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_2 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_another_device_fine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_off ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_on ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_lock_same_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation test.unit.obj.test_diskfile.TestECDiskFileManager.test_replication_one_per_device_deprecation ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files test.unit.obj.test_diskfile.TestECDiskFileManager.test_verify_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_cleans_up_everything ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_empty_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_filters_frag_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_ignores_bad_ondisk_filesets ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_optionally_yields_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_missing_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_data_without_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_newer_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_skips_non_durable_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_suffix_filter_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_hashes_yields_meta_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail test.unit.obj.test_diskfile.TestECDiskFileManager.test_yield_suffixes_dev_path_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_age_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_quarantine_threshold_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_all_404s_fails_custom_request_node_count ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_error_with_invalid_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_errors_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_exceptions_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_duplicate_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_finds_missing_frag_does_not_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_frags_on_handoffs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_invalid_frag_index_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_missing_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixed_meta_timestamps_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_mixture_of_errors_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_errors ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_not_old_enough ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_different_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_etags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_frag_resp_mixed_timestamps ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_invalid_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_etag_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_frag_index_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_missing_timestamp_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_more_than_threshold_frags ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_no_quarantine_non_404_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_absolute_number ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_four_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_three_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_one_rnc_two_replicas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_quarantine_threshold_two_with_quarantine_age ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etag_with_same_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_etags_same_timestamp_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_new_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_old_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_fa_with_mixed_timestamps_etags_fail ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_reconstruct_parity_fa_with_data_node_failure ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf test.unit.obj.test_reconstructor.TestReconstructFragmentArchive.test_request_node_count_conf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path test.unit.obj.test_updater.TestObjectUpdaterFunctions.test_split_update_path ... ok test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: txa034146c4e68439f80139-00665ca461) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: txa034146c4e68439f80139-00665ca461) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: txa034146c4e68439f80139-00665ca461) test ERROR: Aborting listing from shards due to bad response: [404] (txn: txa034146c4e68439f80139-00665ca461) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 5 shards for: /v1/a/c?delimiter=%2F&format=json&states=listing (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ha-ppy', lower='', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&format=json&limit=10000&marker=&end_marker=ha%2Fppy%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Found 2 objects in shard (state=unsharded), total = 2 (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Found 0 objects in shard (state=unsharded), total = 2 (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&format=json&limit=9998&marker=ha%2F&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: tx0976b5bd0d614e4ba7c38-00665ca464) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 6 shards for: /v1/a/c?delimiter=%2F&reverse=on&format=json&states=listing (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='pie', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Get listing from /v1/.shards_a/c_?delimiter=%2F&reverse=on&format=json&limit=10000&marker=&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Get listing from /v1/.shards_a/c_pie?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Found 0 objects in shard (state=unsharded), total = 1 (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_ham', lower='ha/ptic', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Get listing from /v1/.shards_a/c_ham?delimiter=%2F&reverse=on&format=json&limit=9999&marker=pumpkin&end_marker=ha%2Fptic {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ha-ppy', lower='ha.d', upper='ha/ppy') .shards_a/c_ha-ppy with {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Get listing from /v1/.shards_a/c_ha-ppy?delimiter=%2F&reverse=on&format=json&limit=9998&marker=ha%2F&end_marker=ha.d {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Found 1 objects in shard (state=unsharded), total = 3 (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Getting listing part 5 from shard Namespace(name='.shards_a/c_ha.d', lower='', upper='ha.d') .shards_a/c_ha.d with {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Get listing from /v1/.shards_a/c_ha.d?delimiter=%2F&reverse=on&format=json&limit=9997&marker=ha.ggle&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Found 1 objects in shard (state=unsharded), total = 4 (txn: txdfb4dbb6c66541908567d-00665ca467) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txa90c7e99651542e093502-00665ca469) test DEBUG: Setting object count to 3120 (txn: txa90c7e99651542e093502-00665ca469) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: tx7eaac2bee9d9499abf813-00665ca469) test DEBUG: Setting object count to 3120 (txn: tx7eaac2bee9d9499abf813-00665ca469) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: GET listing from 5 shards for: /v1/a/c?format=json&states=listing (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?format=json&limit=9952&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?format=json&limit=8928&marker=%D1%B0&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2096 (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=7904&marker=%E2%A8%83&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3120 (txn: txca5d7d41a1d7424281af8-00665ca46a) test DEBUG: Setting object count to 3120 (txn: txca5d7d41a1d7424281af8-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Getting listing part 2 from shard Namespace(name='a/c', lower='pie', upper='') a/c with {'X-Backend-Record-Type': 'object'} (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Get listing from /v1/a/c?format=json&limit=9952&marker=p&end_marker= {'X-Backend-Record-Type': 'object'} (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Found 3072 objects in shard (state=sharded), total = 3120 (txn: tx742d91c2fcd949bbace9c-00665ca46a) test DEBUG: Setting object count to 3120 (txn: tx742d91c2fcd949bbace9c-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: GET listing from 5 shards for: /v1/a/c?reverse=true&limit=&format=json&states=listing (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_?reverse=true&limit=10000&format=json&marker=&end_marker=%F0%9F%8C%B4 {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1024 (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?reverse=true&limit=8976&format=json&marker=%F0%9F%8C%B5&end_marker=%E2%98%83 {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2048 (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?reverse=true&limit=7952&format=json&marker=%E2%98%84&end_marker=pie {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 3072 (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_pie?reverse=true&limit=6928&format=json&marker=q&end_marker=ham {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 3080 (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Getting listing part 4 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_ham?reverse=true&limit=6920&format=json&marker=i&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 3120 (txn: txdc284e7723134ee59b442-00665ca46a) test DEBUG: Setting object count to 3120 (txn: txdc284e7723134ee59b442-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: GET listing from 5 shards for: /v1/a/c?limit=49&format=json&states=listing (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_ham?limit=49&format=json&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_pie?limit=9&format=json&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?limit=1&format=json&marker=p&end_marker=%E2%98%83%00 {'X-Backend-Record-Type': 'auto'} (txn: tx71487a75a019481cac10c-00665ca46a) test DEBUG: Found 1 objects in shard (state=unsharded), total = 49 (txn: tx71487a75a019481cac10c-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: GET listing from 2 shards for: /v1/a/c?marker=%E2%98%86&format=json&states=listing (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&format=json&limit=10000&end_marker=%F0%9F%8C%B4%00 {'X-Backend-Record-Type': 'auto'} (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: Found 1022 objects in shard (state=unsharded), total = 1022 (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_', lower='🌴', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_?marker=%E2%A8%83&format=json&limit=8978&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 2046 (txn: txf5b507cb1a6a43e599ef6-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: GET listing from 4 shards for: /v1/a/c?end_marker=%E2%98%8A&format=json&states=listing (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_ham?end_marker=ham%00&format=json&limit=10000&marker= {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_pie?end_marker=pie%00&format=json&limit=9960&marker=h {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Found 8 objects in shard (state=unsharded), total = 48 (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_☃', lower='pie', upper='☃') .shards_a/c_☃ with {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%E2%98%83?end_marker=%E2%98%83%00&format=json&limit=9952&marker=p {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_memcache ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharded_container_with_mix_ok_and_deleted_shard ... ok test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache test.unit.proxy.controllers.test_container.TestGetShardedContainerLegacy.test_GET_sharding_container_gap_in_shards_with_memcache ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:19 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd78b1dcd402d4ed291e15-00665ca46f" "proxy-server 3656251" 0.0025 "-" 3656251 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:21 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:44995/sda1/1/a/c" "txd78b1dcd402d4ed291e15-00665ca46f" "container-server 3656251" 0.0022 "-" 3656251 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:21 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:60239/sdb1/1/a/c" "txd78b1dcd402d4ed291e15-00665ca46f" "container-server 3656251" 0.0022 "-" 3656251 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:21 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txd78b1dcd402d4ed291e15-00665ca46f" "proxy-server 3656251" 0.0548 "-" 3656251 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:21 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txd78b1dcd402d4ed291e15-00665ca46f" "proxy-server 3656251" 0.0326 "-" 3656251 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/21 PUT /v1/a/c HTTP/1.0 201 - - t - - - txd78b1dcd402d4ed291e15-00665ca46f - 2.3049 - - 1717347439.674530268 1717347441.979434967 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:21 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txfa0fce44acdb467691683-00665ca471" "proxy-server 3656251" 0.0023 "-" 3656251 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:60239/sdb1/1/a1/c1" "txfa0fce44acdb467691683-00665ca471" "container-server 3656251" 0.0017 "-" 3656251 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:44995/sda1/1/a1/c1" "txfa0fce44acdb467691683-00665ca471" "container-server 3656251" 0.0016 "-" 3656251 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfa0fce44acdb467691683-00665ca471" "proxy-server 3656251" 0.0342 "-" 3656251 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txfa0fce44acdb467691683-00665ca471" "proxy-server 3656251" 0.0199 "-" 3656251 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/22 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txfa0fce44acdb467691683-00665ca471 - 0.0481 - - 1717347441.981583595 1717347442.029663801 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx16651f9dd48544de812d2-00665ca472" "proxy-server 3656251" 0.0114 "-" 3656251 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:44995/sda1/2/a/c1" "tx16651f9dd48544de812d2-00665ca472" "container-server 3656251" 0.0083 "-" 3656251 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:60239/sdb1/2/a/c1" "tx16651f9dd48544de812d2-00665ca472" "container-server 3656251" 0.0020 "-" 3656251 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx16651f9dd48544de812d2-00665ca472" "proxy-server 3656251" 0.0445 "-" 3656251 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx16651f9dd48544de812d2-00665ca472" "proxy-server 3656251" 0.0319 "-" 3656251 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/22 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx16651f9dd48544de812d2-00665ca472 - 0.0677 - - 1717347442.031879663 1717347442.099569559 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa3745311f76b4220bfef3-00665ca472" "proxy-server 3656251" 0.0103 "-" 3656251 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:60239/sdb1/1/a/c2" "txa3745311f76b4220bfef3-00665ca472" "container-server 3656251" 0.0099 "-" 3656251 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:44995/sda1/1/a/c2" "txa3745311f76b4220bfef3-00665ca472" "container-server 3656251" 0.0018 "-" 3656251 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa3745311f76b4220bfef3-00665ca472" "proxy-server 3656251" 0.0416 "-" 3656251 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:22 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa3745311f76b4220bfef3-00665ca472" "proxy-server 3656251" 0.0303 "-" 3656251 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/22 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txa3745311f76b4220bfef3-00665ca472 - 0.0641 - - 1717347442.101713896 1717347442.165773869 2 proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347406.00646', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347406.02749', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_grantee_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_with_owner_permission ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_multipart_uploads_acl_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347408.34976', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347408.36821', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_dst_container_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: txa1d72e7474e64688b8c93-00665ca463) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: txa1d72e7474e64688b8c93-00665ca463) test INFO: Client disconnected on read of '/a/c' (txn: txa1d72e7474e64688b8c93-00665ca463) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_handoff_has_deleted_database ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_reseller_admin ... ok test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error limiting server 1.2.3.5:11211 test ERROR: Error connecting to memcached: ALL: with key_prefix some_key, method set: No more memcached servers to try test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix some_key, method decr, time_spent 5.888938903808594e-05: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 488, in _incr_or_decr line = fp.readline().strip().split() ^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip' test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0, expired ttl=1.23 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method incr, time_spent 1.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 528, in incr new_val = self._incr_or_decr(fp, sock, cmd, delta_val) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 493, in _incr_or_decr return int(line[0].strip()) ^^^^^^^^^^^^^^^^^^^^ ValueError: invalid literal for int() with base 10: b'UNEXPECTED' test WARNING: Item size larger than warning threshold: 80 (80) >= 80 (80) test WARNING: Item size larger than warning threshold: 81 (81) >= 80 (80) test WARNING: Item size larger than warning threshold: 99 (99) >= 80 (80) test WARNING: Item size larger than warning threshold: 100 (100) >= 80 (80) test WARNING: Item size larger than warning threshold: 0 (0) >= 0 (0) test WARNING: Item size larger than warning threshold: 1 (1) >= 0 (0) test WARNING: Item size larger than warning threshold: 50 (50) >= 0 (0) test WARNING: Item size larger than warning threshold: 99 (99) >= 0 (0) test WARNING: Item size larger than warning threshold: 100 (100) >= 0 (0) test WARNING: Item size larger than warning threshold: 2048576 (2Mi) >= 1000000 (977Ki) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix multi_key, method get_multi, time_spent 5.221366882324219e-05, incomplete read test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0, failed to connect test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix get_key, method get, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 460, in get sock.sendall(b'get ' + cmd.hash_key + b'\r\n') File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1282, in handle_get raise Exception('get failed') Exception: get failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method get, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix incr_key, method incr, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 531, in incr new_val = self._add(fp, sock, cmd, add_val, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 496, in _add sock.sendall(b' '.join([ File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1224, in handle_add raise Exception('add failed') Exception: add failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix nvratelimit/v2/wf, method incr, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix set_key, method set, time_spent 0.0: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/memcached.py", line 427, in set sock.sendall(set_msg(cmd.hash_key, flags, timeout, value)) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 107, in sendall handler(*parts[1:]) File "/build/reproducible-path/swift-2.33.0/test/unit/common/test_memcached.py", line 1252, in handle_set raise Exception('set failed') Exception: set failed test ERROR: Timeout talking to memcached: 1.2.3.4:11211: with key_prefix shard-updating-v2/acc, method set, config_timeout 0.01, time_spent 1.0 test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method set, time_spent 0.0, [Errno 32] Broken pipe test ERROR: Error talking to memcached: 1.2.3.5:11211: with key_prefix some_key, method get, time_spent 0.0, [Errno 32] Broken pipe test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache test WARNING: Item size larger than warning threshold: 9 (9) >= 1 (1) test ERROR: Error talking to memcached: 1.2.3.4:11211: with key_prefix too-big, method set, time_spent 0.0, failed set: SERVER_ERROR object too large for cache proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpuqljvb35/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpuqljvb35/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpuqljvb35/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpuqljvb35/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:24 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0148 "-" 3656210 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:24 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0115 "-" 3656210 - test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables test.unit.proxy.test_server.TestECGets.test_GET_with_missing_durables ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_frags_per_node ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_multiple_timestamps_on_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes test.unit.proxy.test_server.TestECGets.test_GET_with_same_frag_index_on_multiple_nodes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options test.unit.proxy.test_server.TestProxyServerLoading.test_concurrent_ec_options ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts test.unit.proxy.test_server.TestProxyServerLoading.test_float_timeouts ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings test.unit.proxy.test_server.TestProxyServerLoading.test_load_policy_rings ... ok test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings test.unit.proxy.test_server.TestProxyServerLoading.test_missing_rings ... ok REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '4749ceef93bef254112d929e9e3360c9', '68e2dc82-e72c-4701-8751-2fe30e106da4-sda', '1717347412.12323', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('68e2dc82-e72c-4701-8751-2fe30e106da4-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_bjbudfp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'db88738a-1dbf-4541-ad22-db191d042ca0-sda', '1717347414.92793', '1717347414', '0', '', '1717347414', 0, 1) test DEBUG: /tmp/tmp_bjbudfp/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'db88738a-1dbf-4541-ad22-db191d042ca0-sda', '1717347414.92793', '1717347414', '0', '', '1717347414', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('db88738a-1dbf-4541-ad22-db191d042ca0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04387 seconds (22.79533/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpswln3i9p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '8bfd79a336725ccac0bf66c8b928ea0c', '6fda4c94-8dee-48a2-b146-eb618eb358a0-sda', '1717347415.02628', '1717347415', '0', '', '1717347415', 1, 0) test DEBUG: /tmp/tmpswln3i9p/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '8bfd79a336725ccac0bf66c8b928ea0c', '6fda4c94-8dee-48a2-b146-eb618eb358a0-sda', '1717347415.02628', '1717347415', '0', '', '1717347415', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('6fda4c94-8dee-48a2-b146-eb618eb358a0-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpswln3i9p/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10287 seconds (9.72131/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpswln3i9p/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'd2d5aae25818f5e999dde7df32ac54d1', 'f7c27bc9-92e3-4f4a-affc-1c04cf98dc18-sda', '1717347415.14765', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('f7c27bc9-92e3-4f4a-affc-1c04cf98dc18-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'd2d5aae25818f5e999dde7df32ac54d1', 'f7c27bc9-92e3-4f4a-affc-1c04cf98dc18-sda', '1717347415.14765', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('f7c27bc9-92e3-4f4a-affc-1c04cf98dc18-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp5eko9vql/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '23870eef-7c63-46d9-9c80-1783b4df7adb-sda', '1717347417.87855', '1717347417', '1717347419', '', '1717347419', 0, 0) test DEBUG: /tmp/tmp5eko9vql/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '23870eef-7c63-46d9-9c80-1783b4df7adb-sda', '1717347417.87855', '1717347417', '1717347419', '', '1717347419', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('23870eef-7c63-46d9-9c80-1783b4df7adb-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04446 seconds (22.49120/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpyuzeuvqf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '148f0b753750de72a7ec739582e98f10', 'fba9180d-7be3-4393-85c4-383ae2503fc7-sda', '1717347417.97632', '1717347417', '1717347419', '', '1717347419', 1, 1) test DEBUG: /tmp/tmpyuzeuvqf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347422.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'fba9180d-7be3-4393-85c4-383ae2503fc7-sda') test DEBUG: /tmp/tmpyuzeuvqf/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0134158s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '148f0b753750de72a7ec739582e98f10', 'fba9180d-7be3-4393-85c4-383ae2503fc7-sda', '1717347417.97632', '1717347417', '1717347419', '', '1717347419', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('fba9180d-7be3-4393-85c4-383ae2503fc7-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpyuzeuvqf/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.10932 seconds (9.14734/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpyuzeuvqf/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'cf7655da2630b751a8df49d733583066', '0eb04e2f-7f0c-4feb-98ca-69f209181005-sda', '1717347418.10236', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0eb04e2f-7f0c-4feb-98ca-69f209181005-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'cf7655da2630b751a8df49d733583066', '0eb04e2f-7f0c-4feb-98ca-69f209181005-sda', '1717347418.10236', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('0eb04e2f-7f0c-4feb-98ca-69f209181005-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpzhmaeg53/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_delete ... ok test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1072 (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Getting listing part 3 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?end_marker=%E2%98%8A&format=json&limit=8928&marker=%D1%B0 {'X-Backend-Record-Type': 'auto'} (txn: tx79f3d1b098b541bc8172f-00665ca46a) test DEBUG: Found 6 objects in shard (state=unsharded), total = 1078 (txn: tx79f3d1b098b541bc8172f-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx12f3cc77bd71460ca6d96-00665ca46a) test DEBUG: GET listing from 5 shards for: /v1/a/c?prefix=hat&format=json&states=listing (txn: tx12f3cc77bd71460ca6d96-00665ca46a) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='ham', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx12f3cc77bd71460ca6d96-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_pie?prefix=hat&format=json&limit=10000&marker=&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx12f3cc77bd71460ca6d96-00665ca46a) test DEBUG: Found 0 objects in shard (state=unsharded), total = 0 (txn: tx12f3cc77bd71460ca6d96-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: tx93a9d02d4b35473a9f1dc-00665ca46a) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json&states=listing (txn: tx93a9d02d4b35473a9f1dc-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: tx93a9d02d4b35473a9f1dc-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%86&end_marker=%E2%98%8A&limit=2&format=json {'X-Backend-Record-Type': 'auto'} (txn: tx93a9d02d4b35473a9f1dc-00665ca46a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: tx93a9d02d4b35473a9f1dc-00665ca46a) test INFO: Caching listing namespaces for shard-listing-v2/a/c (5 namespaces) (txn: txec6afd00df6f4951b18fb-00665ca46a) test DEBUG: GET listing from 1 shards for: /v1/a/c?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json&states=listing (txn: txec6afd00df6f4951b18fb-00665ca46a) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_🌴', lower='☃', upper='🌴') .shards_a/c_🌴 with {'X-Backend-Record-Type': 'auto'} (txn: txec6afd00df6f4951b18fb-00665ca46a) test DEBUG: Get listing from /v1/.shards_a/c_%F0%9F%8C%B4?marker=%E2%98%8A&end_marker=%E2%98%86&limit=2&reverse=true&format=json {'X-Backend-Record-Type': 'auto'} (txn: txec6afd00df6f4951b18fb-00665ca46a) test DEBUG: Found 4 objects in shard (state=unsharded), total = 4 (txn: txec6afd00df6f4951b18fb-00665ca46a) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_b', lower='a', upper='b') .shards_a/c_b with {'X-Backend-Record-Type': 'auto'} (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Get listing from /v1/.shards_a/c_b?format=json&limit=10000&marker=&end_marker=b%00 {'X-Backend-Record-Type': 'auto'} (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 1 (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_c', lower='b', upper='c') .shards_a/c_c with {'X-Backend-Record-Type': 'auto'} (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Get listing from /v1/.shards_a/c_c?format=json&limit=9999&marker=b&end_marker=c%00 {'X-Backend-Record-Type': 'auto'} (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Found 1 objects in shard (state=unsharded), total = 2 (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='c', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9998&marker=c&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test WARNING: Failed to get container listing from /v1/a/c?format=json&states=listing: 404 (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test ERROR: Aborting listing from shards due to bad response: [200, 200, 404] (txn: tx01c5e2f7ca6b498ea0c7c-00665ca46d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c?format=json&states=listing (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Get listing from /v1/.shards_a/c_ham?format=json&limit=10000&marker=&end_marker=ham%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Found 40 objects in shard (state=unsharded), total = 40 (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Getting listing part 1 from shard Namespace(name='.shards_a/c_pie', lower='onion', upper='pie') .shards_a/c_pie with {'X-Backend-Record-Type': 'auto'} (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Get listing from /v1/.shards_a/c_pie?format=json&limit=9960&marker=h&end_marker=pie%00 {'X-Backend-Record-Type': 'auto'} (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Found 1 objects in shard (state=unsharded), total = 41 (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Getting listing part 2 from shard Namespace(name='.shards_a/c_', lower='rhubarb', upper='') .shards_a/c_ with {'X-Backend-Record-Type': 'auto'} (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Get listing from /v1/.shards_a/c_?format=json&limit=9959&marker=p&end_marker= {'X-Backend-Record-Type': 'auto'} (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Found 1024 objects in shard (state=unsharded), total = 1065 (txn: tx6468c0bda77a42b3a548a-00665ca46f) test DEBUG: Setting object count to 1065 (txn: tx6468c0bda77a42b3a548a-00665ca46f) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpexgrk1hl/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpexgrk1hl/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpexgrk1hl/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpexgrk1hl/tmp_test_proxy_server_chunked/container-sync-realms.conf' setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) setUpClass (test.unit.proxy.test_server.TestObjectECRangedGET) ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx788fa7e931f74273b9cf1-00665ca437) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 3, Lengths: [458, 458, 490, 458, 458, 458, 458, 458, 458, 458]: Invalid fragment payload in ECPyECLibDriver.decode (txn: tx4a4625f9b0ad479984f26-00665ca441) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx898d124195514c739e3b1-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx898d124195514c739e3b1-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx898d124195514c739e3b1-00665ca445) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 412 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 412 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from primary (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx12b2a84c64794dfc98c65-00665ca445) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_no_response_error ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_nondurable_when_node_iter_runs_out_of_nodes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_not_found_when_404_newer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_one_short_fragment_archive ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (1-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestReplicatedObjControllerVariousReplicas.test_write_affinity_proxy_server_config ... ok proxy-server DEBUG: Loaded override config for policy 1 (4-replica): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:24 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0100 "-" 3656210 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0146 "-" 3656210 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txfd822253b27f467ebcd2b-00665ca477" "proxy-server 3656210" 0.0036 "-" 3656210 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:46169/sdb1/1/a/c" "txfd822253b27f467ebcd2b-00665ca477" "container-server 3656210" 0.0016 "-" 3656210 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:40691/sda1/1/a/c" "txfd822253b27f467ebcd2b-00665ca477" "container-server 3656210" 0.0015 "-" 3656210 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txfd822253b27f467ebcd2b-00665ca477" "proxy-server 3656210" 0.0200 "-" 3656210 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txfd822253b27f467ebcd2b-00665ca477" "proxy-server 3656210" 0.0372 "-" 3656210 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/27 PUT /v1/a/c HTTP/1.0 201 - - t - - - txfd822253b27f467ebcd2b-00665ca477 - 0.0585 - - 1717347447.658512354 1717347447.717033625 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx3ece96aafc53407caf6c2-00665ca477" "proxy-server 3656210" 0.0021 "-" 3656210 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:46169/sdb1/1/a1/c1" "tx3ece96aafc53407caf6c2-00665ca477" "container-server 3656210" 0.0029 "-" 3656210 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:40691/sda1/1/a1/c1" "tx3ece96aafc53407caf6c2-00665ca477" "container-server 3656210" 0.0018 "-" 3656210 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx3ece96aafc53407caf6c2-00665ca477" "proxy-server 3656210" 0.0311 "-" 3656210 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx3ece96aafc53407caf6c2-00665ca477" "proxy-server 3656210" 0.0491 "-" 3656210 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/27 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx3ece96aafc53407caf6c2-00665ca477 - 0.0613 - - 1717347447.718927860 1717347447.780267239 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txeb6308fcd2e64072b4f3e-00665ca477" "proxy-server 3656210" 0.0146 "-" 3656210 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:40691/sda1/2/a/c1" "txeb6308fcd2e64072b4f3e-00665ca477" "container-server 3656210" 0.0069 "-" 3656210 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:46169/sdb1/2/a/c1" "txeb6308fcd2e64072b4f3e-00665ca477" "container-server 3656210" 0.0018 "-" 3656210 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txeb6308fcd2e64072b4f3e-00665ca477" "proxy-server 3656210" 0.0310 "-" 3656210 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txeb6308fcd2e64072b4f3e-00665ca477" "proxy-server 3656210" 0.0454 "-" 3656210 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/27 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txeb6308fcd2e64072b4f3e-00665ca477 - 0.0706 - - 1717347447.782862663 1717347447.853426933 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd8290b6703ee438386620-00665ca477" "proxy-server 3656210" 0.0110 "-" 3656210 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:40691/sda1/1/a/c2" "txd8290b6703ee438386620-00665ca477" "container-server 3656210" 0.0082 "-" 3656210 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:46169/sdb1/1/a/c2" "txd8290b6703ee438386620-00665ca477" "container-server 3656210" 0.0020 "-" 3656210 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd8290b6703ee438386620-00665ca477" "proxy-server 3656210" 0.0434 "-" 3656210 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:27 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd8290b6703ee438386620-00665ca477" "proxy-server 3656210" 0.0291 "-" 3656210 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/27 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txd8290b6703ee438386620-00665ca477 - 0.0654 - - 1717347447.855965853 1717347447.921339035 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': True, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 4}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.2, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347412.41289', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'X-Amz-Copy-Source': '/src_bucket/src_obj', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347412.43720', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_dst_container_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_copy_without_src_obj_permission ... ok cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0115 "-" 3656187 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0105 "-" 3656187 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0092 "-" 3656187 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0096 "-" 3656187 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx10b45fd1484f46e7a5d83-00665ca470" "proxy-server 3656187" 0.0016 "-" 3656187 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:44043/sdb1/1/a/c" "tx10b45fd1484f46e7a5d83-00665ca470" "container-server 3656187" 0.0016 "-" 3656187 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:40469/sda1/1/a/c" "tx10b45fd1484f46e7a5d83-00665ca470" "container-server 3656187" 0.0014 "-" 3656187 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx10b45fd1484f46e7a5d83-00665ca470" "proxy-server 3656187" 0.0179 "-" 3656187 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx10b45fd1484f46e7a5d83-00665ca470" "proxy-server 3656187" 0.0321 "-" 3656187 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/20 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx10b45fd1484f46e7a5d83-00665ca470 - 0.0426 - - 1717347440.060072660 1717347440.102682114 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "HEAD /sda1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx1e5aa84378784b58bf326-00665ca470" "proxy-server 3656187" 0.0016 "-" 3656187 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:40469/sda1/1/a1/c1" "tx1e5aa84378784b58bf326-00665ca470" "container-server 3656187" 0.0014 "-" 3656187 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:44043/sdb1/1/a1/c1" "tx1e5aa84378784b58bf326-00665ca470" "container-server 3656187" 0.0013 "-" 3656187 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1e5aa84378784b58bf326-00665ca470" "proxy-server 3656187" 0.0168 "-" 3656187 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx1e5aa84378784b58bf326-00665ca470" "proxy-server 3656187" 0.0295 "-" 3656187 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/20 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx1e5aa84378784b58bf326-00665ca470 - 0.0398 - - 1717347440.104632378 1717347440.144428730 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx8329ea1ee1134eb487692-00665ca470" "proxy-server 3656187" 0.0079 "-" 3656187 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:44043/sdb1/2/a/c1" "tx8329ea1ee1134eb487692-00665ca470" "container-server 3656187" 0.0111 "-" 3656187 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:40469/sda1/2/a/c1" "tx8329ea1ee1134eb487692-00665ca470" "container-server 3656187" 0.0016 "-" 3656187 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx8329ea1ee1134eb487692-00665ca470" "proxy-server 3656187" 0.0361 "-" 3656187 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx8329ea1ee1134eb487692-00665ca470" "proxy-server 3656187" 0.0488 "-" 3656187 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/20 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx8329ea1ee1134eb487692-00665ca470 - 0.0661 - - 1717347440.146328688 1717347440.212459564 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx3876a904c1e04811ae7bb-00665ca470" "proxy-server 3656187" 0.0092 "-" 3656187 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:40469/sda1/1/a/c2" "tx3876a904c1e04811ae7bb-00665ca470" "container-server 3656187" 0.0014 "-" 3656187 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:44043/sdb1/1/a/c2" "tx3876a904c1e04811ae7bb-00665ca470" "container-server 3656187" 0.0061 "-" 3656187 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3876a904c1e04811ae7bb-00665ca470" "proxy-server 3656187" 0.0216 "-" 3656187 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:20 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx3876a904c1e04811ae7bb-00665ca470" "proxy-server 3656187" 0.0366 "-" 3656187 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/20 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx3876a904c1e04811ae7bb-00665ca470 - 0.0552 - - 1717347440.214425802 1717347440.269664288 2 test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:48 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:52 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:56 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_with_read_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:54 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_error ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd6e16fcc-5f2e-4a98-85ad-a0e4c61a684b-sda', '1717347418.18180', '1717347418', '1717347420', '', '1717347420', 0, 1) test DEBUG: /tmp/tmpzhmaeg53/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'd6e16fcc-5f2e-4a98-85ad-a0e4c61a684b-sda', '1717347418.18180', '1717347418', '1717347420', '', '1717347420', 0, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('d6e16fcc-5f2e-4a98-85ad-a0e4c61a684b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.92098 seconds (0.34235/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpjlncnunq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e88acacb3adc3a9557a01d11662cb50e', '713d08cc-a781-409f-859b-fed2a82c2d6b-sda', '1717347421.15559', '1717347421', '1717347423', '', '1717347423', 1, 0) test DEBUG: /tmp/tmpjlncnunq/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'e88acacb3adc3a9557a01d11662cb50e', '713d08cc-a781-409f-859b-fed2a82c2d6b-sda', '1717347421.15559', '1717347421', '1717347423', '', '1717347423', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('713d08cc-a781-409f-859b-fed2a82c2d6b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpjlncnunq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.07282 seconds (13.73248/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpjlncnunq/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '4f1add523e72ae88773d9d8a4edf3ebd', 'd1d17292-108e-4d7c-b0b0-b6576d416fed-sda', '1717347421.24252', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d1d17292-108e-4d7c-b0b0-b6576d416fed-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '4f1add523e72ae88773d9d8a4edf3ebd', 'd1d17292-108e-4d7c-b0b0-b6576d416fed-sda', '1717347421.24252', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('d1d17292-108e-4d7c-b0b0-b6576d416fed-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpu4_wt4_7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '61565a39-7810-423a-afb7-d2517d9408f8-sda', '1717347423.84302', '1717347426', '1717347425', '', '1717347426', 0, 0) test DEBUG: /tmp/tmpu4_wt4_7/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '61565a39-7810-423a-afb7-d2517d9408f8-sda', '1717347423.84302', '1717347426', '1717347425', '', '1717347426', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('61565a39-7810-423a-afb7-d2517d9408f8-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04636 seconds (21.56838/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpz3t6nb3a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd855a57d9c6265592162a1dcc9a47395', 'b3977489-183f-4472-80a4-73f2aea3cb6b-sda', '1717347423.97422', '1717347426', '1717347425', '', '1717347426', 1, 1) test DEBUG: /tmp/tmpz3t6nb3a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': '/a/c/o', 'created_at': '1717347429.00000', 'size': 0, 'content_type': 'content-type', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 1}], 'b3977489-183f-4472-80a4-73f2aea3cb6b-sda') test DEBUG: /tmp/tmpz3t6nb3a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0183649s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, 'd855a57d9c6265592162a1dcc9a47395', 'b3977489-183f-4472-80a4-73f2aea3cb6b-sda', '1717347423.97422', '1717347426', '1717347425', '', '1717347426', 1, 1) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b3977489-183f-4472-80a4-73f2aea3cb6b-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmpz3t6nb3a/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.09215 seconds (10.85200/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:1 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmpz3t6nb3a/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8915871a45a559948305c55c7d64ad18', '7eeef043-4be7-4678-9cb3-cc5b37635f7c-sda', '1717347424.08997', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('7eeef043-4be7-4678-9cb3-cc5b37635f7c-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, '8915871a45a559948305c55c7d64ad18', '7eeef043-4be7-4678-9cb3-cc5b37635f7c-sda', '1717347424.08997', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('7eeef043-4be7-4678-9cb3-cc5b37635f7c-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp0vu_9n22/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '30bdbcda-eab2-4bfb-8fc9-43788ddece6f-sda', '1717347424.15606', '1717347427', '1717347426', '', '1717347427', 0, 0) test DEBUG: /tmp/tmp0vu_9n22/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '30bdbcda-eab2-4bfb-8fc9-43788ddece6f-sda', '1717347424.15606', '1717347427', '1717347426', '', '1717347427', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_recreate_policy_over_older_local_recreate ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings test.unit.container.test_replicator.TestReplicatorSync.test_sync_remote_with_timings ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:testertest:testerFULL_CONTROL test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:58 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_copy_without_src_obj_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.15108', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.15523', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.15801', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.16104', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.16448', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.16860', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.17165', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.17437', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.17779', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.19053', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347417.19345', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:56:57 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_quota_exceeded ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_upload_initiate_no_content_type ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:read:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_list_parts_acl_without_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:03 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347425.83259', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165705Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347425.85994', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_bad_hash ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:05 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.73305', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.73701', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.74108', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.74502', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.74912', 'Content-Type': 'binary/octet-stream'} test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1637, in get_response return self.get_acl_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1613, in get_acl_response resp = self._get_response( ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.75501', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.75958', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.76353', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.76718', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.78207', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347421.78576', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:01 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_quota_exceeded ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:04 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket GET {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_list_parts_error ... ok REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('30bdbcda-eab2-4bfb-8fc9-43788ddece6f-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 2.45228 seconds (0.40778/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmp_qou9grt/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '59a760e71f244a38ea9ff8e0eb8b41d0', 'f1528825-490a-4fbe-b189-5092546109c9-sda', '1717347426.66813', '1717347429', '1717347428', '', '1717347429', 1, 0) test DEBUG: /tmp/tmp_qou9grt/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '59a760e71f244a38ea9ff8e0eb8b41d0', 'f1528825-490a-4fbe-b189-5092546109c9-sda', '1717347426.66813', '1717347429', '1717347428', '', '1717347429', 1, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('f1528825-490a-4fbe-b189-5092546109c9-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test DEBUG: Adding 1 objects to the reconciler at /tmp/tmp_qou9grt/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.11329 seconds (8.82724/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Replicating 1 reconciler containers test DEBUG: Replicating db /tmp/tmp_qou9grt/sda/containers/0/16f/8f3afd7cc0a55bbf641992e992c0b16f/8f3afd7cc0a55bbf641992e992c0b16f.db REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'b13a02a579261d007ff26e1a740aacb3', '2dfde993-163b-46f9-82c8-cd57e0a0cad9-sda', '1717347426.81971', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdb/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('2dfde993-163b-46f9-82c8-cd57e0a0cad9-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, sync, (1, 'b13a02a579261d007ff26e1a740aacb3', '2dfde993-163b-46f9-82c8-cd57e0a0cad9-sda', '1717347426.81971', '1717344000', '0', '', '1717344000', 1, 0) REPLICATE: /sdc/0/8f3afd7cc0a55bbf641992e992c0b16f, complete_rsync, ('2dfde993-163b-46f9-82c8-cd57e0a0cad9-sda', '8f3afd7cc0a55bbf641992e992c0b16f.db') test INFO: Cleaning up 0 reconciler containers test INFO: Finished reconciler replication REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '59a760e71f244a38ea9ff8e0eb8b41d0', '4e5c3510-a16e-4e18-a19a-cf15fa8725b9-sda', '1717347430.92056', '1717347430.00000', '0', '{"x-container-meta-test": ["foo", "1717347430.00000"]}', '1717347430.00000', 1, 0) test DEBUG: replicator-rpc-sync time for info: 0.03s test DEBUG: replicator-rpc-sync time for update_metadata: 0.01s test DEBUG: replicator-rpc-sync time for merge_timestamps: 0.01s test DEBUG: replicator-rpc-sync time for get_sync: 0.00s test DEBUG: replicator-rpc-sync time for merge_syncs: 0.01s REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.10419', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.10419', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10419', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347431.10451', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.10456', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347431.10456', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10456', 'epoch': None, 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347431.14180', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0121539s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.10419', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.10419', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10419', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347431.10451', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.10456', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347431.10456', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10456', 'epoch': None, 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.10419', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.10419', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10419', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 13, 'bytes_used': 123, 'meta_timestamp': '1717347431.25158', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.10456', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347431.10456', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10456', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.10419', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.10419', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10419', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347431.28462', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.10456', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347431.10456', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10456', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.31988', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.31988', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347431.31988', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347431.28462', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.10456', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347431.10456', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10456', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.31988', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.31988', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347431.31988', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347431.28462', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.35540', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 30, 'meta_timestamp': '1717347431.35540', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.35540', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_error ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx3600541572514cc6b2f94-00665ca463) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx3600541572514cc6b2f94-00665ca463) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx3600541572514cc6b2f94-00665ca463) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx3600541572514cc6b2f94-00665ca463) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: tx54b1214ebed0434d9ba37-00665ca46c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: txaf3daa60340043deb9c3b-00665ca46c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: txcba6b8d3fecc49a08647f-00665ca46c) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx5443fa0856ea46cba328f-00665ca46c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: tx6d99b32a1daa4ea98a8a3-00665ca46c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx4a00b80bbaf44a59ae012-00665ca46c) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx954245fbdf7743fb9aeb3-00665ca46c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: tx25ff423eb0e34c33a8258-00665ca46c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: txa2f66db77bc74ae791b09-00665ca46c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx159f041ba11e461891f6d-00665ca46c) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd (txn: txfc30832678a8415fa312e-00665ca46c) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: tx570360aa713e4370b8bff-00665ca46c) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txfb48a08f8fef4f7e9b223-00665ca46d) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx841db8a25b874138a2581-00665ca46d) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: tx35d6e2c6abce4484a7c33-00665ca46d) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txef27e1d07dd9491d95d26-00665ca46d) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx12a65e87694f4baba05ab-00665ca46d) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx811b7df27098406d934e7-00665ca46d) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: txc319dd1f258e4450b9853-00665ca46d) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txefe5f542fdb441c3bee5d-00665ca46d) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx138d42b50fc847bf8813f-00665ca46d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx216ecc516b1d45928f4bb-00665ca46d) test INFO: Client disconnected on read of '/a/c' (txn: tx8260447a12964379a0270-00665ca46d) test INFO: Client disconnected on read of '/a/c' (txn: txaef9b46ad2fe4134a5fa0-00665ca46d) test INFO: Client disconnected on read of '/a/c' (txn: txaef9b46ad2fe4134a5fa0-00665ca46d) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_DELETE ... ok test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:07 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'X-Amz-Storage-Class': 'STANDARD', 'Content-Md5': 'XUFAKrxLKna5cZ2REBfFkg==', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '5d41402abc4b2a76b9719d911017c592', 'X-Timestamp': '1717347430.69969', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165710Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347430.72986', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:10 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_bad_hash ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:08 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165711Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347431.63953', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165711Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347431.66891', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_fullcontrol_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6', 'Content-Length': '0'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'Content-Md5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': 'd41d8cd98f00b204e9800998ecf8427e'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'Content-Md5': 'CY9rzUYh03PK3k6DJie09g==', 'Transfer-Encoding': 'chunked', 'User-Agent': 'Mozzarella Foxfire', 'Etag': '098f6bcd4621d373cade4e832627b4f6'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:13 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multi_upload_empty_body ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165716Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'not the hash', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347436.52849', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test:tester/20240602/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=hmac', 'X-Amz-Date': '20240602T165716Z', 'X-Amz-Storage-Class': 'STANDARD', 'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347436.55370', 'Content-Length': '5', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_fullcontrol_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:11 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:14 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347437.37470', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347437.39773', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:17 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_acl_without_permission ... ok test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:16 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347442.73756', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347442.75491', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:22 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_owner_permission ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_with_write_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:19 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_abort_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_error ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:23 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347445.83615', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347445.85364', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_object_PUT_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:27 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347450.81540', 'Content-Type': 'binary/octet-stream'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire', 'X-Timestamp': '1717347450.83777', 'Content-Type': 'binary/octet-stream'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_object_PUT_without_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:26 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_invalid_source ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347431.31988', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.31988', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347431.31988', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347431.28462', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.35540', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717347431.39569', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.35540', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347436.71613', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.31988', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.31988', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347431.28462', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.35540', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717347431.39569', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.35540', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpjgz9b_j8/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '475cb277e4bf00da22ecac8f2faaedc3', '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda', '1717347431.06949', '1717347431.05767', '0', '', '1717347431.05767', 1, 0) REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sda/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347436.71613', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347431.31988', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.31988', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347431.10451', 'lower': 'g', 'upper': 'r', 'object_count': 99, 'bytes_used': 0, 'meta_timestamp': '1717347431.28462', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.10451', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347431.35540', 'lower': 'r', 'upper': '', 'object_count': 0, 'bytes_used': 2030, 'meta_timestamp': '1717347431.39569', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347431.35540', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347431.24260', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347431.24260', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347431.24243', 'epoch': '1717347431.24243', 'reported': 0, 'tombstones': -1}], '4be22ad9-3b6f-4cc8-8f0d-1ec14f66d4c0-sda') test DEBUG: /tmp/tmpjgz9b_j8/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.0:1000/sda test WARNING: /tmp/tmpjgz9b_j8/sdb/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.0:1000/sda; have shard ranges and will wait for cleaving REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '71c4ec154cd246fb277ad596c6d7c8f3', '5eb1339a-40af-4906-8daa-4ad110984c7c-sda', '1717347438.17102', '1717347438.16580', '0', '', '1717347438.16580', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': 'a/c', 'timestamp': '1717347438.84513', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347438.84513', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347438.84493', 'epoch': '1717347438.84493', 'reported': 0, 'tombstones': -1}], '5eb1339a-40af-4906-8daa-4ad110984c7c-sda') test ERROR: ERROR Bad response 500 from 10.0.0.1 test DEBUG: /tmp/tmp23be4xz3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, starting at row -1 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347438.84457', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '5eb1339a-40af-4906-8daa-4ad110984c7c-sda') test DEBUG: /tmp/tmp23be4xz3/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db usyncing chunks to 10.0.0.1:1001/sdb, finished at row 1 (0.0122607s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'be02c37c-4f6b-4e9e-81d0-2db6ad474482-sda', '1717347439.58030', '1717347439.57477', '0', '', '1717347439.57477', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347439.60282', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347439.60282', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.60282', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347439.60291', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347439.60291', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.60291', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347439.60294', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347439.60294', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.60294', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'be02c37c-4f6b-4e9e-81d0-2db6ad474482-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 3 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '5cc56f5a-38c9-4e3b-9b8b-5dcdcf3f2445-sda', '1717347439.64355', '1717347439.64042', '0', '', '1717347439.64042', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347439.66150', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347439.66150', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.66150', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347439.66156', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347439.66156', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.66156', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347439.66159', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347439.66159', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.66159', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57412', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57412', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57412', 'epoch': None, 'reported': 0, 'tombstones': -1}], '5cc56f5a-38c9-4e3b-9b8b-5dcdcf3f2445-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'bf19b4ac-6f7d-428b-b321-6044b3441459-sda', '1717347439.70251', '1717347439.69954', '0', '', '1717347439.69954', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347439.71944', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347439.71944', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.71944', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347439.71950', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347439.71950', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.71950', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347439.71952', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347439.71952', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.71952', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57412', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57412', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57412', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'bf19b4ac-6f7d-428b-b321-6044b3441459-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'dd6fae91-0237-4131-99e3-ac668d5095ca-sda', '1717347439.75749', '1717347439.75440', '0', '', '1717347439.75440', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347439.77347', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347439.77347', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.77347', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347439.77352', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347439.77352', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.77352', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347439.77355', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347439.77355', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.77355', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57412', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57412', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57412', 'epoch': None, 'reported': 0, 'tombstones': -1}], 'dd6fae91-0237-4131-99e3-ac668d5095ca-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '841758d0-722c-4552-86e4-93446c222012-sda', '1717347439.81106', '1717347439.80798', '0', '', '1717347439.80798', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347439.82766', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347439.82766', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.82766', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347439.82771', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347439.82771', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.82771', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347439.82773', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347439.82773', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.82773', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57412', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57412', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57412', 'epoch': None, 'reported': 0, 'tombstones': -1}], '841758d0-722c-4552-86e4-93446c222012-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '5785dd65-b317-46f7-9afc-dda6407031c5-sda', '1717347439.86710', '1717347439.86388', '0', '', '1717347439.86388', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347439.88257', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347439.88257', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.88257', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347439.88262', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347439.88262', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.88262', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347439.88264', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347439.88264', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347439.88264', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57437', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57437', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57437', 'epoch': '1717347439.57441', 'reported': 0, 'tombstones': -1}], '5785dd65-b317-46f7-9afc-dda6407031c5-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'dd99c6a8-d005-41fd-80bd-736ba9f24a1f-sda', '1717347441.64079', '1717347441.63629', '0', '', '1717347441.63629', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347441.65995', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347441.65995', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.65995', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347441.66001', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347441.66001', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.66001', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347441.66004', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347441.66004', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.66004', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57437', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57437', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57437', 'epoch': '1717347439.57441', 'reported': 0, 'tombstones': -1}], 'dd99c6a8-d005-41fd-80bd-736ba9f24a1f-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '77e22f19-127c-4ee7-a060-36c38b6eda83-sda', '1717347441.73697', '1717347441.73169', '0', '', '1717347441.73169', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347441.76043', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347441.76043', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.76043', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347441.76050', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347441.76050', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.76050', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347441.76053', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347441.76053', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.76053', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57437', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57437', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57437', 'epoch': '1717347439.57441', 'reported': 0, 'tombstones': -1}], '77e22f19-127c-4ee7-a060-36c38b6eda83-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '7921d357-8c05-4ecd-be9e-c56b5ed1c51e-sda', '1717347441.82073', '1717347441.81753', '0', '', '1717347441.81753', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347441.83747', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347441.83747', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.83747', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347441.83753', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347441.83753', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.83753', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347441.83755', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347441.83755', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.83755', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57454', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57454', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57454', 'epoch': '1717347439.57457', 'reported': 0, 'tombstones': -1}], '7921d357-8c05-4ecd-be9e-c56b5ed1c51e-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '09801a03-2edc-4ade-96aa-a6208eafb349-sda', '1717347441.88489', '1717347441.88143', '0', '', '1717347441.88143', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_merge_remote_osr ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_none_to_sync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_timeout_in_fetch ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_trouble_receiving_so_none_to_sync ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '13'} test DEBUG: Start tag expected, '<' not found, line 1, column 1 (, line 1) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '221'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:25 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_complete_s3acl ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_upload_initiate_no_content_type ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_500_resp ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_content_type ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_over_twice_max_length ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_file_too_long ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_bad_path ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_call_and_content_type ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_fails ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_container_delete_retry_and_success ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_get_objs ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_max_failures ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_body ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_no_files_in_body ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_too_many_newlines ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_unauth ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_uses_predefined_object_errors ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_extra_newlines_extra_quoting ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_unicode ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_DELETE_verb ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_bulk_delete_works_with_POST_verb ... ok test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set test.unit.common.middleware.test_bulk.TestConcurrentDelete.test_concurrency_set ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_allow_reserved_names ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_duplicate_headers_not_removed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_location_header_fixed ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_ok_header ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_inbound ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_removed_outbound ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunt_bypassed_inbound ... ok test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound test.unit.common.middleware.test_gatekeeper.TestGatekeeper.test_reserved_header_shunted_inbound ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_add_to_response_headers ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_match_path ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_overwritten_by_response_header ... ok test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update test.unit.common.middleware.test_helpers.TestFakeSwiftStickyHeaders.test_sticky_headers_update ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_other_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_token_scoped_to_tenant ... ok test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token test.unit.common.middleware.test_keystoneauth.TestIsNameAllowedInACL.test_is_name_allowed_in_acl_with_unscoped_token ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags test.unit.common.middleware.test_slo.TestSloHeadManifest.test_etag_is_hash_of_segment_etags ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough test.unit.common.middleware.test_slo.TestSloHeadManifest.test_get_manifest_passthrough ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient test.unit.common.middleware.test_slo.TestSloHeadManifest.test_head_manifest_is_efficient ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_match_etag_not_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override test.unit.common.middleware.test_slo.TestSloHeadManifest.test_if_none_match_etag_matching_with_override ... ok test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest test.unit.common.middleware.test_slo.TestSloHeadManifest.test_zero_byte_manifest ... ok test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_with_POST ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:33 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObjNonUTCAcl.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_controller_allow_multipart_uploads ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_header_authorization ... ok test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization test.unit.common.middleware.s3api.test_s3request.TestSigV4Request.test_init_query_authorization ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_exception ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_ioerror ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_owner_permission ... ok test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (204, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_POST ... ok proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to GET /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2629, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server ERROR: Object returning 503 for [] (txn: txa2b53155c2de4ec490201-00665ca445) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Node is error limited: 10.0.0.12:1012/sdm (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.13:1013/sdn (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.14:1014/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.15:1015/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.16:1016/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.17:1017/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.18:1018/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.19:1019/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.20:1020/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.21:1021/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.22:1022/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.23:1023/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.24:1024/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.25:1025/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.26:1026/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Node is error limited: 10.0.0.27:1027/sda (txn: txdddccaacda8242319ba58-00665ca452) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txf271ada0bf534de2a36c0-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx4d068873f8d642dbaefbd-00665ca455) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx61da3ac53f92432ba328e-00665ca455) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txa8948605542644d092168-00665ca455) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: txf227999bcfce4b50aaa9a-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: Object returning 503 for [] (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_primaries_error_during_rebalance ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_simple_x_newest ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_timeout ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347441.90404', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347441.90404', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.90404', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347441.90411', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347441.90411', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.90411', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347441.90413', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347441.90413', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347441.90413', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57437', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57437', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57437', 'epoch': '1717347439.57441', 'reported': 0, 'tombstones': -1}], '09801a03-2edc-4ade-96aa-a6208eafb349-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '0fc6e1c4-7f8b-475b-a47a-bf2650506917-sda', '1717347447.73461', '1717347447.73157', '0', '', '1717347447.73157', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347447.76600', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347447.76600', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347447.76600', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347447.76605', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347447.76605', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347447.76605', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347447.76608', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347447.76608', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347447.76608', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57437', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57437', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57437', 'epoch': '1717347439.57441', 'reported': 0, 'tombstones': -1}], '0fc6e1c4-7f8b-475b-a47a-bf2650506917-sda') test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', '381ae20d-caa4-46ba-92f4-2c71ebe9a008-sda', '1717347447.82176', '1717347447.81865', '0', '', '1717347447.81865', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347447.84499', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347447.84499', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347447.84499', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347447.84504', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347447.84504', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347447.84504', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347447.84506', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347447.84506', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347447.84506', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347439.57469', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57412', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57412', 'epoch': None, 'reported': 0, 'tombstones': -1}], '381ae20d-caa4-46ba-92f4-2c71ebe9a008-sda') test WARNING: Ignoring remote osr w/o epoch, own_sr: {'name': 'a/c', 'timestamp': '1717347439.57437', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57437', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57437', 'epoch': '1717347439.57441', 'reported': 0, 'tombstones': -1}, remote_sr: {'name': 'a/c', 'timestamp': '1717347439.57469', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347439.57412', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347439.57412', 'epoch': None, 'reported': 0, 'tombstones': -1}, source: repl_req test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test DEBUG: /tmp/tmpirvoj96e/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '1468ded83a60f919286a16b64082f736', '4268b422-6ea4-4617-b6e5-3a3bac0b41de-sda', '1717347447.90109', '1717347447.89617', '0', '', '1717347447.89617', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347447.92043', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '4268b422-6ea4-4617-b6e5-3a3bac0b41de-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test DEBUG: Replicating db /tmp/tmpsa32c15a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'a1d81cba-4d89-41d8-9050-235039881f72-sda', '1717347447.98213', '1717347447.97871', '0', '', '1717347447.97871', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () test ERROR: ERROR syncing /tmp/tmpsa32c15a/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db with node {'ip': '10.0.0.1', 'replication_ip': '10.0.0.1', 'port': 1001, 'replication_port': 1001, 'device': 'sdb', 'zone': 1, 'region': 1, 'id': 1, 'weight': 1, 'index': 1}: Timeout (0.001s) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (1, '35058e057c0e8f293054af2843ef9f1e', '1ccdf12f-d5a8-4cdc-8590-2ce899851cf8-sda', '1717347448.02768', '1717347448.02403', '0', '', '1717347448.02403', 1, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_items, ([{'ROWID': 1, 'name': 'obj', 'created_at': '1717347448.05494', 'size': 0, 'content_type': 'text/plain', 'etag': 'd41d8cd98f00b204e9800998ecf8427e', 'deleted': 0, 'storage_policy_index': 0}], '1ccdf12f-d5a8-4cdc-8590-2ce899851cf8-sda') REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_syncs, ([],) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwn4mgr9i/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync test.unit.container.test_replicator.TestReplicatorSync.test_sync_shard_ranges_with_rsync ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change test.unit.container.test_replicator.TestReplicatorSync.test_sync_status_change ... ok test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced test.unit.container.test_replicator.TestReplicatorSync.test_sync_to_remote_with_misplaced ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'X-Amz-Acl': 'public-read', 'X-Amz-Meta-Foo': 'bar', 'User-Agent': 'Mozzarella Foxfire', 'X-Object-Sysmeta-S3Api-Has-Content-Type': 'no', 'Content-Type': 'application/directory'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:28 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:30 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_multipart_uploads_list ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test ERROR: Error retrieving recon data: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 228, in raise_Exception raise Exception Exception test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test ERROR: Error reading recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: [Errno None] None test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror test.unit.common.middleware.test_recon.TestReconSuccess.test_from_recon_cache_valueerror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_async_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_account ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_object_parallel_once ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_auditor_info_unrecognized ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror test.unit.common.middleware.test_recon.TestReconSuccess.test_get_diskusage_oserror ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_driveaudit_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_expirer_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load test.unit.common.middleware.test_recon.TestReconSuccess.test_get_load ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mem ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count test.unit.common.middleware.test_recon.TestReconSuccess.test_get_quarantine_count ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction test.unit.common.middleware.test_recon.TestReconSuccess.test_get_reconstruction ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_relinker_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_account ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_info_unrecognized ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_replication_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5 ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_configured_missing_obj_rings ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_excludes_unconfigured_present_obj_rings ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_failed_ring_hash_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_ioerror_produces_none_hash ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart test.unit.common.middleware.test_recon.TestReconSuccess.test_get_ring_md5_missing_ring_recovers_without_restart ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_sharding_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info test.unit.common.middleware.test_recon.TestReconSuccess.test_get_socket_info ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time test.unit.common.middleware.test_recon.TestReconSuccess.test_get_time ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_all_mounted ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_checkmount_fail ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_excludes_files ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts test.unit.common.middleware.test_recon.TestReconSuccess.test_get_unmounted_no_mounts ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_container ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_object ... ok test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized test.unit.common.middleware.test_recon.TestReconSuccess.test_get_updater_info_unrecognized ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_extra_key ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_simple_etag ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_extract_symlink_path_json_symlink_path_and_account ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_error_cases ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_simple_with_listing_format_xml ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_get_container_with_subdir ... ok test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request test.unit.common.middleware.test_symlink.TestSymlinkContainerContext.test_no_affect_for_account_request ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_options ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_incoming_header_startswith_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict test.unit.common.middleware.test_tempurl.TestTempURL.test_allow_trumps_outgoing_header_startswith_conflict ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value test.unit.common.middleware.test_tempurl.TestTempURL.test_auth_scheme_value ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope test.unit.common.middleware.test_tempurl.TestTempURL.test_authorize_limits_scope ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_ip_range_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path test.unit.common.middleware.test_tempurl.TestTempURL.test_bad_path ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_expires_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_path_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_changed_sig_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_incoming_headers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers test.unit.common.middleware.test_tempurl.TestTempURL.test_clean_outgoing_headers ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_allowed ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_delete_when_forbidden_by_config ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_different_key_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest test.unit.common.middleware.test_tempurl.TestTempURL.test_disallowed_header_object_manifest ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_filename_trailing_slash ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs test.unit.common.middleware.test_tempurl.TestTempURL.test_get_hmacs ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_get_not_allowed_by_put ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts test.unit.common.middleware.test_tempurl.TestTempURL.test_get_path_parts ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info test.unit.common.middleware.test_tempurl.TestTempURL.test_get_temp_url_info ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_but_404 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_container_keys ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_key2 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_fake_ip_from_x_forwarded_for ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_filename_and_inline ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_inline ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_from_remote_addr ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ip_range ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_ipv6_range ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_no_client_address ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_and_staticweb ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_prefix_empty ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 test.unit.common.middleware.test_tempurl.TestTempURL.test_get_valid_with_single_ipv6 ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_get ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_post ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put test.unit.common.middleware.test_tempurl.TestTempURL.test_head_allowed_by_put ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_head_and_get_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_head_otherwise_not_allowed ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename test.unit.common.middleware.test_tempurl.TestTempURL.test_head_valid_with_filename ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error test.unit.common.middleware.test_tempurl.TestTempURL.test_ip_range_value_error ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_expires ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig test.unit.common.middleware.test_tempurl.TestTempURL.test_missing_sig ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key test.unit.common.middleware.test_tempurl.TestTempURL.test_no_key ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_no_prefix_match_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires test.unit.common.middleware.test_tempurl.TestTempURL.test_normalize_temp_url_expires ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_odd_chars_in_content_disposition_metadata ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash test.unit.common.middleware.test_tempurl.TestTempURL.test_obj_trailing_slash ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid test.unit.common.middleware.test_tempurl.TestTempURL.test_object_url_with_prefix_invalid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough test.unit.common.middleware.test_tempurl.TestTempURL.test_passthrough ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config test.unit.common.middleware.test_tempurl.TestTempURL.test_post_when_forbidden_by_config ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get test.unit.common.middleware.test_tempurl.TestTempURL.test_put_not_allowed_by_get ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list test.unit.common.middleware.test_tempurl.TestTempURL.test_put_response_headers_in_list ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid test.unit.common.middleware.test_tempurl.TestTempURL.test_put_valid ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_header ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_incoming_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_header ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match test.unit.common.middleware.test_tempurl.TestTempURL.test_removed_outgoing_headers_match ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim test.unit.common.middleware.test_tempurl.TestTempURL.test_signature_trim ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value test.unit.common.middleware.test_tempurl.TestTempURL.test_unicode_metadata_value ... ok test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed test.unit.common.middleware.test_tempurl.TestTempURL.test_unknown_not_allowed ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_multiple_tier_dispersion ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_rebalance_dispersion ... ok test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion test.unit.common.ring.test_builder.TestRingBuilderDispersion.test_weight_dispersion ... ok test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval test.unit.common.test_container_sync_realms.TestUtils.test_bad_mtime_check_interval ... ok test.unit.common.test_container_sync_realms.TestUtils.test_empty test.unit.common.test_container_sync_realms.TestUtils.test_empty ... ok test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm test.unit.common.test_container_sync_realms.TestUtils.test_empty_realm ... ok test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing test.unit.common.test_container_sync_realms.TestUtils.test_error_parsing ... ok test.unit.common.test_container_sync_realms.TestUtils.test_get_sig test.unit.common.test_container_sync_realms.TestUtils.test_get_sig ... ok test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there test.unit.common.test_container_sync_realms.TestUtils.test_no_file_there ... ok test.unit.common.test_container_sync_realms.TestUtils.test_one_realm test.unit.common.test_container_sync_realms.TestUtils.test_one_realm ... ok test.unit.common.test_container_sync_realms.TestUtils.test_os_error test.unit.common.test_container_sync_realms.TestUtils.test_os_error ... ok test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default test.unit.common.test_container_sync_realms.TestUtils.test_two_realms_and_change_a_default ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_exactly_two_batches ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_one_large_batch ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches test.unit.common.test_db.TestTombstoneReclaimer.test_batched_reclaim_several_small_batches ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_get_tombstone_count_with_leftover ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count ... ok test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover test.unit.common.test_db.TestTombstoneReclaimer.test_reclaim_get_tombstone_count_with_leftover ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_300_status ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_already_in_sync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_drive_not_mounted ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_metadata_update ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_found ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_not_response ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_rsync_success ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_small_container_always_usync ... ok test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success test.unit.common.test_db_replicator.TestReplToNode.test_repl_to_node_usync_success ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_failure ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_account_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_obj_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_container_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_delete_object_with_timestamp ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_account_no_content_does_not_parse_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_no_content_does_not_decode_body ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_container_with_extra_params ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_chunks ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_recon ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_503 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_get_suffix_hashes_507 ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_deleted ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_container_replication_net ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_head_object_not_found ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_post_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_container_object_error ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_args ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_chunked ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_fail ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_header_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length test.unit.common.test_direct_client.TestUTF8DirectClient.test_direct_put_object_with_content_length ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers test.unit.common.test_direct_client.TestUTF8DirectClient.test_gen_headers ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_client_exception ... ok test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception test.unit.common.test_direct_client.TestUTF8DirectClient.test_retry_http_exception ... ok test.unit.common.test_manager.TestServer.test_conf_dir test.unit.common.test_manager.TestServer.test_conf_dir ... ok test.unit.common.test_manager.TestServer.test_conf_files test.unit.common.test_manager.TestServer.test_conf_files ... ok test.unit.common.test_manager.TestServer.test_create_server test.unit.common.test_manager.TestServer.test_create_server ... ok test.unit.common.test_manager.TestServer.test_expirer_conf_files test.unit.common.test_manager.TestServer.test_expirer_conf_files ... ok test.unit.common.test_manager.TestServer.test_get_conf_file_name test.unit.common.test_manager.TestServer.test_get_conf_file_name ... ok test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name test.unit.common.test_manager.TestServer.test_get_custom_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_pid_file_name test.unit.common.test_manager.TestServer.test_get_pid_file_name ... ok test.unit.common.test_manager.TestServer.test_get_running_pids test.unit.common.test_manager.TestServer.test_get_running_pids ... ok test.unit.common.test_manager.TestServer.test_interact test.unit.common.test_manager.TestServer.test_interact ... ok test.unit.common.test_manager.TestServer.test_iter_pid_files test.unit.common.test_manager.TestServer.test_iter_pid_files ... ok test.unit.common.test_manager.TestServer.test_kill_running_pids test.unit.common.test_manager.TestServer.test_kill_running_pids ... ok test.unit.common.test_manager.TestServer.test_launch test.unit.common.test_manager.TestServer.test_launch ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir test.unit.common.test_manager.TestServer.test_named_conf_dir ... ok test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files test.unit.common.test_manager.TestServer.test_named_conf_dir_pid_files ... ok test.unit.common.test_manager.TestServer.test_proxy_conf_dir test.unit.common.test_manager.TestServer.test_proxy_conf_dir ... ok test.unit.common.test_manager.TestServer.test_server_equality test.unit.common.test_manager.TestServer.test_server_equality ... ok test.unit.common.test_manager.TestServer.test_server_repr test.unit.common.test_manager.TestServer.test_server_repr ... ok test.unit.common.test_manager.TestServer.test_server_to_string test.unit.common.test_manager.TestServer.test_server_to_string ... ok test.unit.common.test_manager.TestServer.test_signal_pids test.unit.common.test_manager.TestServer.test_signal_pids ... ok test.unit.common.test_manager.TestServer.test_spawn test.unit.common.test_manager.TestServer.test_spawn ... ok test.unit.common.test_manager.TestServer.test_status test.unit.common.test_manager.TestServer.test_status ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object DELETE {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:35 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test DEBUG: Names allowed in acls. test INFO: - - 02/Jun/2024/16/57/37 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347457.687731028 1717347457.688053608 0 test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0013 - - 1717347457.687653542 1717347457.688977957 0 test INFO: - - 02/Jun/2024/16/57/37 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717347457.691320658 1717347457.691707850 0 test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0013 - - 1717347457.691249371 1717347457.692559958 0 test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347457.694826603 1717347457.695143461 0 test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0014 - - 1717347457.694694042 1717347457.696053267 0 test INFO: - - 02/Jun/2024/16/57/37 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717347457.698261499 1717347457.698621511 0 test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0019 - - 1717347457.698189259 1717347457.700106144 0 test INFO: - - 02/Jun/2024/16/57/37 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347457.702328920 1717347457.702610254 0 test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0013 - - 1717347457.702258348 1717347457.703512669 0 test INFO: - - 02/Jun/2024/16/57/37 HEAD /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347457.706272364 1717347457.706568241 0 test INFO: - - 02/Jun/2024/16/57/37 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0004 SSC - 1717347457.707182646 1717347457.707588434 0 test INFO: - - 02/Jun/2024/16/57/37 HEAD /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347457.708367825 1717347457.708639145 0 test INFO: - - 02/Jun/2024/16/57/37 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0035 - - 1717347457.706017733 1717347457.709514856 - test INFO: - - 02/Jun/2024/16/57/37 GET /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717347457.711999893 1717347457.712372065 0 test INFO: - - 02/Jun/2024/16/57/37 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0004 SSC - 1717347457.712907314 1717347457.713299036 0 test INFO: - - 02/Jun/2024/16/57/37 GET /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717347457.714039326 1717347457.714391708 0 test INFO: - - 02/Jun/2024/16/57/37 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0036 - - 1717347457.711756229 1717347457.715376854 - test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from test.unit.common.middleware.test_subrequest_logging.TestSubRequestLogging.test_subrequest_logged_x_copy_from ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens test.unit.common.middleware.test_tempauth.TestTokenHandling.test_invalid_tokens ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_given_and_needed ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted test.unit.common.middleware.test_tempauth.TestTokenHandling.test_service_token_omitted ... ok test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user test.unit.common.middleware.test_tempauth.TestTokenHandling.test_tokens_set_remote_user ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_bad_config ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_no_deprecated_digests ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_non_default_methods ... ok test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_tempurl.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set test.unit.common.middleware.test_versioned_writes.TestSwiftInfo.test_registered_explicitly_set ... ok test.unit.common.test_db.TestChexor.test_chexor test.unit.common.test_db.TestChexor.test_chexor ... ok test.unit.common.test_db.TestChexor.test_invalid_old_hash test.unit.common.test_db.TestChexor.test_invalid_old_hash ... ok test.unit.common.test_db.TestChexor.test_no_name test.unit.common.test_db.TestChexor.test_no_name ... ok test.unit.common.test_db.TestChexor.test_normal_case test.unit.common.test_db.TestChexor.test_normal_case ... ok test.unit.common.test_db.TestHelperFunctions.test_zero_like test.unit.common.test_db.TestHelperFunctions.test_zero_like ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_headers ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params test.unit.common.test_registry.TestSensitiveRegistry.test_get_sensitive_params ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_header ... ok test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param test.unit.common.test_registry.TestSensitiveRegistry.test_register_sensitive_param ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_200 ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_multiple_ranges ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_206_single_range ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_multiple_resolove_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolove_ignore_range_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_resolve_etag_is_at_header ... ok test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header test.unit.common.test_request_helpers.TestHTTPResponseToDocumentIters.test_update_etag_is_at_header ... ok test.unit.common.test_swob.TestRange.test_range test.unit.common.test_swob.TestRange.test_range ... ok test.unit.common.test_swob.TestRange.test_range_invalid_syntax test.unit.common.test_swob.TestRange.test_range_invalid_syntax ... ok test.unit.common.test_swob.TestRange.test_ranges_for_large_length test.unit.common.test_swob.TestRange.test_ranges_for_large_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length test.unit.common.test_swob.TestRange.test_ranges_for_length ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_edges test.unit.common.test_swob.TestRange.test_ranges_for_length_edges ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_multi test.unit.common.test_swob.TestRange.test_ranges_for_length_multi ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end test.unit.common.test_swob.TestRange.test_ranges_for_length_no_end ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start test.unit.common.test_swob.TestRange.test_ranges_for_length_no_start ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending test.unit.common.test_swob.TestRange.test_ranges_for_length_nonascending ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping test.unit.common.test_swob.TestRange.test_ranges_for_length_overlapping ... ok test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many test.unit.common.test_swob.TestRange.test_ranges_for_length_too_many ... ok test.unit.common.test_swob.TestRange.test_str test.unit.common.test_swob.TestRange.test_str ... ok test.unit.common.test_swob.TestRange.test_upsidedown_range test.unit.common.test_swob.TestRange.test_upsidedown_range ... ok test.unit.container.test_server.TestContainerController.test_DELETE test.unit.container.test_server.TestContainerController.test_DELETE ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate ... ok test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race test.unit.container.test_server.TestContainerController.test_DELETE_PUT_recreate_replication_race ... ok test.unit.container.test_server.TestContainerController.test_DELETE_account_update test.unit.container.test_server.TestContainerController.test_DELETE_account_update ... ok test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found test.unit.container.test_server.TestContainerController.test_DELETE_container_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition test.unit.container.test_server.TestContainerController.test_DELETE_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_DELETE_not_found test.unit.container.test_server.TestContainerController.test_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found test.unit.container.test_server.TestContainerController.test_DELETE_obj_not_found ... ok test.unit.container.test_server.TestContainerController.test_DELETE_object test.unit.container.test_server.TestContainerController.test_DELETE_object ... ok proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to GET /v1/a/c/o: Timeout (Nones) (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Ignoring 404 from primary (txn: tx47e1a47d494b4cbbb9849-00665ca455) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: tx174b80c2385c47b59f614-00665ca478) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to GET /v1/a/c/o: Timeout (0.01s) (txn: tx174b80c2385c47b59f614-00665ca478) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Error decoding fragments for '/a/c/o'. Segments decoded: 0, Lengths: [490, 490, 490, 490, 490, 490, 490, 490, 490, 490]: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx156985ff2023488080356-00665ca478) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_trigger_ec_metadata_check_failure ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_concurrent_ec_extra_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_and_hidden_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_duplicate_but_insufficient_frag_indexes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_frags_swapped_around ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_invalid_ranges ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_many_missed_overwrite_will_need_handoff ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:31 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:34 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_permission ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aa99217d-89f8-4823-9605-654cc459fee7-sda', '1717347448.11095', '1717347448.10743', '0', '', '1717347448.10743', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('aa99217d-89f8-4823-9605-654cc459fee7-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aa99217d-89f8-4823-9605-654cc459fee7-sda', '1717347448.11095', '1717347448.10743', '0', '', '1717347448.10743', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('aa99217d-89f8-4823-9605-654cc459fee7-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 4.96702 seconds (0.20133/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:2 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpwn4mgr9i/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aa99217d-89f8-4823-9605-654cc459fee7-sda', '1717347448.11095', '1717347448.10743', '0', '', '1717347448.10743', 0, 0) REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347453.13628', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347453.13628', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347453.13628', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347448.14232', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347448.14232', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347448.14232', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347448.14235', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347448.14235', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347448.14235', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347448.14257', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347448.14257', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347448.14238', 'epoch': '1717347448.14238', 'reported': 0, 'tombstones': -1}], 'aa99217d-89f8-4823-9605-654cc459fee7-sda') test DEBUG: /tmp/tmpwn4mgr9i/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.1:1001/sdb test WARNING: /tmp/tmpwn4mgr9i/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.1:1001/sdb; have shard ranges and will wait for cleaving REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'aa99217d-89f8-4823-9605-654cc459fee7-sda', '1717347448.11095', '1717347448.10743', '0', '', '1717347448.10743', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, get_shard_ranges, () REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, merge_shard_ranges, ([{'name': '.shards_a/sr-g', 'timestamp': '1717347453.13628', 'lower': '', 'upper': 'g', 'object_count': 1, 'bytes_used': 10, 'meta_timestamp': '1717347453.13628', 'deleted': 1, 'state': 10, 'state_timestamp': '1717347453.13628', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-r', 'timestamp': '1717347448.14232', 'lower': 'g', 'upper': 'r', 'object_count': 2, 'bytes_used': 20, 'meta_timestamp': '1717347448.14232', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347448.14232', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/sr-', 'timestamp': '1717347448.14235', 'lower': 'r', 'upper': '', 'object_count': 3, 'bytes_used': 30, 'meta_timestamp': '1717347448.14235', 'deleted': 0, 'state': 10, 'state_timestamp': '1717347448.14235', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347448.14257', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347448.14257', 'deleted': 0, 'state': 60, 'state_timestamp': '1717347448.14238', 'epoch': '1717347448.14238', 'reported': 0, 'tombstones': -1}], 'aa99217d-89f8-4823-9605-654cc459fee7-sda') test DEBUG: /tmp/tmpwn4mgr9i/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db synced 4 shard ranges to 10.0.0.2:1002/sdc test WARNING: /tmp/tmpwn4mgr9i/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db is able to shard -- refusing to replicate objects to peer 10.0.0.2:1002/sdc; have shard ranges and will wait for cleaving test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.04309 seconds (23.20597/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0 REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e35d08f5-6b69-4216-b87c-780d74990635-sda', '1717347453.21000', '1717347453.20649', '1717347453.23367', '', '1717347453.23367', 0, 0) test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmpq7xr2yzu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b7a48410-1cc4-4d41-aca9-e725abacb63d-sda', '1717347453.26613', '1717347453.00000', '0', '', '1717347453.00000', 0, 0) test DEBUG: /tmp/tmpq7xr2yzu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'b7a48410-1cc4-4d41-aca9-e725abacb63d-sda', '1717347453.26613', '1717347453.00000', '0', '', '1717347453.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('b7a48410-1cc4-4d41-aca9-e725abacb63d-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 1 dbs in 0.05460 seconds (18.31482/s) test INFO: Removed 0 dbs test INFO: 2 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:1 remote_merge:0 rsync:1 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptn9nhpsu/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '7ce50ea4-4122-43da-bc9e-ccd470f4cf21-sda', '1717347453.37444', '1717347454.00000', '0', '{"X-Container-Sync-To": ["a", "1717347455.00000"], "X-Container-Sync-Key": ["secret", "1717347456.00000"]}', '1717347454.00000', 0, 0) REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('7ce50ea4-4122-43da-bc9e-ccd470f4cf21-sda', 'c48d89b33840539f9c5442ab3b10027d.db') REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '7ce50ea4-4122-43da-bc9e-ccd470f4cf21-sda', '1717347453.37444', '1717347454.00000', '0', '{"X-Container-Sync-To": ["a", "1717347455.00000"], "X-Container-Sync-Key": ["secret", "1717347456.00000"]}', '1717347454.00000', 0, 0) REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, complete_rsync, ('7ce50ea4-4122-43da-bc9e-ccd470f4cf21-sda', 'c48d89b33840539f9c5442ab3b10027d.db') test DEBUG: Replicating db /tmp/tmptn9nhpsu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e5be1365-76ad-45dd-afd7-d10bb9b4cb9c-sda', '1717347453.36250', '1717347453.00000', '0', '', '1717347453.00000', 0, 0) test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update test.unit.container.test_replicator.TestReplicatorSync.test_sync_triggers_sync_store_update ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store ... ok test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception test.unit.container.test_replicator.TestReplicatorSync.test_update_sync_store_exception ... ok test.unit.obj.test_diskfile.TestDiskFile.test_close_error test.unit.obj.test_diskfile.TestDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit test.unit.obj.test_diskfile.TestDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_delete test.unit.obj.test_diskfile.TestDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_not_opened test.unit.obj.test_diskfile.TestDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE test.unit.obj.test_server.TestObjectController.test_DELETE ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp test.unit.obj.test_server.TestObjectController.test_DELETE_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired test.unit.obj.test_server.TestObjectController.test_DELETE_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_DELETE_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates test.unit.obj.test_server.TestObjectController.test_DELETE_container_updates ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes test.unit.obj.test_server.TestObjectController.test_DELETE_if_delete_at_expired_still_deletes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_DELETE_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive test.unit.obj.test_server.TestObjectController.test_EC_GET_quarantine_invalid_frag_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit test.unit.obj.test_server.TestObjectController.test_EC_PUT_GET_data_no_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET test.unit.obj.test_server.TestObjectController.test_GET ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences test.unit.obj.test_server.TestObjectController.test_GET_HEAD_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_but_expired test.unit.obj.test_server.TestObjectController.test_GET_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config test.unit.obj.test_server.TestObjectController.test_GET_cooperative_period_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match test.unit.obj.test_server.TestObjectController.test_GET_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at test.unit.obj.test_server.TestObjectController.test_GET_if_match_etag_is_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since test.unit.obj.test_server.TestObjectController.test_GET_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_none_match test.unit.obj.test_server.TestObjectController.test_GET_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_GET_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_private_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_config_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_no_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo test.unit.obj.test_server.TestObjectController.test_GET_keep_cache_slo_manifest_not_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine test.unit.obj.test_server.TestObjectController.test_GET_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range test.unit.obj.test_server.TestObjectController.test_GET_quarantine_range ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_GET_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable test.unit.obj.test_server.TestObjectController.test_GET_range_not_satisfiable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object test.unit.obj.test_server.TestObjectController.test_GET_range_zero_byte_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD test.unit.obj.test_server.TestObjectController.test_HEAD ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired test.unit.obj.test_server.TestObjectController.test_HEAD_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_modified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match test.unit.obj.test_server.TestObjectController.test_HEAD_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since test.unit.obj.test_server.TestObjectController.test_HEAD_if_unmodified_since ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_HEAD_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_OPTIONS test.unit.obj.test_server.TestObjectController.test_OPTIONS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp test.unit.obj.test_server.TestObjectController.test_POST_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_but_expired test.unit.obj.test_server.TestObjectController.test_POST_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at test.unit.obj.test_server.TestObjectController.test_POST_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_POST_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST test.unit.obj.test_server.TestObjectController.test_POST_conflicts_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_connection test.unit.obj.test_server.TestObjectController.test_POST_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence test.unit.obj.test_server.TestObjectController.test_POST_container_updates_precedence ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy test.unit.obj.test_server.TestObjectController.test_POST_container_updates_with_replication_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock test.unit.obj.test_server.TestObjectController.test_POST_delete_at_in_past_with_skewed_clock ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_invalid_path test.unit.obj.test_server.TestObjectController.test_POST_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp test.unit.obj.test_server.TestObjectController.test_POST_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_not_exist test.unit.obj.test_server.TestObjectController.test_POST_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp test.unit.obj.test_server.TestObjectController.test_POST_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte test.unit.obj.test_server.TestObjectController.test_POST_quarantine_zbyte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_system_metadata test.unit.obj.test_server.TestObjectController.test_POST_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type test.unit.obj.test_server.TestObjectController.test_POST_then_fetch_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta test.unit.obj.test_server.TestObjectController.test_POST_transient_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_update_meta test.unit.obj.test_server.TestObjectController.test_POST_update_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive test.unit.obj.test_server.TestObjectController.test_POST_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json test.unit.obj.test_server.TestObjectController.test_PUT_bad_footer_json ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_bad_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding test.unit.obj.test_server.TestObjectController.test_PUT_bad_transfer_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_calls_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_deleting_expirer_queue_but_still_inserts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue test.unit.obj.test_server.TestObjectController.test_PUT_can_skip_updating_expirer_queue ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection test.unit.obj.test_server.TestObjectController.test_PUT_client_closed_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout test.unit.obj.test_server.TestObjectController.test_PUT_client_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_commits_data test.unit.obj.test_server.TestObjectController.test_PUT_commits_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_common test.unit.obj.test_server.TestObjectController.test_PUT_common ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_connection test.unit.obj.test_server.TestObjectController.test_PUT_container_connection ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides test.unit.obj.test_server.TestObjectController.test_PUT_container_update_overrides ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_old_style_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard test.unit.obj.test_server.TestObjectController.test_PUT_container_update_to_shard ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past test.unit.obj.test_server.TestObjectController.test_PUT_delete_at_in_past ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch test.unit.obj.test_server.TestObjectController.test_PUT_etag_in_footer_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored test.unit.obj.test_server.TestObjectController.test_PUT_extra_mime_docs_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_but_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star test.unit.obj.test_server.TestObjectController.test_PUT_if_none_match_star ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type test.unit.obj.test_server.TestObjectController.test_PUT_invalid_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag test.unit.obj.test_server.TestObjectController.test_PUT_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path test.unit.obj.test_server.TestObjectController.test_PUT_invalid_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer test.unit.obj.test_server.TestObjectController.test_PUT_meta_in_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum test.unit.obj.test_server.TestObjectController.test_PUT_missing_footer_checksum ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_new_object_really_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_eexist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent test.unit.obj.test_server.TestObjectController.test_PUT_next_part_power_races_around_makedirs_enoent ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length test.unit.obj.test_server.TestObjectController.test_PUT_no_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type test.unit.obj.test_server.TestObjectController.test_PUT_no_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_etag test.unit.obj.test_server.TestObjectController.test_PUT_no_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_no_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_object_really_new_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp test.unit.obj.test_server.TestObjectController.test_PUT_old_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite test.unit.obj.test_server.TestObjectController.test_PUT_overwrite ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_newer_ts_failed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_to_older_ts_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at test.unit.obj.test_server.TestObjectController.test_PUT_overwrite_w_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path test.unit.obj.test_server.TestObjectController.test_PUT_redirected_async_pending_with_old_style_container_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_slow test.unit.obj.test_server.TestObjectController.test_PUT_slow ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag test.unit.obj.test_server.TestObjectController.test_PUT_ssync_multi_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST test.unit.obj.test_server.TestObjectController.test_PUT_succeeds_with_later_POST ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_EC_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_async_pendings_with_repl_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_POST_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata test.unit.obj.test_server.TestObjectController.test_PUT_then_fetch_system_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr test.unit.obj.test_server.TestObjectController.test_PUT_user_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers test.unit.obj.test_server.TestObjectController.test_PUT_with_replication_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length test.unit.obj.test_server.TestObjectController.test_PUT_zero_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol test.unit.obj.test_server.TestObjectController.test_REPLICATE_pickle_protocol ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones test.unit.obj.test_server.TestObjectController.test_REPLICATE_reclaims_tombstones ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout test.unit.obj.test_server.TestObjectController.test_REPLICATE_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REPLICATE_works test.unit.obj.test_server.TestObjectController.test_REPLICATE_works ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS test.unit.obj.test_server.TestObjectController.test_REQUEST_SPECIAL_CHARS ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called test.unit.obj.test_server.TestObjectController.test_SSYNC_can_be_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test__call__returns_500 test.unit.obj.test_server.TestObjectController.test__call__returns_500 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_allowed_headers test.unit.obj.test_server.TestObjectController.test_allowed_headers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx test.unit.obj.test_server.TestObjectController.test_async_update_does_not_save_on_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_http_connect test.unit.obj.test_server.TestObjectController.test_async_update_http_connect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_exception ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_non_2xx ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout test.unit.obj.test_server.TestObjectController.test_async_update_saves_on_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_bad_sinces test.unit.obj.test_server.TestObjectController.test_bad_sinces ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_method test.unit.obj.test_server.TestObjectController.test_call_bad_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_bad_request test.unit.obj.test_server.TestObjectController.test_call_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_name_collision test.unit.obj.test_server.TestObjectController.test_call_name_collision ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_call_not_found test.unit.obj.test_server.TestObjectController.test_call_not_found ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_DELETE_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive test.unit.obj.test_server.TestObjectController.test_chunked_PUT_with_full_drive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero test.unit.obj.test_server.TestObjectController.test_chunked_content_length_mismatch_zero ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_chunked_put test.unit.obj.test_server.TestObjectController.test_chunked_put ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout test.unit.obj.test_server.TestObjectController.test_container_update_as_greenthread_with_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_async test.unit.obj.test_server.TestObjectController.test_container_update_async ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_bad_args test.unit.obj.test_server.TestObjectController.test_container_update_bad_args ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update test.unit.obj.test_server.TestObjectController.test_container_update_no_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_container_update_success test.unit.obj.test_server.TestObjectController.test_container_update_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_content_encoding test.unit.obj.test_server.TestObjectController.test_content_encoding ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_correct_allowed_method test.unit.obj.test_server.TestObjectController.test_correct_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container test.unit.obj.test_server.TestObjectController.test_create_reserved_namespace_object_in_user_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_cap test.unit.obj.test_server.TestObjectController.test_delete_at_cap ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_negative test.unit.obj.test_server.TestObjectController.test_delete_at_negative ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries test.unit.obj.test_server.TestObjectController.test_delete_at_update_cleans_old_entries ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete test.unit.obj.test_server.TestObjectController.test_delete_at_update_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_empty_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_container ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host test.unit.obj.test_server.TestObjectController.test_delete_at_update_put_with_info_but_missing_host ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_delete_backend_replication test.unit.obj.test_server.TestObjectController.test_delete_backend_replication ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_device_unavailable test.unit.obj.test_server.TestObjectController.test_device_unavailable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_dynamic_datadir test.unit.obj.test_server.TestObjectController.test_dynamic_datadir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing test.unit.obj.test_server.TestObjectController.test_global_conf_callback_does_nothing ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore test.unit.obj.test_server.TestObjectController.test_global_conf_callback_replication_semaphore ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config test.unit.obj.test_server.TestObjectController.test_handling_of_replication_semaphore_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_init test.unit.obj.test_server.TestObjectController.test_init ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_false ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true test.unit.obj.test_server.TestObjectController.test_insufficient_storage_mount_check_true ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist test.unit.obj.test_server.TestObjectController.test_invalid_method_doesnt_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public test.unit.obj.test_server.TestObjectController.test_invalid_method_is_not_public ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_list_allowed_methods test.unit.obj.test_server.TestObjectController.test_list_allowed_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_log_line_format test.unit.obj.test_server.TestObjectController.test_log_line_format ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_object_name_length test.unit.obj.test_server.TestObjectController.test_max_object_name_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_max_upload_time test.unit.obj.test_server.TestObjectController.test_max_upload_time ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining test.unit.obj.test_server.TestObjectController.test_multiphase_put_draining ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_allowed_method test.unit.obj.test_server.TestObjectController.test_not_allowed_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests test.unit.obj.test_server.TestObjectController.test_not_utf8_and_not_logging_requests ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update test.unit.obj.test_server.TestObjectController.test_object_delete_at_async_update ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_object_update_with_offset test.unit.obj.test_server.TestObjectController.test_object_update_with_offset ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object test.unit.obj.test_server.TestObjectController.test_other_methods_reserved_namespace_object ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header test.unit.obj.test_server.TestObjectController.test_override_etag_footer_trumps_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer test.unit.obj.test_server.TestObjectController.test_override_etag_lone_header_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_footer_trumps_backend_header ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_header_trumps_backend_footer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend test.unit.obj.test_server.TestObjectController.test_override_etag_sysmeta_trumps_backend ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine test.unit.obj.test_server.TestObjectController.test_race_doesnt_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods test.unit.obj.test_server.TestObjectController.test_replication_server_call_all_methods ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_serv_reserv test.unit.obj.test_server.TestObjectController.test_serv_reserv ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_short_body test.unit.obj.test_server.TestObjectController.test_short_body ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated test.unit.obj.test_server.TestObjectController.test_storage_policy_index_is_validated ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers test.unit.obj.test_server.TestObjectController.test_updating_multiple_delete_at_container_servers ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats test.unit.obj.test_server.TestObjectController.test_x_if_delete_at_formats ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted test.unit.obj.test_ssync_sender.TestModuleMethods.test_decode_wanted ... ok test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing test.unit.obj.test_ssync_sender.TestModuleMethods.test_encode_missing ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_multiple_buckets ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_deferral_single_bucket ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_init ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_ratelimited ... ok test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited test.unit.obj.test_updater.TestBucketizedUpdateSkippingLimiter.test_iteration_unlimited ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_not_available ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_reading ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_recheck_time_zero ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_cache_writing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdateShardUTF8.test_get_update_shard_empty_body ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire', 'Content-Length': '11'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_object_upload_part_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_fullcontrol_permission ... ok test ERROR: Error parsing recon cache file: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 85, in _from_recon_cache with openr(cache_file, 'r') as f: ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 225, in raise_ValueError raise ValueError ValueError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/test_recon.py", line 386, in fake_hash_for_file raise IOError OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError test ERROR: Error reading ringfile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/recon.py", line 276, in get_ring_md5 sums[ringfile] = md5_hash_for_file(ringfile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError fake INFO: - - 02/Jun/2024/16/57/50 GET /v1/a/c/o%3Ftemp_url_sig%3Ddeadbeefdeadbeef...%26temp_url_expires%3D1717348470 HTTP/1.0 404 - - - - - - - - 0.0002 - - 1717347470.642782688 1717347470.643017769 - test ERROR: Error in '/tmp/tmpxtltyk99/container-sync-realms.conf' with mtime_check_interval: could not convert string to float: 'invalid' test ERROR: Could not load '/tmp/tmpqcr0hc7m/container-sync-realms.conf': File contains no section headers. file: '/tmp/tmpqcr0hc7m/container-sync-realms.conf', line: 1 'invalid' test DEBUG: Could not load '959a90bc35134d6098535278b22ece44': [Errno 2] No such file or directory: '959a90bc35134d6098535278b22ece44' test ERROR: Could not load '/tmp/tmp67olzynv/container-sync-realms.conf': [Errno 13] Permission denied: '/tmp/tmp67olzynv/container-sync-realms.conf' direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Object server 1.2.3.4:6200 direct DELETE '/sda/0/%D8%AA%20account/%D8%AA%20container/%D8%AA%20obj/name' gave status 500: 1.2.3.4:6200 500 Internal Error: device sda direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! direct-client-test ERROR: Kaboom! Unable to locate config number 5 for account-server Unable to locate config number 5 for object-server Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 15 Signal object-server pid: 1 signal: 1 Signal object-replicator pid: 11 signal: 15 Signal object-replicator pid: 12 signal: 15 Signal proxy-server pid: 1 signal: 1 output Removing pid file /tmp/tmpia9la_pe/var/run/one-server.pid with wrong pid 3 output Removing pid file /tmp/tmpia9la_pe/var/run/zero-server.pid with invalid pid output Removing pid file /tmp/tmpia9la_pe/var/run/invalid-server.pid with invalid pid Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 1 signal: 15 Signal account-reaper pid: 2 signal: 15 test.unit.common.test_manager.TestServer.test_stop test.unit.common.test_manager.TestServer.test_stop ... ok test.unit.common.test_manager.TestServer.test_wait test.unit.common.test_manager.TestServer.test_wait ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix test.unit.common.test_memcached.TestMemcacheCommand.test_get_key_prefix ... ok test.unit.common.test_memcached.TestMemcacheCommand.test_init test.unit.common.test_memcached.TestMemcacheCommand.test_init ... ok test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers test.unit.common.test_swob.TestResponse.test_200_with_body_and_headers ... ok test.unit.common.test_swob.TestResponse.test_507 test.unit.common.test_swob.TestResponse.test_507 ... ok test.unit.common.test_swob.TestResponse.test_app_iter test.unit.common.test_swob.TestResponse.test_app_iter ... ok test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk test.unit.common.test_swob.TestResponse.test_call_finds_nonempty_chunk ... ok test.unit.common.test_swob.TestResponse.test_call_preserves_closeability test.unit.common.test_swob.TestResponse.test_call_preserves_closeability ... ok test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary test.unit.common.test_swob.TestResponse.test_call_reifies_request_if_necessary ... ok test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called test.unit.common.test_swob.TestResponse.test_call_requires_that_start_response_is_called ... ok test.unit.common.test_swob.TestResponse.test_charset test.unit.common.test_swob.TestResponse.test_charset ... ok test.unit.common.test_swob.TestResponse.test_charset_content_type test.unit.common.test_swob.TestResponse.test_charset_content_type ... ok test.unit.common.test_swob.TestResponse.test_content_type test.unit.common.test_swob.TestResponse.test_content_type ... ok test.unit.common.test_swob.TestResponse.test_empty_body test.unit.common.test_swob.TestResponse.test_empty_body ... ok test.unit.common.test_swob.TestResponse.test_etag test.unit.common.test_swob.TestResponse.test_etag ... ok test.unit.common.test_swob.TestResponse.test_host_url_default test.unit.common.test_swob.TestResponse.test_host_url_default ... ok test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_default_port_squelched ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_https test.unit.common.test_swob.TestResponse.test_host_url_host_https ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_override test.unit.common.test_swob.TestResponse.test_host_url_host_override ... ok test.unit.common.test_swob.TestResponse.test_host_url_host_port_override test.unit.common.test_swob.TestResponse.test_host_url_host_port_override ... ok test.unit.common.test_swob.TestResponse.test_host_url_https test.unit.common.test_swob.TestResponse.test_host_url_https ... ok test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched test.unit.common.test_swob.TestResponse.test_host_url_https_port_squelched ... ok test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter test.unit.common.test_swob.TestResponse.test_init_with_body_headers_app_iter ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite test.unit.common.test_swob.TestResponse.test_location_no_rewrite ... ok test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to test.unit.common.test_swob.TestResponse.test_location_no_rewrite_when_told_not_to ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite test.unit.common.test_swob.TestResponse.test_location_rewrite ... ok test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host test.unit.common.test_swob.TestResponse.test_location_rewrite_no_host ... ok test.unit.common.test_swob.TestResponse.test_multi_range_body test.unit.common.test_swob.TestResponse.test_multi_range_body ... ok test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges test.unit.common.test_swob.TestResponse.test_multi_ranges_wo_iter_ranges ... ok test.unit.common.test_swob.TestResponse.test_multi_response_iter test.unit.common.test_swob.TestResponse.test_multi_response_iter ... ok test.unit.common.test_swob.TestResponse.test_properties test.unit.common.test_swob.TestResponse.test_properties ... ok test.unit.common.test_swob.TestResponse.test_range_body test.unit.common.test_swob.TestResponse.test_range_body ... ok test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range test.unit.common.test_swob.TestResponse.test_single_range_wo_iter_range ... ok test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_closes_large_HEAD_resp_iter ... ok test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter test.unit.common.test_swob.TestResponse.test_swob_drains_small_HEAD_resp_iter ... ok test.unit.common.test_swob.TestResponse.test_unicode_body test.unit.common.test_swob.TestResponse.test_unicode_body ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value test.unit.common.test_utils.TestAffinityKeyFunction.test_all_whitespace_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value test.unit.common.test_utils.TestAffinityKeyFunction.test_bogus_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value test.unit.common.test_utils.TestAffinityKeyFunction.test_empty_value ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific test.unit.common.test_utils.TestAffinityKeyFunction.test_more_specific_after_less_specific ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple test.unit.common.test_utils.TestAffinityKeyFunction.test_multiple ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region test.unit.common.test_utils.TestAffinityKeyFunction.test_single_region ... ok test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero test.unit.common.test_utils.TestAffinityKeyFunction.test_with_zone_zero ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_reserved test.unit.common.test_utils.TestFallocateReserve.test_all_reserved ... ok test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct test.unit.common.test_utils.TestFallocateReserve.test_all_space_reserved_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_disabled test.unit.common.test_utils.TestFallocateReserve.test_disabled ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space test.unit.common.test_utils.TestFallocateReserve.test_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve test.unit.common.test_utils.TestFallocateReserve.test_free_space_under_reserve ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_large ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_pct ... ok test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks test.unit.common.test_utils.TestFallocateReserve.test_not_enough_space_small_blocks ... ok test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve test.unit.common.test_utils.TestFallocateReserve.test_zero_reserve ... ok test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir test.unit.common.test_utils.TestSetSwiftDir.test_set_swift_dir ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_empty_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_enable_sharding ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_exception ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_sharding_states ... ok REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e5be1365-76ad-45dd-afd7-d10bb9b4cb9c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e5be1365-76ad-45dd-afd7-d10bb9b4cb9c-sda', '1717347453.36250', '1717347453.00000', '0', '', '1717347453.00000', 0, 0) REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, complete_rsync, ('e5be1365-76ad-45dd-afd7-d10bb9b4cb9c-sda', '5267a50d4a8f6ea3ee1dbecb8592e872.db') test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 4.22820 seconds (0.47301/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:4 ts_repl:0 test INFO: Beginning replication run test DEBUG: Replicating db /tmp/tmptn9nhpsu/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db REPLICATE: /sdb/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '7ce50ea4-4122-43da-bc9e-ccd470f4cf21-sda', '1717347453.37444', '1717347454.00000', '0', '{"X-Container-Sync-To": ["", "1717347457.00000"], "X-Container-Sync-Key": ["secret", "1717347456.00000"]}', '1717347454.00000', 0, 0) test DEBUG: /tmp/tmptn9nhpsu/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/c48d89b33840539f9c5442ab3b10027d, sync, (-1, '00000000000000000000000000000000', '7ce50ea4-4122-43da-bc9e-ccd470f4cf21-sda', '1717347453.37444', '1717347454.00000', '0', '{"X-Container-Sync-To": ["", "1717347457.00000"], "X-Container-Sync-Key": ["secret", "1717347456.00000"]}', '1717347454.00000', 0, 0) test DEBUG: /tmp/tmptn9nhpsu/sda/containers/0/27d/c48d89b33840539f9c5442ab3b10027d/c48d89b33840539f9c5442ab3b10027d.db in sync with 10.0.0.2:1002/sdc, nothing to do test DEBUG: Replicating db /tmp/tmptn9nhpsu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db REPLICATE: /sdb/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e5be1365-76ad-45dd-afd7-d10bb9b4cb9c-sda', '1717347453.36250', '1717347453.00000', '0', '', '1717347453.00000', 0, 0) test DEBUG: /tmp/tmptn9nhpsu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.1:1001/sdb, nothing to do REPLICATE: /sdc/0/5267a50d4a8f6ea3ee1dbecb8592e872, sync, (-1, '00000000000000000000000000000000', 'e5be1365-76ad-45dd-afd7-d10bb9b4cb9c-sda', '1717347453.36250', '1717347453.00000', '0', '', '1717347453.00000', 0, 0) test DEBUG: /tmp/tmptn9nhpsu/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db in sync with 10.0.0.2:1002/sdc, nothing to do test INFO: Replication run OVER test INFO: Attempted to replicate 2 dbs in 0.03979 seconds (50.26447/s) test INFO: Removed 0 dbs test INFO: 4 successes, 0 failures test INFO: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:4 remote_merge:0 rsync:0 ts_repl:0 test ERROR: Failed to update sync_store /tmp/tmp_g3my6p0/sda/containers/0/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/replicator.py", line 301, in _post_replicate_hook self.sync_store.update_sync_store(broker) File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_replicator.py", line 1226, in update_sync_store raise OSError(1, '1') PermissionError: [Errno 1] 1 proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=%E1%88%B41_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server INFO: Caching updating shards for shard-updating-v2/a/c (3 shards) (txn: tx28a0bf3a805740f2982b0-00665ca496) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txf998e9a8bf4e4332b37da-00665ca497) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpphyh365j/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpphyh365j/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpphyh365j/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpphyh365j/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0165 "-" 3656181 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0164 "-" 3656181 - test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_default_proxy_config ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_log_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_bad_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_equality ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_inherits_defaults ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_read_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_sorting_method_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_node_count_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_invalid_write_affinity_value ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_none_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_one_configured ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_affinity ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_concurrency_settings ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_overrides_default_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_section_name_inherits_from_app_section_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_sets_timing_sorting_method ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_about_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_warns_override_sorting_method_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_DEFAULT_options ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy test.unit.proxy.test_server.TestProxyServerConfigStringLoading.test_per_policy_conf_with_unknown_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults test.unit.proxy.test_server.TestSwiftInfo.test_registered_defaults ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:36 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_acl_without_src_permission ... ok proxy-server ERROR: ERROR Unhandled exception in request: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/server.py", line 598, in handle_request return handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 375, in wrapped return func(*a, **kw) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 275, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 263, in GETorHEAD resp = self._get_or_head_response(req, node_iter, partition, policy) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2930, in _get_or_head_response app_iter.kickoff(req, resp) File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1184, in kickoff self.stashed_iter = reiterate(self._real_iter(req, resp.headers)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3749, in reiterate chunk = next(iterator) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 4452, in _get_next_item return super(StringAlong, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1470, in _iter_one_range for i, next_seg in enumerate(segment_iter): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1549, in _decode_segments_from_fragments segment = self.policy.pyeclib_driver.decode(fragments) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/ec_iface.py", line 300, in decode return self.ec_lib_reference.decode(fragment_payloads, ranges, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyeclib/core.py", line 91, in decode return pyeclib_c.decode(self.handle, _fragment_payloads, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pyeclib.ec_iface.ECInvalidFragmentMetadata: pyeclib_c_decode ERROR: Fragment integrity check failed. Please inspect syslog for liberasurecode error report. (txn: tx156985ff2023488080356-00665ca478) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server ERROR: Object returning 503 for [] (txn: tx07a4f48a02714fd1a01ac-00665ca47d) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_may_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_stop ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_mixed_frags_will_dig_deep_but_succeed ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_and_range_unsatisifiable ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durable_files_and_mixed_etags ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_non_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_missing_durables_and_older_obscured_durables ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_and_nondurable_frags_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_durable_frags_and_no_quorum_will_503 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_items_since ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_root_container_fill_gap ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (201, 404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (201, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_code_for_PUT ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_namespaces_state_filtering ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects ... ok acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0140 "-" 3656181 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0120 "-" 3656181 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txf3e2cc860c2f45d48e59c-00665ca497" "proxy-server 3656181" 0.0020 "-" 3656181 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:55659/sda1/1/a/c" "txf3e2cc860c2f45d48e59c-00665ca497" "container-server 3656181" 0.0015 "-" 3656181 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:40707/sdb1/1/a/c" "txf3e2cc860c2f45d48e59c-00665ca497" "container-server 3656181" 0.0014 "-" 3656181 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf3e2cc860c2f45d48e59c-00665ca497" "proxy-server 3656181" 0.0212 "-" 3656181 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txf3e2cc860c2f45d48e59c-00665ca497" "proxy-server 3656181" 0.0384 "-" 3656181 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/59 PUT /v1/a/c HTTP/1.0 201 - - t - - - txf3e2cc860c2f45d48e59c-00665ca497 - 0.0505 - - 1717347479.166903734 1717347479.217406273 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx7810fb5a14cb4589b89f5-00665ca497" "proxy-server 3656181" 0.0018 "-" 3656181 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:55659/sda1/1/a1/c1" "tx7810fb5a14cb4589b89f5-00665ca497" "container-server 3656181" 0.0015 "-" 3656181 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:40707/sdb1/1/a1/c1" "tx7810fb5a14cb4589b89f5-00665ca497" "container-server 3656181" 0.0014 "-" 3656181 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx7810fb5a14cb4589b89f5-00665ca497" "proxy-server 3656181" 0.0338 "-" 3656181 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx7810fb5a14cb4589b89f5-00665ca497" "proxy-server 3656181" 0.0205 "-" 3656181 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/59 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx7810fb5a14cb4589b89f5-00665ca497 - 0.0455 - - 1717347479.219350100 1717347479.264870644 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx69300af86547403e9ba1a-00665ca497" "proxy-server 3656181" 0.0085 "-" 3656181 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:55659/sda1/2/a/c1" "tx69300af86547403e9ba1a-00665ca497" "container-server 3656181" 0.0016 "-" 3656181 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:40707/sdb1/2/a/c1" "tx69300af86547403e9ba1a-00665ca497" "container-server 3656181" 0.0094 "-" 3656181 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx69300af86547403e9ba1a-00665ca497" "proxy-server 3656181" 0.0499 "-" 3656181 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx69300af86547403e9ba1a-00665ca497" "proxy-server 3656181" 0.0361 "-" 3656181 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/59 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx69300af86547403e9ba1a-00665ca497 - 0.0686 - - 1717347479.266844988 1717347479.335444927 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txd3b977c68d884436bbae8-00665ca497" "proxy-server 3656181" 0.0156 "-" 3656181 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:55659/sda1/1/a/c2" "txd3b977c68d884436bbae8-00665ca497" "container-server 3656181" 0.0014 "-" 3656181 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:40707/sdb1/1/a/c2" "txd3b977c68d884436bbae8-00665ca497" "container-server 3656181" 0.0107 "-" 3656181 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd3b977c68d884436bbae8-00665ca497" "proxy-server 3656181" 0.0282 "-" 3656181 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:57:59 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txd3b977c68d884436bbae8-00665ca497" "proxy-server 3656181" 0.0428 "-" 3656181 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/57/59 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txd3b977c68d884436bbae8-00665ca497 - 0.0677 - - 1717347479.337375164 1717347479.405058622 2 test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_objects_since_row ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_policy_info ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:39 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_acl_without_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_range_rows_with_limit ... ok test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:38 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:40 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_match_and_s3acl ... ok test INFO: - - 02/Jun/2024/16/57/37 POST /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0003 FA - 1717347457.718191862 1717347457.718505383 0 test INFO: - - 02/Jun/2024/16/57/40 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0008 SSC - 1717347460.342068911 1717347460.342830896 0 test INFO: - - 02/Jun/2024/16/57/40 POST /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717347460.344229937 1717347460.344680786 0 test INFO: - - 02/Jun/2024/16/57/40 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 2.6281 - - 1717347457.717950106 1717347460.346053600 - test INFO: - - 02/Jun/2024/16/57/40 PUT /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0006 FA - 1717347460.350036144 1717347460.350604773 0 test INFO: - - 02/Jun/2024/16/57/40 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0006 SSC - 1717347460.351636171 1717347460.352230072 0 test INFO: - - 02/Jun/2024/16/57/40 PUT /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717347460.353355885 1717347460.353859425 0 test INFO: - - 02/Jun/2024/16/57/40 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0056 - - 1717347460.349660397 1717347460.355302811 - test INFO: - - 02/Jun/2024/16/57/40 DELETE /v1/a/c/sub_get HTTP/1.0 200 - FakeApp - - - - - - 0.0005 FA - 1717347460.359236956 1717347460.359701157 0 test INFO: - - 02/Jun/2024/16/57/40 GET /v1/a/test/obj HTTP/1.0 200 - Swift - - - - - - 0.0006 SSC - 1717347460.360497952 1717347460.361081362 0 test INFO: - - 02/Jun/2024/16/57/40 DELETE /v1/a/c/sub_put HTTP/1.0 200 - FakeApp - - - - - - 0.0004 FA - 1717347460.362193823 1717347460.362572908 0 test INFO: - - 02/Jun/2024/16/57/40 PUT /v1/a/c/o HTTP/1.0 200 - - - - - - - - 0.0052 - - 1717347460.358836889 1717347460.364013910 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:41 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0663 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:41 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:41 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0107 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:41 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:41 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:41 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0024 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0169 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:42 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0104 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:42 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:42 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0197 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:42 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:44 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.4561 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:44 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.4949 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:44 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:44 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:46 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 2.0149 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:47 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.6581 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:47 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.5776 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:48 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.5187 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:48 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.5209 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.1616 "-" 3656236 0 test ERROR: ERROR Account update failed with 127.0.0.1:55207/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.1648 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0269 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0130 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c" 404 - "-" "-" "-" 0.0006 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0134 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c" 409 - "-" "-" "-" 0.0206 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0011 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0214 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0023 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0161 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/bashful" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0179 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy10" 204 - "-" "-" "-" 0.0027 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy10_no_header" 204 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0044 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0096 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0030 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy50" 204 - "-" "-" "-" 0.0022 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0082 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy60_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0065 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy70" 204 - "-" "-" "-" 0.0027 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy70_no_header" 204 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy80" 204 - "-" "-" "-" 0.0022 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpy80_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier10" 204 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier10_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0057 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier30_no_header" 204 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier40_no_header" 204 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0050 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier50" 204 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier50_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0015 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier60_no_header" 204 - "-" "-" "-" 0.0048 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier70" 204 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier70_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier80" 204 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/grumpier80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/dopey" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0094 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/dopey10" 204 - "-" "-" "-" 0.0025 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/dopey10_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0023 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:50 +0000] "DELETE /sda1/p/a/c/dopey20_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey30_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey40_no_header" 204 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey50" 204 - "-" "-" "-" 0.0021 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey50_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0109 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey60_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey70" 204 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey70_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0107 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey80" 204 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey80_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0014 "-" 3656236 - test DEBUG: Unlinked retiring db '/tmp/tmpd53vsei6/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest10" 204 - "-" "-" "-" 0.0017 "-" 3656236 - test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_DELETE_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float test.unit.container.test_server.TestContainerController.test_DELETE_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid test.unit.container.test_server.TestContainerController.test_GET_accept_not_valid ... ok test.unit.container.test_server.TestContainerController.test_GET_auto_record_type test.unit.container.test_server.TestContainerController.test_GET_auto_record_type ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter test.unit.container.test_server.TestContainerController.test_GET_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii test.unit.container.test_server.TestContainerController.test_GET_delimiter_non_ascii ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server ERROR: Object returning 503 for [] (txn: tx3c651062e95d4894900c3-00665ca47e) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server ERROR: Object returning 503 for [] (txn: txb79aa4728af24aa59fa40-00665ca481) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx27d61cb3115b41249edac-00665ca481) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from primary (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx3efedeab15a44799868b0-00665ca481) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx5646fdb9f1a2488fbd2ce-00665ca482) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Object returning 503 for [] (txn: tx500994ecc0a84a229ecb5-00665ca482) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_etags_at_same_timestamp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_mixed_nondurable_frags_and_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_short_resume_body ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_slow_body ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAclNonUTC.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_cors_preflight ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_delete_marker ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_current_version_id_is_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_with_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_in_versioned_container_without_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_missing ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_no_multipart ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_old_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_DELETE_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Range_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_Response ... ok test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest10_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0067 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest20_no_header" 204 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest30_no_header" 204 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest40_no_header" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0072 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest50" 204 - "-" "-" "-" 0.0054 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest50_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0074 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0025 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest60_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0081 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest70" 204 - "-" "-" "-" 0.0026 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest70_no_header" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest80" 204 - "-" "-" "-" 0.0025 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c/grumpiest80_no_header" 204 - "-" "-" "-" 0.0010 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0144 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "DELETE /sda1/p/a/c" 400 26 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0154 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 406 - "-" "-" "-" 0.0005 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0139 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0081 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0037 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0047 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0060 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0056 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0047 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0053 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0089 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0046 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0049 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0054 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0058 "-" 3656236 0 test DEBUG: Unlinked retiring db '/tmp/tmp73axgmp6/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0043 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 648 "-" "-" "-" 0.0040 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0049 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0085 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 66 "-" "-" "-" 0.0216 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/1" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/a/%E2%9D%A5/2" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/1" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/a/%EA%99%AE/2" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 50 "-" "-" "-" 0.0104 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 14 "-" "-" "-" 0.0036 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0123 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml ... ok test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes test.unit.container.test_server.TestContainerController.test_GET_delimiter_xml_with_quotes ... ok test.unit.container.test_server.TestContainerController.test_GET_invalid_accept test.unit.container.test_server.TestContainerController.test_GET_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_GET_json test.unit.container.test_server.TestContainerController.test_GET_json ... ok test.unit.container.test_server.TestContainerController.test_GET_json_last_modified test.unit.container.test_server.TestContainerController.test_GET_json_last_modified ... ok test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter test.unit.container.test_server.TestContainerController.test_GET_leading_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_limit test.unit.container.test_server.TestContainerController.test_GET_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_false test.unit.container.test_server.TestContainerController.test_GET_log_requests_false ... ok test.unit.container.test_server.TestContainerController.test_GET_log_requests_true test.unit.container.test_server.TestContainerController.test_GET_log_requests_true ... ok test.unit.container.test_server.TestContainerController.test_GET_marker test.unit.container.test_server.TestContainerController.test_GET_marker ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:41 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:42 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_invalid_source ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Problem with fragment response: ETag mismatch (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server ERROR: Object returning 503 for [] (txn: tx529eb7ab92f54190a5a14-00665ca484) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.13:1013/sdn (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.10:1010/sdk (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from primary (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server ERROR: Exception fetching fragments for '/a/c/o': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 1496, in put_fragments_in_queue for fragment in frag_iter: File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3709, in __next__ return self._get_next_item() ^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 6533, in _get_next_item return super(CooperativeIterator, self)._get_next_item() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3705, in _get_next_item return next(self.wrapped_iter) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2518, in _iter_bytes_from_response_part six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/obj.py", line 2490, in _iter_bytes_from_response_part chunk = part_file.read(self.app.object_chunk_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1133, in read raise ShortReadError( swift.common.exceptions.ShortReadError: Too few bytes; read 415, expecting 490 (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server WARNING: Un-recoverable fragment rebuild. Only received 9/10 fragments for '/a/c/o' (txn: tx9ae02a8d4396440084875-00665ca486) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.4:1004/sde (txn: txb6b462c89550456b81600-00665ca486) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_multirange_unable_to_resume_body_started ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_durable_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_get_shard_usage ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:44 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:full_control:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_fullcontrol_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_owner_permission ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:45 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 175, in _restore_on_delete resp = req.get_response(self.app, 'GET', req.container_name, '', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 236, in DELETE new_resp = self._restore_on_delete(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 187, in _restore_on_delete resp = req.get_response(self.app, 'PUT', query={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: skipping failed PUT?version-id=1574341899.21751 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 229, in DELETE resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_version_id_not_implemented ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_GET_with_version_id_but_not_enabled ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_Range_with_invalid_value ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_HEAD_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_POST_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_headers_with_not_match ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_copy_self_metadata_replace ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_error ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_headers ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_quota_exceeded ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4 ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_bad_hash ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_v4_unsigned_payload ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_PUT_with_version ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_object_policy_index_logging ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_simple_object_copy ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_DELETE ... ok test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE test.unit.common.middleware.s3api.test_obj.TestS3ApiObj.test_slo_object_async_DELETE ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_auth_uris ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorization_nova_toconnect ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key ... ok test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-OK-A" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-OK-B" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "PUT /sda1/p/a/c/US-UT-A" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:51 +0000] "GET /sda1/p/a/c" 200 221 "-" "-" "-" 0.0130 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0193 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/%3C%27sub%27%20%22dir%22%3E/object" 201 - "-" "-" "-" 0.0014 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 200 159 "-" "-" "-" 0.0226 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 400 21 "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0150 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 2 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc/2" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0153 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0071 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0028 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0045 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0047 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 351 "-" "-" "-" 0.0041 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "HEAD /sda1/p/a/jsonc" 204 - "-" "-" "-" 0.0022 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc/0" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/jsonc/1" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/jsonc" 200 234 "-" "-" "-" 0.0173 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0216 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/US-TX-A" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/US-TX-B" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/-UK" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/-CH" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 200 36 "-" "-" "-" 0.0154 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0213 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 200 4 "-" "-" "-" 0.0122 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0007 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "PUT /sda1/p/a/c/2" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0117 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:52 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0045 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0044 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0041 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~TX~~A" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~TX~~B" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~OK~~A" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~OK~~B" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~A" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~OK~Tulsa~~B" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~UT~~A" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c/US~~UT~~~B" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0116 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 102 "-" "-" "-" 0.0144 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0047 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 24 "-" "-" "-" 0.0045 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0046 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 150 "-" "-" "-" 0.0047 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter test.unit.container.test_server.TestContainerController.test_GET_multichar_delimiter ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors test.unit.container.test_server.TestContainerController.test_GET_namespaces_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported test.unit.container.test_server.TestContainerController.test_GET_namespaces_not_supported ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params test.unit.container.test_server.TestContainerController.test_GET_namespaces_other_params ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_sharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_listing ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_root_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_listing ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.11:1011/sdl (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server ERROR: ERROR Unhandled exception in request: ChunkReadTimeout (0.01s after 0.02s) (txn: txa3f8875e5f504e63b387b-00665ca48a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read next part of EC fragment multi-part GET (retrying) 10.0.0.11:1011/sdl (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from primary (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to read during GET: ChunkReadTimeout (0.01s after 0.02s) (txn: tx60d18ecbf7884e51b12fa-00665ca48a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx84b403efc0a9459cac13f-00665ca48c) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from primary (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_no_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_one_slow_frag_lane ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_only_handoffs ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_quorum_durable_files ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_range_unsatisfiable_mixed_success ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_durable_file ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_single_missed_overwrite_does_not_need_handoff ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_nodes_and_failures ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_slow_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_some_slow_primaries ... ok test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.3:1003/sdd test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (503, 503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: Container GET returning 503 for [] (txn: txa29e3f0fde6e4077a1bfe-00665ca47e) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.6:1006/sda (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.7:1007/sda (txn: tx1d57dc3e78b547efb7916-00665ca48a) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500, 500, 500] (txn: tx1d57dc3e78b547efb7916-00665ca48a) test INFO: Client disconnected on read of '/a/c' (txn: tx988d803af51e447cb2165-00665ca48a) test INFO: Client disconnected on read of '/a/c' (txn: tx23bb716970574b81b3f40-00665ca48a) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_swift_owner ... ok test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 136, in GET return self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 123, in HEAD resp = self.GETorHEAD(req) ^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 102, in GETorHEAD resp = req.get_response(self.app, query=query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test ERROR: InternalError: 500 Internal Server Error (unexpected status code 500): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 349, in __call__ resp = self.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3api.py", line 389, in handle_request res = handler(req) ^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/controllers/obj.py", line 157, in PUT resp = req.get_response(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1455, in get_response return self._get_response(app, method, container, obj, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/middleware/s3api/s3request.py", line 1440, in _get_response raise InternalError('unexpected status code %d' % status) swift.common.middleware.s3api.s3response.InternalError: InternalError: 500 Internal Server Error (unexpected status code 500) test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/57/54 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0021 - - 1717347474.429262877 1717347474.431411505 1 test DEBUG: Calling S3Api Middleware test INFO: - - 02/Jun/2024/16/57/54 GET /bucket/object HTTP/1.0 200 - Mozzarella%20Foxfire - - 5 - - - 0.0019 - - 1717347474.432722807 1717347474.434615850 0 test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Calling S3Api Middleware test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: FORCED_TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorize_with_access_key_and_unquote_chars ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_bytes ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_http ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_trailing_slash ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_authorized_v3 ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_bad_auth_uris ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_http_timeout_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_insecure_option ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_no_path_request ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_nukes_auth_headers ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_reseller_prefix ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_is_cached ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_secret_sets_cache ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_tolerate_missing_token_id ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_unicode_path ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_auth_storage_token ... ok test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization test.unit.common.middleware.s3api.test_s3token.S3TokenMiddlewareTestGood.test_without_authorization ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_authorized_for_permitted_referrer ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:46 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:47 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_with_write_permission ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_permission ... ok test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0049 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 251 "-" "-" "-" 0.0047 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 126 "-" "-" "-" 0.0046 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0192 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0075 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0026 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0093 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0158 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0069 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0069 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0071 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0070 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0064 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 400 36 "-" "-" "-" 0.0025 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 400 35 "-" "-" "-" 0.0023 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0068 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0118 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.1399 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.3334 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0316 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 192 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 202 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 63 "-" "-" "-" 0.0020 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/a/c" 200 323 "-" "-" "-" 0.0020 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0156 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0189 "-" 3656236 0 test DEBUG: Unlinked retiring db '/tmp/tmpzfkhkbbr/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0033 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0143 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:53 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0183 "-" 3656236 0 test DEBUG: Unlinked retiring db '/tmp/tmpil05phf6/tmp_test_container_server_ContainerController/sda1/containers/p/61c/b6a56f59d3a8b4d57e122a32d715b61c/b6a56f59d3a8b4d57e122a32d715b61c.db' test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 1672 "-" "-" "-" 0.0032 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0136 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0026 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0128 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/a/c" 200 277 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0027 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0145 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0054 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0120 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0291 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 285 "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0027 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating test.unit.container.test_server.TestContainerController.test_GET_namespaces_unsharded_subshard_state_updating ... ok test.unit.container.test_server.TestContainerController.test_GET_non_ascii test.unit.container.test_server.TestContainerController.test_GET_non_ascii ... ok test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies test.unit.container.test_server.TestContainerController.test_GET_objects_of_different_policies ... ok test.unit.container.test_server.TestContainerController.test_GET_over_limit test.unit.container.test_server.TestContainerController.test_GET_over_limit ... ok test.unit.container.test_server.TestContainerController.test_GET_path test.unit.container.test_server.TestContainerController.test_GET_path ... ok test.unit.container.test_server.TestContainerController.test_GET_plain test.unit.container.test_server.TestContainerController.test_GET_plain ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_has_other_shard_ranges ... ok proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Ignoring 404 from handoff (txn: txf73a47fb460945f2b303d-00665ca48c) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server WARNING: Client disconnected on read of EC frag '/a/c/o' (txn: tx59945fbbbbfa4b36a2228-00665ca491) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Ignoring 404 from primary (txn: tx301a77d639b44a9da5d42-00665ca491) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx0fe163fa49774454a10a8-00665ca491) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR 500 Trying to GET /a/c/o From Object Server 10.0.0.12:1012/sdm (txn: tx2c45140cc47743d984bde-00665ca491) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from primary (txn: tx8767127defaa469a8b30d-00665ca492) test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_404_will_404 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_with_success_and_507_will_503 ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_GET_write_timeout ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_initialize_container_broker_in_default ... ok test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc8d291b971f746c1b6049-00665ca48a) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc8d291b971f746c1b6049-00665ca48a) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc8d291b971f746c1b6049-00665ca48a) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc8d291b971f746c1b6049-00665ca48a) test ERROR: Container GET returning 503 for [] (txn: txc8d291b971f746c1b6049-00665ca48a) test INFO: Client disconnected on read of '/a/c' (txn: txc8d291b971f746c1b6049-00665ca48a) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: Container GET returning 503 for [] (txn: txc6cc0c853eef4c778d4d8-00665ca48e) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: Container GET returning 503 for [] (txn: tx7d16e71f4b0d43d0a9110-00665ca48e) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb913f5ad864b420a97c05-00665ca48e) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb913f5ad864b420a97c05-00665ca48e) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txb913f5ad864b420a97c05-00665ca48e) test ERROR: Container GET returning 503 for [] (txn: txb913f5ad864b420a97c05-00665ca48e) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx53d884b48d89469385a3f-00665ca48e) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx53d884b48d89469385a3f-00665ca48e) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx0183ad4302a74ed88f71f-00665ca48e) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx0183ad4302a74ed88f71f-00665ca48e) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sdd (txn: tx0183ad4302a74ed88f71f-00665ca48e) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx0183ad4302a74ed88f71f-00665ca48e) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: txc7befb9d12e54c65b9963-00665ca48e) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx868aafe1dacc46f58a5f7-00665ca48e) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx868aafe1dacc46f58a5f7-00665ca48e) test INFO: Client disconnected on read of '/a/c' (txn: tx868aafe1dacc46f58a5f7-00665ca48e) test ERROR: Container GET returning 503 for [] (txn: tx6aa9ad3d289b467ea659f-00665ca48e) test INFO: Client disconnected on read of '/a/c' (txn: tx6aa9ad3d289b467ea659f-00665ca48e) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx95e2e416a83f441aa6cfb-00665ca490) test INFO: Client disconnected on read of '/a/c' (txn: tx95e2e416a83f441aa6cfb-00665ca490) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test ERROR: ERROR with Container server 10.0.0.3:1003/sdd re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_POST ... ok test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController4Replicas.test_sys_meta_headers_PUT ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_record_type_object_makes_no_cache_lookup ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_include_deleted_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_listing_shards ... ok test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards test.unit.proxy.controllers.test_container.TestGetExplicitRecordType.test_GET_shard_ranges_no_cache_when_requesting_updating_shards ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_empty_body ... ok test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} test DEBUG: Error on keystone reply: 201 {"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}} - rejecting request test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Not a path query: /, skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Cached creds valid test DEBUG: Connecting with tenant: tenant_id test DEBUG: Starting the S3 Token Authentication component test DEBUG: Starting the S3 Token Authentication component test INFO: Caching s3tokens for 20 seconds test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Cached keystone credentials test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: Connecting to Keystone sending this JSON: {"credentials": {"access": "access", "token": "dG9rZW4=", "signature": "signature"}} test DEBUG: Keystone Reply: Status: 201, Output: b'{"access": {"user": {"username": "S3_USER", "name": "S3_USER", "id": "USER_ID", "roles": [{"name": "swift-user"}, {"name": "_member_"}]}, "token": {"id": "TOKEN_ID", "tenant": {"id": "TENANT_ID", "name": "TENANT_NAME"}}}}' test DEBUG: Connecting with tenant: TENANT_ID test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Starting the S3 Token Authentication component test DEBUG: Calling S3Token middleware. test DEBUG: No authorization details from s3api. skipping. test DEBUG: Authorizing as anonymous test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_is_not_authorized_for_unknown_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_anonymous_with_validtoken_authorized_for_permitted_referrer ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_auth_scheme ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_blank_reseller_prefix ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_confirmed_identity_is_not_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_denied_responses ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_detect_reseller_request ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_identified_options_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_invalid_request_non_authorized ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_and_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_asked_for_but_not_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_override_default_allowed ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_not_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_to_unknown ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_project_domain_id_sysmeta_set_unknown_with_v2 ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_added_underscore ... ok test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores test.unit.common.middleware.test_keystoneauth.SwiftAuthMultiple.test_reseller_prefix_not_added_double_underscores ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_access_log_headers_only ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_app_exception ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_basic_req_second_time ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_body_iter_updates_environ_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_chunked_response ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_client_logging ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_read ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_disconnect_on_readline ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_dual_logging_middlewares_w_inner ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ensure_fields ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_app_explodes ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_and_body_explodes ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_environ_has_proxy_logging_status_unread_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_exploding_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_facility ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_filter ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_invalid_log_config ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_ipv6 ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_iterator_closing ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_auth_token ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_headers ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_multiple ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_none ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_info_single ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_msg_template_s3api ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_query_string ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_custom ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_last_modified_time ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other WRITE test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:49 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_acl_without_src_permission ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter ... ok test INFO: - - - [02/Jun/2024:16:57:54 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0027 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/jsonc" 201 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/jsonc/Sp%C3%A4tzle" 201 - "-" "-" "-" 0.0011 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/jsonc/%E3%83%A9%E3%83%BC%E3%83%A1%E3%83%B3" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/jsonc" 200 268 "-" "-" "-" 0.0130 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/jsonc" 200 22 "-" "-" "-" 0.0033 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0137 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_0" 201 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_1" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_2" 201 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_3" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_4" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_5" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_6" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_7" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_8" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_9" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_default_10" 201 - "-" "-" "-" 0.0012 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_0" 201 - "-" "-" "-" 0.0005 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_1" 201 - "-" "-" "-" 0.0005 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_2" 201 - "-" "-" "-" 0.0016 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_3" 201 - "-" "-" "-" 0.0013 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_4" 201 - "-" "-" "-" 0.0006 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_5" 201 - "-" "-" "-" 0.0006 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_6" 201 - "-" "-" "-" 0.0005 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_7" 201 - "-" "-" "-" 0.0015 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_8" 201 - "-" "-" "-" 0.0012 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/obj_1_9" 201 - "-" "-" "-" 0.0005 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0167 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 200 155 "-" "-" "-" 0.0037 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 200 80 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 400 40 "-" "-" "-" 0.0018 "-" 3656236 3 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0171 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/US/TX" 201 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/US/TX/B" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/US/OK" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/US/OK/B" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "PUT /sda1/p/a/c/US/UT/A" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:56 +0000] "GET /sda1/p/a/c" 200 242 "-" "-" "-" 0.0181 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/plainc" 201 - "-" "-" "-" 0.0151 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0032 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/plainc/0" 201 - "-" "-" "-" 0.0021 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/plainc/1" 201 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/plainc/2" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0182 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "HEAD /sda1/p/a/plainc" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0073 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0041 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0032 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0031 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/plainc" 200 6 "-" "-" "-" 0.0037 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0130 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c/a1" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c/b1" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c/a2" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test.unit.container.test_server.TestContainerController.test_GET_prefix test.unit.container.test_server.TestContainerController.test_GET_prefix ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_auditing ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_errors ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_from_compacted_shard ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_include_deleted ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_using_state_aliases ... ok test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header test.unit.container.test_server.TestContainerController.test_GET_shard_ranges_with_format_header ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_make_tuple_for_pickle ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx23038c4a621b4304b6b26-00665ca490) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: tx23038c4a621b4304b6b26-00665ca490) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: ERROR with Container server 10.0.0.6:1006/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: ERROR with Container server 10.0.0.7:1007/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9862cfbb4977404387b34-00665ca497) test ERROR: Container GET returning 503 for (200, 503, 503, 503) (txn: tx9862cfbb4977404387b34-00665ca497) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: JSONDecodeError('Expecting value: line 1 column 1 (char 0)') (txn: txbd73d65f7bff4618834c9-00665ca497) test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_invalid_shard_range ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_key_missing ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_missing_record_type ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_not_a_list ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_request_failed ... ok test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type test.unit.proxy.controllers.test_obj.TestGetUpdatingNamespacesErrors.test_get_namespaces_wrong_record_type ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_is_green ... ok test DEBUG: Authorizing as anonymous test DEBUG: Using identity: {'user': ('42', 'usr'), 'tenant': ('1', 'acct'), 'roles': ['admin'], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_acct != 1 test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Using identity: {'user': (None, None), 'tenant': (None, None), 'roles': [], 'service_roles': [], 'user_domain': (None, None), 'project_domain': (None, None), 'auth_version': 0} test DEBUG: tenant mismatch: AUTH_foo != None test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing as anonymous test DEBUG: Authorizing from an overriding middleware test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 200 - - - - 8 - - First:%201%0ASecond:%202 0.0001 - - 1717347483.950355291 1717347483.950484037 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 500 - - - - - - - - 0.0000 - - 1717347483.955118179 1717347483.955137730 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347483.957174778 1717347483.957283497 - test INFO: GET /v1/a/c 205 205 test INFO: GET /v1/a/c 209 205 test INFO: GET /v1/a/c 210 205 test INFO: 1.2.3.4 1.2.3.4 02/Jun/2024/16/58/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347483.974861860 1717347483.974967718 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717347483.977072477 1717347483.977115870 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 499 - - - - - - - - 0.0000 - - 1717347483.978978872 1717347483.979004145 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347483.980922699 1717347483.981025219 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347483.983143568 1717347483.983238220 - test INFO: - - 02/Jun/2024/16/58/03 GET / HTTP/1.0 200 - - - - 15 - - - 0.0009 - - 1717347483.983094692 1717347483.983967066 - test INFO: - - 26/Apr/1970/17/46/41 GET / HTTP/1.0 200 - - - - 8 - - - 1.0000 - - 10000000.000000000 10000001.000000000 - test INFO: GET /v1/a/c 503 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c 429 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c True 503 test INFO: GET /v1/a/c - 503 test INFO: GET /v1/a/c parsing%20ok 503 test INFO: GET /v1/a/c 500 500 test INFO: GET /v1/a/c 500 568 test INFO: GET /v1/a/c 499 503 test INFO: GET /v1/a/c 499 503 test INFO: GET / 500 200 test INFO: 2001:db8:85a3:8d3:1319:8a2e:370:7348 2001:db8:85a3:8d3:1319:8a2e:370:7348 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.820293903 1717347484.820356846 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.823270321 1717347484.823332787 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - b05bf940-0464-4c... - 8 - - - 0.0005 - - 1717347484.824368238 1717347484.824882507 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.825935364 1717347484.825990200 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - b05bf940... - 8 - - - 0.0001 - - 1717347484.827140093 1717347484.827204943 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717347484.828300953 1717347484.828355789 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - b05bf940-0464-4c0e-8c70-87717d2d73e8 - 8 - - - 0.0001 - - 1717347484.829331636 1717347484.829388857 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.830358028 1717347484.830414295 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - ... - 8 - - - 0.0001 - - 1717347484.831444025 1717347484.831500053 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717347484.832875490 1717347484.832934141 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - Host:%20localhost:80 0.0001 - - 1717347484.833977938 1717347484.834036112 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one%2Cand%20two 1717347484.835400105 1717347484.835457802 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.836711168 1717347484.836766481 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.837714911 1717347484.837769747 - test INFO: - - 02/Jun/2024/16/58/04 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - one 1717347484.839062214 1717347484.839119911 - test INFO: template which can be edited in config: HTTP/1.0 / GET {SMD5}c65475e457fea0951fbb9ec9596b2177 - 1.0000 26/Apr/1970/17/46/40 10000001.000000000 0.5 example.com test INFO: HTTP/1.0 /bucket/path/to/key GET AUTH_test bucket path/to/key test INFO: - - 02/Jun/2024/16/58/04 GET /%3Fx%3D3 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347484.844380617 1717347484.844445944 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/c HTTP/1.0 911 - - - 4 43 - - - 1.0100 - - 10000.000000000 10001.010000000 - test INFO: - - 01/Jan/1970/02/46/41 foo /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 - /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUTT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 SPECIAL /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 GET /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 PUT /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 COPY /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 HEAD /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 POST /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_method_filtering_default ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_bad ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_stat_type_good ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_log_request_statsd_invalid_stats_types ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_logger_statsd_prefix ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_multi_segment_resp ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_empty_strings ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_generator ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_no_content_length_no_transfer_encoding_with_list_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_obscure_req ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_policy_index ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_proxy_client_logging ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_req_path_info_popping ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_sensitive_headers_registered ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_unread_body ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_line ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size ... ok test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy test.unit.common.middleware.test_proxy_logging.TestProxyLogging.test_upload_size_no_policy ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_passes_through_concurrency ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_reused_logger ... ok test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes test.unit.common.middleware.test_slo.TestSloBulkDeleter.test_uses_big_max_deletes ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_bad_keys ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_multiple ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_path ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_ranges ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_last_segment ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_self_referential_non_ascii ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_size_bytes ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_undersize_segment ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range test.unit.common.middleware.test_slo.TestSloMiddleware.test_bogus_input_unsatisfiable_range ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing test.unit.common.middleware.test_slo.TestSloMiddleware.test_container_listing ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj test.unit.common.middleware.test_slo.TestSloMiddleware.test_handle_multipart_no_obj ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_PUT_env_override ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned test.unit.common.middleware.test_slo.TestSloMiddleware.test_slo_header_assigned ... ok test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input test.unit.common.middleware.test_slo.TestSloMiddleware.test_valid_input ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_bad_etag_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_failure_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_long_running_put_success_json ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_allow_empty_last_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_bad_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_check_data_bad ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_disallow_empty_first_segment ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_fast_heartbeat ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_invalid_data ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_no_xml ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_etag_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_no_early_bailout ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_skip_size_check_still_uses_min_size ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_conditional ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_success_unicode ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_manipulator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_multipart_put_with_validator_callback ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_single_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges test.unit.common.middleware.test_slo.TestSloPutManifest.test_handle_unsatisfiable_ranges ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_bad_etag_fail ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_no_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success test.unit.common.middleware.test_slo.TestSloPutManifest.test_manifest_put_with_etag_with_quotes_success ... ok test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail test.unit.common.middleware.test_slo.TestSloPutManifest.test_put_manifest_too_quick_fail ... ok test.unit.common.test_db.TestDatabaseConnectionError.test_str test.unit.common.test_db.TestDatabaseConnectionError.test_str ... ok test.unit.common.test_swob.TestTransferEncoding.test_is_chunked test.unit.common.test_swob.TestTransferEncoding.test_is_chunked ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst test.unit.common.test_utils.TestEventletRateLimiter.test_burst ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle test.unit.common.test_utils.TestEventletRateLimiter.test_burst_after_idle ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by test.unit.common.test_utils.TestEventletRateLimiter.test_incr_by ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_init test.unit.common.test_utils.TestEventletRateLimiter.test_init ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking test.unit.common.test_utils.TestEventletRateLimiter.test_non_blocking ... ok test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit test.unit.common.test_utils.TestEventletRateLimiter.test_ratelimit ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception test.unit.common.test_utils.TestSwiftLoggerAdapter.test_exception ... ok test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals test.unit.common.test_utils.TestSwiftLoggerAdapter.test_thread_locals ... ok test.unit.common.test_utils.TestWatchdog.test_start_stop test.unit.common.test_utils.TestWatchdog.test_start_stop ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency test.unit.common.test_utils.TestWatchdog.test_timeout_concurrency ... ok test.unit.common.test_utils.TestWatchdog.test_timeout_expire test.unit.common.test_utils.TestWatchdog.test_timeout_expire ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ip ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv4 ... ok test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 test.unit.common.utils.test_ipaddrs.TestIsValidIP.test_is_valid_ipv6 ... ok test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration test.unit.container.test_auditor.TestAuditorMigrations.test_db_migration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_deleted ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: ValueError("upper ('a') must be greater than or equal to lower ('z')") (txn: txe17c2c9c74d3454cbc310-00665ca49a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get namespaces from /v1/a/c/o: invalid data: KeyError('name') (txn: txfd7f64e32ebb45ea8e8b0-00665ca49b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type None (txn: txba3bac0a58d74d2080244-00665ca49b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Problem with listing response from /v1/a/c/o: ValueError('not a list') (txn: tx0953285678c241dfaceba-00665ca49b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Failed to get container listing from /v1/a/c/o: 404 (txn: tx55f8a032e53e4a15be440-00665ca49b) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Get listing from /v1/a/c?format=json&states=updating&includes=1_test {'X-Backend-Record-Type': 'shard', 'X-Backend-Record-Shard-Format': 'namespace'} (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Failed to get shard ranges from /v1/a/c/o: unexpected record type 'object' (txn: tx6056f51f5a76450db28fd-00665ca49b) test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:write:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:51 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {} test DEBUG: test:testertest:testertest:writetest:writeWRITE test DEBUG: permission denined: 403 Forbidden test:write READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:53 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Modified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_error ... ok test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_match_and_s3acl ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty ... ok test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c/b2" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c/a3" 201 - "-" "-" "-" 0.0013 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "PUT /sda1/p/a/c/b3" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:57:57 +0000] "GET /sda1/p/a/c" 200 9 "-" "-" "-" 0.0175 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0233 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0116 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0049 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0044 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0040 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1287 "-" "-" "-" 0.0040 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1274 "-" "-" "-" 0.0042 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 633 "-" "-" "-" 0.0040 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0191 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0172 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 400 9 "-" "-" "-" 0.0031 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 412 22 "-" "-" "-" 0.0002 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 507 - "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:00 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0185 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0032 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/.shards_a/c_f" 201 - "-" "-" "-" 0.0204 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/.shards_a/c_f" 202 - "-" "-" "-" 0.0164 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0113 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0040 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0037 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0039 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0037 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 328 "-" "-" "-" 0.0031 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0037 "-" 3656236 0 test DEBUG: Unlinked retiring db '/tmp/tmp_re2enl5/tmp_test_container_server_ContainerController/sda1/containers/p/189/b0d72ea4144f5f4c7f596aa4fe1f3189/b0d72ea4144f5f4c7f596aa4fe1f3189.db' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0081 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0044 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c_f" 200 656 "-" "-" "-" 0.0043 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0150 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0121 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0042 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0038 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0144 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1332 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0070 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0103 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0036 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 2662 "-" "-" "-" 0.0038 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0180 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0045 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0139 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0183 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0038 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0198 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 1645 "-" "-" "-" 0.0049 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0111 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/.shards_a/c" 202 - "-" "-" "-" 0.0196 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/.shards_a/c" 200 1653 "-" "-" "-" 0.0053 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0181 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0043 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:00 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0035 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_namespaces ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_for_shard_ranges_db_unlinked ... ok test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects test.unit.container.test_server.TestContainerController.test_GET_with_override_deleted_ignored_for_objects ... ok test.unit.container.test_server.TestContainerController.test_GET_xml test.unit.container.test_server.TestContainerController.test_GET_xml ... ok test.unit.container.test_server.TestContainerController.test_HEAD test.unit.container.test_server.TestContainerController.test_HEAD ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept test.unit.container.test_server.TestContainerController.test_HEAD_invalid_accept ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type test.unit.container.test_server.TestContainerController.test_HEAD_invalid_content_type ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format test.unit.container.test_server.TestContainerController.test_HEAD_invalid_format ... ok test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition test.unit.container.test_server.TestContainerController.test_HEAD_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_HEAD_not_found test.unit.container.test_server.TestContainerController.test_HEAD_not_found ... ok test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset test.unit.container.test_server.TestContainerController.test_HEAD_timestamp_with_offset ... ok test.unit.container.test_server.TestContainerController.test_OPTIONS test.unit.container.test_server.TestContainerController.test_OPTIONS ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_metadata ... ok test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:57:55 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'X-Amz-Copy-Source-If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Modified-Since': 'Fri, 01 Apr 2014 11:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'X-Amz-Copy-Source-If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'X-Amz-Copy-Source-If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: checking permission: src_bucket src_obj HEAD {'If-None-Match': '7dfa07a8e59ddbcd1dc84d4c4f82aea1', 'If-Unmodified-Since': 'Fri, 01 Apr 2014 12:00:00 GMT'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:tester:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket None HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:00 -0000', 'User-Agent': 'Mozzarella Foxfire'} test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl test.unit.common.middleware.s3api.test_multi_upload.TestS3ApiMultiUploadAcl.test_upload_part_copy_headers_with_not_match_and_s3acl ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_exceeded_readline ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_read_sized ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole ... ok test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline test.unit.common.middleware.test_formpost.TestCappedFileLikeObject.test_whole_readline ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_character ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_length ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_invalid_regexp ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_maximum_length_from_config ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_length_and_character ... ok test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp test.unit.common.middleware.test_name_check.TestNameCheckMiddleware.test_valid_regexp ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_configured_options ... ok test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults test.unit.common.middleware.test_name_check.TestSwiftInfo.test_registered_defaults ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_DELETE_on_expired_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_delete_latest_version_success ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_denied_DELETE_of_versioned_object ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster ... ok test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page test.unit.common.middleware.test_versioned_writes.VersionedWritesOldContainersTestCase.test_partially_upgraded_cluster_single_result_on_second_page ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon test.unit.common.test_daemon.TestWorkerDaemon.test_my_worker_daemon ... ok test.unit.common.test_daemon.TestWorkerDaemon.test_stubs test.unit.common.test_daemon.TestWorkerDaemon.test_stubs ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_admin_info_with_disallowed_sub_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections test.unit.common.test_registry.TestSwiftInfo.test_get_swift_info_with_unmatched_disallowed_sections ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_admin_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info ... ok test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils test.unit.common.test_registry.TestSwiftInfo.test_register_swift_info_import_from_utils ... ok test.unit.common.test_swob.TestStatusMap.test_status_map test.unit.common.test_swob.TestStatusMap.test_status_map ... ok test.unit.common.test_utils.TestParseContentRange.test_bad test.unit.common.test_utils.TestParseContentRange.test_bad ... ok test.unit.common.test_utils.TestParseContentRange.test_good test.unit.common.test_utils.TestParseContentRange.test_good ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_ip_specific ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_bind_to_all ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_getifaddrs ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_error ... ok test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 test.unit.common.utils.test_ipaddrs.TestWhatAreMyIPs.test_whataremyips_netifaces_ipv6 ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing test.unit.common.utils.test_libc.Test_LibcWrapper.test_argument_plumbing ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_available_function ... ok test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function test.unit.common.utils.test_libc.Test_LibcWrapper.test_unavailable_function ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_batch_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_chexor ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_create_broker ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_creation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_db_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_db_does_not_clear_particular_sharding_meta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_delete_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_double_check_trailing_delimiter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:03 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0339 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:03 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0207 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:03 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0164 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:03 +0000] "GET /sda1/p/a/c" 200 53 "-" "-" "-" 0.0090 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:03 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0017 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:04 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.4062 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:05 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.3339 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:05 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.4976 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:05 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:05 +0000] "GET /sda1/p/a/c" 200 321 "-" "-" "-" 0.0052 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:05 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0015 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:06 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 1.0079 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:07 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.6236 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:07 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.4262 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:07 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0004 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.2082 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0089 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0028 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/xmlc" 201 - "-" "-" "-" 0.0138 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/xmlc/0" 201 - "-" "-" "-" 0.0015 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/xmlc/1" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/xmlc/2" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0129 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0017 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0018 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/xmlc" 204 - "-" "-" "-" 0.0017 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "GET /sda1/p/a/xmlc" 200 545 "-" "-" "-" 0.0033 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0140 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0192 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 400 21 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 406 - "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 400 34 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0005 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0144 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0014 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0157 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0131 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0074 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0152 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:08 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.1607 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:08 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0022 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:08 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.2631 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update test.unit.container.test_server.TestContainerController.test_POST_HEAD_no_timestamp_update ... ok test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata test.unit.container.test_server.TestContainerController.test_POST_HEAD_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found test.unit.container.test_server.TestContainerController.test_POST_after_DELETE_not_found ... ok test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change test.unit.container.test_server.TestContainerController.test_POST_ignores_policy_change ... ok test.unit.container.test_server.TestContainerController.test_POST_insufficient_space test.unit.container.test_server.TestContainerController.test_POST_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to test.unit.container.test_server.TestContainerController.test_POST_invalid_container_sync_to ... ok test.unit.container.test_server.TestContainerController.test_POST_invalid_partition test.unit.container.test_server.TestContainerController.test_POST_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync test.unit.container.test_server.TestContainerController.test_POST_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float test.unit.container.test_server.TestContainerController.test_POST_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT test.unit.container.test_server.TestContainerController.test_PUT ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_object_timing_metrics ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_old_style_shard_container ... ok test INFO: - - - [02/Jun/2024:16:58:09 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.7587 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:09 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 1.4400 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.6514 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0818 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0075 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0165 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0016 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0145 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0055 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0145 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0093 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c/" 404 - "-" "-" "-" 0.0015 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:11 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0122 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0071 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:11 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "POST /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0195 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0076 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0211 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "POST /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0157 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0077 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0207 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0041 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0124 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0037 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0196 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0096 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0145 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0122 "-" 3656236 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" - "-" 3656236 - test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" - "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0143 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c/obj_0" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c/obj_1" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0092 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0051 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 255 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1683 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 332 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0061 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0024 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges test.unit.container.test_server.TestContainerController.test_PUT_GET_shard_ranges ... ok proxy-server DEBUG: Ignoring 404 from primary (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Ignoring 404 from handoff (txn: tx8767127defaa469a8b30d-00665ca492) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.11:1011/sdl, error: ERROR Insufficient Storage (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Node is error limited: 10.0.0.11:1011/sdl (txn: tx3343decd5e694d6080472-00665ca495) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.3:1003/sdd, error: ERROR Insufficient Storage (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Node is error limited: 10.0.0.3:1003/sdd (txn: tx3343decd5e694d6080472-00665ca495) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.10:1010/sdk, error: ERROR Insufficient Storage (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Node is error limited: 10.0.0.10:1010/sdk (txn: tx3343decd5e694d6080472-00665ca495) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx3343decd5e694d6080472-00665ca495) proxy-server ERROR: Node will be error limited for 60.00s: 10.0.0.5:1005/sdf, error: ERROR Insufficient Storage (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Node is error limited: 10.0.0.5:1005/sdf (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Ignoring 507 from handoff (txn: tx3343decd5e694d6080472-00665ca495) proxy-server ERROR: Object returning 503 for [507, 507, 507, 507, 507] (txn: tx3343decd5e694d6080472-00665ca495) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server ERROR: ChunkWriteTimeout feeding fragments for '/a/c/o': ChunkWriteTimeout (0.01s after 0.01s) (txn: tx0cc22e10b2034a8da8307-00665ca495) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txa2c9ef27e22a4ed3946a8-00665ca496) proxy-server ERROR: ERROR with Object server 10.0.0.4:1004/sde re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.10:1010/sdk re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.3:1003/sdd re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.5:1005/sdf re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.6:1006/sdg re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.13:1013/sdn re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.11:1011/sdl re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.9:1009/sdj re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.8:1008/sdi re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.12:1012/sdm re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.7:1007/sdh re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_if_none_match ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_simple ... ok test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 330 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 670 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1323 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 657 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1339 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 318 "-" "-" "-" 0.0060 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 324 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0051 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1017 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1006 "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1351 "-" "-" "-" 0.0053 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 677 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 676 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1021 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 685 "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 345 "-" "-" "-" 0.0021 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0018 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0018 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0038 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0035 "-" 3656236 0 test DEBUG: Unlinked retiring db '/tmp/tmpodsdwrgw/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0032 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 666 "-" "-" "-" 0.0056 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1337 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 336 "-" "-" "-" 0.0030 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1353 "-" "-" "-" 0.0034 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0066 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 1347 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 340 "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0028 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0054 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0030 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c/" 200 330 "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c/" 200 1017 "-" "-" "-" 0.0031 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_GET_shards_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata test.unit.container.test_server.TestContainerController.test_PUT_GET_sys_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container test.unit.container.test_server.TestContainerController.test_PUT_GET_to_sharding_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates test.unit.container.test_server.TestContainerController.test_PUT_HEAD_put_timestamp_updates ... ok test.unit.container.test_server.TestContainerController.test_PUT_account_update test.unit.container.test_server.TestContainerController.test_PUT_account_update ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_bad_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics test.unit.container.test_server.TestContainerController.test_PUT_container_timing_metrics ... ok test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_good_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space test.unit.container.test_server.TestContainerController.test_PUT_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition test.unit.container.test_server.TestContainerController.test_PUT_invalid_partition ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change test.unit.container.test_server.TestContainerController.test_PUT_no_policy_change ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default test.unit.container.test_server.TestContainerController.test_PUT_no_policy_for_existing_non_default ... ok proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txc222aeb3d06348c8a83d2-00665ca49a) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0141 "-" 3656236 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" - "-" 3656236 0 test INFO: - - - [01/Jan/1970:00:16:40 +0000] "GET /sda1/p/a/c" 200 664 "-" "-" "-" - "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0220 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0032 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0085 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0040 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0159 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0117 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c/unsharded" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0127 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c/sharding" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0214 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c/racing_update" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0060 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:12 +0000] "GET /sda1/p/a/c" 200 134 "-" "-" "-" 0.0256 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:12 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0185 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0183 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0166 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0018 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0155 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0091 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0095 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0260 "-" 3656236 - test ERROR: ERROR Account update failed with 127.0.0.1:52851/sda1 (will retry later): Response 503 OK test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0151 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0234 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 409 - "-" "-" "-" 0.0015 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0028 "-" 3656236 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 400 45 "-" "-" "-" 0.0004 "-" 3656236 nada test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [01/Jan/1970:00:16:40 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" - "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0058 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/./a/c" 400 20 "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0064 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0105 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0036 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0133 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0129 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0066 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified test.unit.container.test_server.TestContainerController.test_PUT_no_policy_specified ... ok test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata test.unit.container.test_server.TestContainerController.test_PUT_non_utf8_metadata ... ok test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found test.unit.container.test_server.TestContainerController.test_PUT_obj_not_found ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_old_style_shard_container ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txb7f98028cce542638851d-00665ca49f) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_different_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_some_missing ... ok test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0119 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0125 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0061 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 400 28 "-" "-" "-" 0.0022 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0023 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0005 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0109 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/bashful" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0153 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0079 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy10" 201 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy10_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy20" 301 - "-" "-" "-" 0.0050 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy30" 301 - "-" "-" "-" 0.0015 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy30_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0076 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy40" 301 - "-" "-" "-" 0.0023 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy40_no_header" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0068 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy50" 201 - "-" "-" "-" 0.0023 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy50_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy60" 301 - "-" "-" "-" 0.0031 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy60_no_header" 201 - "-" "-" "-" 0.0049 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy70" 201 - "-" "-" "-" 0.0024 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy70_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0095 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy80" 201 - "-" "-" "-" 0.0024 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpy80_no_header" 201 - "-" "-" "-" 0.0011 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0091 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpier10" 201 - "-" "-" "-" 0.0022 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:13 +0000] "PUT /sda1/p/a/c/grumpier10_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0098 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier20" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier20_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0086 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier30" 301 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier30_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier40" 301 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier40_no_header" 201 - "-" "-" "-" 0.0012 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier50" 201 - "-" "-" "-" 0.0058 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier50_no_header" 201 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0089 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier60" 301 - "-" "-" "-" 0.0021 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier60_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier70" 201 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier70_no_header" 201 - "-" "-" "-" 0.0013 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0101 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier80" 201 - "-" "-" "-" 0.0058 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/grumpier80_no_header" 201 - "-" "-" "-" 0.0011 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0117 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0202 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey10" 201 - "-" "-" "-" 0.0026 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey10_no_header" 201 - "-" "-" "-" 0.0014 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0178 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey20" 301 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey20_no_header" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0088 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey30" 301 - "-" "-" "-" 0.0028 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey30_no_header" 201 - "-" "-" "-" 0.0011 "-" 3656236 - test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard test.unit.container.test_server.TestContainerController.test_PUT_object_update_redirected_to_shard ... ok test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default test.unit.container.test_server.TestContainerController.test_PUT_proxy_default_no_policy_for_existing_default ... ok test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync test.unit.container.test_server.TestContainerController.test_PUT_reset_container_sync ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_autocreates_shard_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body test.unit.container.test_server.TestContainerController.test_PUT_shard_range_json_in_body ... ok proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9da0304c3ad84afa848ef-00665ca4a1) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server 10.0.0.14:1014/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.15:1015/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.16:1016/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.17:1017/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.18:1018/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps test.unit.proxy.controllers.test_obj.TestECObjController.test_HEAD_x_newest_with_two_vector_timestamps ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_limits_expirer_queue_deletes ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_POST_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_in_past ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_delete_at_non_integer ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_limits_expirer_queue_deletes ... ok test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0108 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey40" 301 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0090 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey50" 201 - "-" "-" "-" 0.0047 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey50_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey60" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey60_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0135 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey70" 201 - "-" "-" "-" 0.0020 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0084 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:14 +0000] "PUT /sda1/p/a/c/dopey80" 201 - "-" "-" "-" 0.0022 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/dopey80_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/bashful" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/dopey" 301 - "-" "-" "-" 0.0015 "-" 3656236 - test DEBUG: Unlinked retiring db '/tmp/tmpw1b8ihvg/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0080 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest10" 201 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest10_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0062 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest20" 301 - "-" "-" "-" 0.0019 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest20_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0113 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest30" 301 - "-" "-" "-" 0.0018 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest30_no_header" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0146 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest40" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest40_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0157 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest50" 201 - "-" "-" "-" 0.0055 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest50_no_header" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0115 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest60" 301 - "-" "-" "-" 0.0016 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest60_no_header" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0087 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest70" 201 - "-" "-" "-" 0.0034 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest70_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0085 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest80" 201 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c/grumpiest80_no_header" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0155 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0078 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0020 "-" 3656236 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0274 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0040 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0104 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/.shards_a/shard_c" 404 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/.shards_a/shard_c" 400 49 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/.shards_a/shard_c" 201 - "-" "-" "-" 0.0443 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0179 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/.shards_a/shard_c" 202 - "-" "-" "-" 0.0086 "-" 3656236 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 404 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:15 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0162 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0202 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0093 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0166 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0128 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 400 74 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 400 69 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 400 75 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 400 35 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0102 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0100 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container test.unit.container.test_server.TestContainerController.test_PUT_shard_range_to_deleted_container ... ok test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race test.unit.container.test_server.TestContainerController.test_PUT_simulated_create_race ... ok test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float test.unit.container.test_server.TestContainerController.test_PUT_timestamp_not_float ... ok test.unit.container.test_server.TestContainerController.test_PUT_utf8 test.unit.container.test_server.TestContainerController.test_PUT_utf8 ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works test.unit.container.test_server.TestContainerController.test_REPLICATE_complete_rsync_works ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space test.unit.container.test_server.TestContainerController.test_REPLICATE_insufficient_space ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works test.unit.container.test_server.TestContainerController.test_REPLICATE_rsync_then_merge_works ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync test.unit.container.test_server.TestContainerController.test_REPLICATE_unknown_sync ... ok test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works test.unit.container.test_server.TestContainerController.test_REPLICATE_value_error_works ... ok test.unit.container.test_server.TestContainerController.test_UPDATE test.unit.container.test_server.TestContainerController.test_UPDATE ... ok test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate test.unit.container.test_server.TestContainerController.test_UPDATE_autocreate ... ok test.unit.container.test_server.TestContainerController.test__call__raise_timeout test.unit.container.test_server.TestContainerController.test__call__raise_timeout ... ok test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted test.unit.container.test_server.TestContainerController.test_account_update_account_override_deleted ... ok test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device test.unit.container.test_server.TestContainerController.test_account_update_mismatched_host_device ... ok test.unit.container.test_server.TestContainerController.test_acl_container test.unit.container.test_server.TestContainerController.test_acl_container ... ok test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_storage_policy_via_DELETE_then_PUT ... ok proxy-server ERROR: ERROR with Object server 10.0.0.19:1019/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.20:1020/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.21:1021/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.22:1022/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.23:1023/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.24:1024/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.25:1025/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.26:1026/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server ERROR: ERROR with Object server 10.0.0.27:1027/sda re: Trying to HEAD /v1/a/c/o: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx4211759d4df740ec9161f-00665ca4a3) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_negative_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_non_int_delete_after ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length test.unit.proxy.controllers.test_obj.TestECObjController.test_PUT_requires_length ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_connect_put_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete test.unit.proxy.controllers.test_obj.TestECObjController.test_container_sync_delete ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_container_update_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_delete_at_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations test.unit.proxy.controllers.test_obj.TestECObjController.test_determine_chunk_destinations ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders test.unit.proxy.controllers.test_obj.TestECObjController.test_ec_concurrent_GET_with_slow_leaders ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates test.unit.proxy.controllers.test_obj.TestECObjController.test_expirer_DELETE_suppresses_expirer_queue_updates ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries test.unit.proxy.controllers.test_obj.TestECObjController.test_feed_remaining_primaries ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD test.unit.proxy.controllers.test_obj.TestECObjController.test_fix_response_HEAD ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_handoff_local_first_non_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_best_effort ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_moves_locals_first ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity test.unit.proxy.controllers.test_obj.TestECObjController.test_iter_nodes_local_handoff_first_noops_when_no_affinity ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket test.unit.proxy.controllers.test_obj.TestECObjController.test_non_durable_ec_response_bucket ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0239 "-" 3656236 0 test DEBUG: Unlinked retiring db '/tmp/tmpxce_1kfi/tmp_test_container_server_ContainerController/sda1/containers/p/872/5267a50d4a8f6ea3ee1dbecb8592e872/5267a50d4a8f6ea3ee1dbecb8592e872.db' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0149 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0027 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0181 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0144 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0054 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0166 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0025 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0126 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 400 26 "-" "-" "-" 0.0001 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/b%27%5Cxe2%5Cx98%5Cx83%27" 201 - "-" "-" "-" 0.0164 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:58:16 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0005 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:58:16 +0000] "REPLICATE /sda1/p/a/" 204 - "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:58:16 +0000] "REPLICATE /sda1/p/a/" 404 - "-" "-" "-" 0.0005 "-" 3656236 - test ERROR: ERROR __call__ error with REPLICATE /sda1/p/a/ : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 1519, in _timing_stats resp = func(ctrl, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 949, in REPLICATE ret = self.replicator_rpc.dispatch(post_args, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/db_replicator.py", line 897, in dispatch return getattr(self, op)( ^^^^^^^^^^^^^^^^^ AttributeError: 'ContainerReplicatorRpc' object has no attribute 'unknown_sync' test DEBUG: - - - [02/Jun/2024:16:58:16 +0000] "REPLICATE /sda1/p/a/" 500 847 "-" "-" "-" 0.0044 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: - - - [02/Jun/2024:16:58:16 +0000] "REPLICATE /sda1/p/a/" 400 49 "-" "-" "-" 0.0019 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0162 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "UPDATE /sda1/p/a/c" 400 41 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "UPDATE /sda1/p/a/c" 202 - "-" "-" "-" 0.0131 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/a/c" 200 131 "-" "-" "-" 0.0033 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/.a/c" 404 - "-" "-" "-" 0.0006 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "UPDATE /sda1/p/.a/c" 202 - "-" "-" "-" 0.0271 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "GET /sda1/p/.a/c" 200 131 "-" "-" "-" 0.0033 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR __call__ error with PUT /sda1/p/a/c : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 1025, in __call__ res = getattr(self, req.method)(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_server.py", line 6118, in mock_put_method raise Exception() Exception test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0159 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR Account update failed: different numbers of hosts and devices in request: "127.0.0.1:0" vs "sda1,sda2" test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0149 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0193 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0167 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c2" 201 - "-" "-" "-" 0.0208 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/c2" 204 - "-" "-" "-" 0.0018 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0183 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0103 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0175 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0024 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT test.unit.container.test_server.TestContainerController.test_change_to_default_storage_policy_via_DELETE_then_PUT ... ok test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD test.unit.container.test_server.TestContainerController.test_content_type_on_HEAD ... ok test.unit.container.test_server.TestContainerController.test_correct_allowed_method test.unit.container.test_server.TestContainerController.test_correct_allowed_method ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_empty_shard_container ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container test.unit.container.test_server.TestContainerController.test_create_reserved_namespace_container ... ok test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container test.unit.container.test_server.TestContainerController.test_create_reserved_object_in_container ... ok test.unit.container.test_server.TestContainerController.test_creation test.unit.container.test_server.TestContainerController.test_creation ... ok test.unit.container.test_server.TestContainerController.test_delete_auto_create test.unit.container.test_server.TestContainerController.test_delete_auto_create ... ok test.unit.container.test_server.TestContainerController.test_deleted_headers test.unit.container.test_server.TestContainerController.test_deleted_headers ... ok test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved test.unit.container.test_server.TestContainerController.test_delimiter_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index test.unit.container.test_server.TestContainerController.test_get_and_validate_policy_index ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_false ... ok test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true test.unit.container.test_server.TestContainerController.test_insufficient_storage_mount_check_true ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist test.unit.container.test_server.TestContainerController.test_invalid_method_doesnt_exist ... ok test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public test.unit.container.test_server.TestContainerController.test_invalid_method_is_not_public ... ok test.unit.container.test_server.TestContainerController.test_list_allowed_methods test.unit.container.test_server.TestContainerController.test_list_allowed_methods ... ok test.unit.container.test_server.TestContainerController.test_listing_with_reserved test.unit.container.test_server.TestContainerController.test_listing_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_log_line_format test.unit.container.test_server.TestContainerController.test_log_line_format ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_enable_sharding ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: ERROR with Object server :/ re: Expect: 100-continue on /a/c/o: ConnectionTimeout (0.05s) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default test.unit.proxy.controllers.test_obj.TestECObjController.test_object_DELETE_backend_update_container_ip_default ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_short_resp_timeout ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp test.unit.proxy.controllers.test_obj.TestECObjController.test_ranged_get_with_slow_resp ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip test.unit.proxy.controllers.test_obj.TestECObjController.test_repl_object_DELETE_backend_update_container_repl_ip ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests test.unit.proxy.controllers.test_obj.TestECObjController.test_smooth_distributed_backend_requests ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s test.unit.proxy.controllers.test_obj.TestECObjController.test_some_404s_and_507s ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_not_configured ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_exception ... ok test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0371 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0144 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0188 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/o" 201 - "-" "-" "-" 0.0175 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0025 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0023 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0024 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "HEAD /sda1/p/a/o" 204 - "-" "-" "-" 0.0022 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/%00c" 201 - "-" "-" "-" 0.0153 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/%00c%00test" 201 - "-" "-" "-" 0.0177 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c/" 201 - "-" "-" "-" 0.0156 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:16 +0000] "PUT /sda1/p/a/c/%00o" 400 61 "-" "-" "-" 0.0001 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/.a/c/o" 204 - "-" "-" "-" 0.0162 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/.c/.o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0188 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:16 +0000] "DELETE /sda1/p/.shards_a/c/o" 204 - "-" "-" "-" 0.0139 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0168 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0132 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0022 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 0.0018 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0178 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0017 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0171 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 26 "-" "-" "-" 0.0035 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 32 "-" "-" "-" 0.0035 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/container_default" 201 - "-" "-" "-" 0.0152 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "HEAD /sda1/p/a/container_default" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c_nada" 400 45 "-" "-" "-" 0.0003 "-" 3656236 nada test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c_999" 400 42 "-" "-" "-" 0.0003 "-" 3656236 999 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c_nulo" 201 - "-" "-" "-" 0.0189 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "HEAD /sda1/p/a/c_nulo" 204 - "-" "-" "-" 0.0019 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c_unu" 201 - "-" "-" "-" 0.0190 "-" 3656236 1 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "HEAD /sda1/p/a/c_unu" 204 - "-" "-" "-" 0.0021 "-" 3656236 1 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "method_doesnt_exist /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "__init__ /sda1/p/a/c" 405 - "-" "-" "-" 0.0003 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0179 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0171 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 183 "-" "-" "-" 0.0036 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: 1.2.3.4 - - [01/Jan/1970:02:46:42 +0000] "HEAD /sda1/p/a/c" 404 - "-" "-" "-" 2.0000 "-" 1234 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0258 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test.unit.container.test_server.TestContainerController.test_markers_with_reserved test.unit.container.test_server.TestContainerController.test_markers_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_not_allowed_method test.unit.container.test_server.TestContainerController.test_not_allowed_method ... ok test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps test.unit.container.test_server.TestContainerController.test_object_update_with_multiple_timestamps ... ok test.unit.container.test_server.TestContainerController.test_object_update_with_offset test.unit.container.test_server.TestContainerController.test_object_update_with_offset ... ok test.unit.container.test_server.TestContainerController.test_params_format test.unit.container.test_server.TestContainerController.test_params_format ... ok test.unit.container.test_server.TestContainerController.test_params_utf8 test.unit.container.test_server.TestContainerController.test_params_utf8 ... ok test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_and_delim_with_reserved ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_newid ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.7:1007/sdh (txn: tx5fa4ec5086284310a1235-00665ca4a8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.5:1005/sdf (txn: tx0dba4ddb52d84a4093128-00665ca4a8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 10.0.0.7:1007/sdh (txn: tx74c5947a6310442789d4d-00665ca4a8) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '5', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_per_policy_config_overrides_and_inherits ... ok test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config test.unit.proxy.controllers.test_obj.TestECObjController.test_write_affinity_proxy_server_config ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_empty_shard_container ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 0 (ec): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r1', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for policy 1 (unu): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '3 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': 'r0', 'write_affinity_node_count': '1 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_root_container ... ok test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0322 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0052 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0099 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 184 "-" "-" "-" 0.0047 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0208 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0026 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0150 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0171 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0157 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0010 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0076 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0169 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0158 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0215 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0048 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 127 "-" "-" "-" 0.0202 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0163 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "HEAD /sda1/p/a/c" 204 - "-" "-" "-" 0.0017 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 117 "-" "-" "-" 0.0170 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0012 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 116 "-" "-" "-" 0.0166 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0145 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 120 "-" "-" "-" 0.0158 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "DELETE /sda1/p/a/c/o" 204 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0104 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0184 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 61 "-" "-" "-" 0.0033 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 200 2 "-" "-" "-" 0.0030 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 37 "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 33 "-" "-" "-" 0.0001 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 32 "-" "-" "-" 0.0001 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 38 "-" "-" "-" 0.0001 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 400 34 "-" "-" "-" 0.0001 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0132 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0100 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0031 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0029 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/c" 204 - "-" "-" "-" 0.0030 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0215 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0008 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0007 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0226 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 207 "-" "-" "-" 0.0062 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null" 201 - "-" "-" "-" 0.0136 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test01" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00test02" 201 - "-" "-" "-" 0.0011 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00null%00foo" 201 - "-" "-" "-" 0.0021 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/%00null/%00nullish" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 2 "-" "-" "-" 0.0214 "-" 3656236 0 test.unit.container.test_server.TestContainerController.test_prefix_with_reserved test.unit.container.test_server.TestContainerController.test_prefix_with_reserved ... ok test.unit.container.test_server.TestContainerController.test_put_auto_create test.unit.container.test_server.TestContainerController.test_put_auto_create ... ok test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods test.unit.container.test_server.TestContainerController.test_replication_server_call_all_methods ... ok test.unit.container.test_server.TestContainerController.test_serv_reserv test.unit.container.test_server.TestContainerController.test_serv_reserv ... ok test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type test.unit.container.test_server.TestContainerController.test_swift_bytes_in_content_type ... ok test.unit.container.test_server.TestContainerController.test_through_call test.unit.container.test_server.TestContainerController.test_through_call ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path test.unit.container.test_server.TestContainerController.test_through_call_invalid_path ... ok test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 test.unit.container.test_server.TestContainerController.test_through_call_invalid_path_utf8 ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE test.unit.container.test_server.TestContainerController.test_update_sync_store_on_DELETE ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST test.unit.container.test_server.TestContainerController.test_update_sync_store_on_POST ... ok test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT test.unit.container.test_server.TestContainerController.test_update_sync_store_on_PUT ... ok test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers test.unit.container.test_server.TestContainerController.test_updating_multiple_container_servers ... ok test.unit.container.test_server.TestContainerController.test_weird_content_types test.unit.container.test_server.TestContainerController.test_weird_content_types ... ok test.unit.container.test_sync.TestContainerSync.test_FileLikeIter test.unit.container.test_sync.TestContainerSync.test_FileLikeIter ... ok test.unit.container.test_sync.TestContainerSync.test_container_first_loop test.unit.container.test_sync.TestContainerSync.test_container_first_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_report test.unit.container.test_sync.TestContainerSync.test_container_report ... ok test.unit.container.test_sync.TestContainerSync.test_container_second_loop test.unit.container.test_sync.TestContainerSync.test_container_second_loop ... ok test.unit.container.test_sync.TestContainerSync.test_container_stop_at test.unit.container.test_sync.TestContainerSync.test_container_stop_at ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted test.unit.container.test_sync.TestContainerSync.test_container_sync_deleted ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db test.unit.container.test_sync.TestContainerSync.test_container_sync_missing_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key test.unit.container.test_sync.TestContainerSync.test_container_sync_no_to_or_key ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_db ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db test.unit.container.test_sync.TestContainerSync.test_container_sync_not_my_db ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_enable_sharding ... ok test INFO: - - - [02/Jun/2024:16:58:17 +0000] "GET /sda1/p/a/%00null" 200 367 "-" "-" "-" 0.0039 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 404 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/.a/c/o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/.a/c/o" 201 - "-" "-" "-" 0.0171 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/.c/o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/.o" 404 - "-" "-" "-" 0.0003 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/a/c/o" 201 - "-" "-" "-" 0.0169 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/.shards_a/c/o" 404 - "-" "-" "-" 0.0004 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/.shards_a/c" 404 - "-" "-" "-" 0.0003 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:17 +0000] "PUT /sda1/p/.shards_a/c" 201 - "-" "-" "-" 0.0151 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0179 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c/o1" 201 - "-" "-" "-" 0.0009 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c/o2" 201 - "-" "-" "-" 0.0005 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:18 +0000] "GET /sda1/p/a/c" 200 280 "-" "-" "-" 0.0142 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "GET /sda1/p/a/c" 404 - "-" "-" "-" 0.0009 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "GET /bob" 400 18 "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "GET /sda1/p/a/c%D8%3E%2520/%25" 412 29 "-" "-" "-" 0.0004 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0287 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:18 +0000] "DELETE /sda1/p/a/c" 204 - "-" "-" "-" 0.0347 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0210 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:18 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0192 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:18 +0000] "POST /sda1/p/a/c" 204 - "-" "-" "-" 0.0166 "-" 3656236 - test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0257 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 202 - "-" "-" "-" 0.0195 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test ERROR: ERROR account update failed with 1.2.3.4:5/sdb1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test ERROR: ERROR account update failed with 6.7.8.9:10/sdf1 (will retry later): Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/server.py", line 270, in account_update account_response = conn.getresponse() ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'getresponse' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0175 "-" 3656236 0 test DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c" 201 - "-" "-" "-" 0.0223 "-" 3656236 0 test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c/0" 201 - "-" "-" "-" 0.0021 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:18 +0000] "PUT /sda1/p/a/c/1" 201 - "-" "-" "-" 0.0006 "-" 3656236 - test INFO: - - - [02/Jun/2024:16:58:18 +0000] "GET /sda1/p/a/c" 200 248 "-" "-" "-" 0.0143 "-" 3656236 0 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717347498.3659756, time window end: 1717347498.3659756, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 5, sync_point2: None, total_rows: 1 test-container-sync ERROR: ERROR Syncing : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 388, in container_sync if not self.container_sync_row( ^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: TestContainerSync.test_container_report..fake_container_sync_row() missing 1 required positional argument: 'realm_key' test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync INFO: Container sync report: a/c, time window start: 1717347498.370225, time window end: 1717347498.3702402, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync ERROR: ERROR Syncing {'ROWID': 1, 'name': 'o'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 534, in container_sync_row row['created_at']) ~~~^^^^^^^^^^^^^^ KeyError: 'created_at' test-container-sync INFO: Container sync report: a/c, time window start: 1717347498.3704448, time window end: 1717347498.3709173, puts: 0, posts: 0, deletes: 0, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync INFO: Container sync report: a/c, time window start: 1717347498.3710842, time window end: 1717347498.371176, puts: 0, posts: 0, deletes: 1, bytes: 0, sync_point1: 1, sync_point2: -1, total_rows: 1 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for optionstest.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_delete_using_realms ... ok test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_exception ... ok test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 883, in fake_delete_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': True, 'name': 'object', 'created_at': '1.2'}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 543, in container_sync_row delete_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 900, in fake_delete_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test-container-sync DEBUG: Could not load '/etc/swift/container-sync-realms.conf': [Errno 2] No such file or directory: '/etc/swift/container-sync-realms.conf' test-container-sync WARNING: Configuration option internal_client_conf_path not defined. Using default configuration, See internal-client.conf-sample for options test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 70}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1146, in fake_get_object raise exc[-1] Exception: test exception test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 80}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 595, in container_sync_row raise exc File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 572, in container_sync_row self.swift.get_object(info['account'], File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1173, in fake_get_object raise exc[-1] swift.common.exceptions.ClientException: test client exception test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync INFO: Not found 'a/c' => 'http://sync/to/path' - object 'object' test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms test.unit.container.test_sync.TestContainerSync.test_container_sync_row_put_using_realms ... ok test.unit.container.test_sync.TestContainerSync.test_run_forever test.unit.container.test_sync.TestContainerSync.test_run_forever ... ok test.unit.container.test_sync.TestContainerSync.test_run_once test.unit.container.test_sync.TestContainerSync.test_run_once ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_None ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_multiple ... ok test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one test.unit.container.test_sync.TestContainerSync.test_select_http_proxy_one ... ok test.unit.obj.test_updater.TestObjectUpdater.test_conf_params test.unit.obj.test_updater.TestObjectUpdater.test_conf_params ... ok test.unit.obj.test_updater.TestObjectUpdater.test_creation test.unit.obj.test_updater.TestObjectUpdater.test_creation ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_with_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception test.unit.obj.test_updater.TestObjectUpdater.test_listdir_without_exception ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_bad_location_header ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_missing_location_header ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_root_update_redirected_previous_success ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges ... ok test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'size': 50}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 612, in container_sync_row put_object(sync_to, name=row['name'], headers=headers, File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1239, in fake_put_object raise ClientException('test client exception', http_status=503) swift.common.exceptions.ClientException: test client exception: 503 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.20000 test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.30000 test-container-sync INFO: Unauth 'a/c' => 'http://sync/to/path' test-container-sync ERROR: ERROR Syncing {'deleted': False, 'name': 'object', 'created_at': '0000000001.20000', 'etag': '1111', 'size': 10}: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 558, in container_sync_row if self._object_in_remote_container(row['name'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/sync.py", line 490, in _object_in_remote_container metadata, _ = head_object(sync_to, name=name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sync.py", line 1338, in fake_head_object raise Exception() Exception test-container-sync DEBUG: remote obj timestamp 0000000001.20000 local obj 0000000001.10000 test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test WARNING: The slowdown option is deprecated in favor of objects_per_second. This option may be ignored in a future release. test ERROR: ERROR: Unable to access foo/bar: permission_denied test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpkkxrdufa/devices/sda1 (pid: 3656236) test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test ERROR: Container update failed for '/a/c/o'; problem with redirect location: Invalid path: bad%20bad%20bad test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpkkxrdufa/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347499.00000 test INFO: Object update sweep completed on /tmp/tmpkkxrdufa/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp8o5q5aal/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmp8o5q5aal/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347499.00000 test INFO: Object update sweep completed on /tmp/tmp8o5q5aal/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpxkqiualr/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmpxkqiualr/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_new test DEBUG: Update sent for /.shards_a/c_shard_new/o /tmp/tmpxkqiualr/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 test INFO: Object update sweep completed on /tmp/tmpxkqiualr/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp9qffpu86/devices/sda1 (pid: 3656236) test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 507 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmp9qffpu86/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 test INFO: Object update sweep completed on /tmp/tmp9qffpu86/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp9qffpu86/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmp9qffpu86/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_1 test DEBUG: Update sent for /.shards_a/c_shard_1/o /tmp/tmp9qffpu86/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 test INFO: Object update sweep completed on /tmp/tmp9qffpu86/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 1 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpoliervd9/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_older/o /tmp/tmpoliervd9/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_new test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_new/o /tmp/tmpoliervd9/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_newer test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_shard_update_redirected_twice ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_update_redirection_loop ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_errors ... ok test INFO: Object update sweep completed on /tmp/tmpoliervd9/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpoliervd9/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /.shards_a/c_shard_newer/o /tmp/tmpoliervd9/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 test INFO: Object update sweep completed on /tmp/tmpoliervd9/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1itvge54/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_2 test INFO: Object update sweep completed on /tmp/tmp1itvge54/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1itvge54/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_2/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to None test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /a/c/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_3 test INFO: Object update sweep completed on /tmp/tmp1itvge54/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1itvge54/devices/sda1 (pid: 3656236) test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_3/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to .shards_a/c_shard_1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Error code 301 is returned from remote server 127.0.0.1:67890/sda1 test DEBUG: Update redirected for /.shards_a/c_shard_1/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 to None test INFO: Object update sweep completed on /tmp/tmp1itvge54/devices/sda1 in 0.02s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 2 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp1itvge54/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp1itvge54/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347498.00000 test INFO: Object update sweep completed on /tmp/tmp1itvge54/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_g01q5r8/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /.shards_a/shard_c/o /tmp/tmp_g01q5r8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347499.00000 test INFO: Object update sweep completed on /tmp/tmp_g01q5r8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_g01q5r8/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp_g01q5r8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.00000 test INFO: Object update sweep completed on /tmp/tmp_g01q5r8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_async_updates ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_g01q5r8/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp_g01q5r8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347501.00000 test INFO: Object update sweep completed on /tmp/tmp_g01q5r8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_g01q5r8/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp_g01q5r8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347502.00000 test INFO: Object update sweep completed on /tmp/tmp_g01q5r8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_g01q5r8/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp_g01q5r8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347503.00000 test INFO: Object update sweep completed on /tmp/tmp_g01q5r8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp_g01q5r8/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp_g01q5r8/devices/sda1/async_pending-1/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347504.00000 test INFO: Object update sweep completed on /tmp/tmp_g01q5r8/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp3gicmj5u/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp3gicmj5u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.00000 test INFO: Object update sweep completed on /tmp/tmp3gicmj5u/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp3gicmj5u/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmp3gicmj5u/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347502.00000 test INFO: Object update sweep completed on /tmp/tmp3gicmj5u/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates test.unit.obj.test_updater.TestObjectUpdater.test_obj_put_legacy_updates ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_gone_missing ... ok test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine test.unit.obj.test_updater.TestObjectUpdater.test_obj_update_quarantine ... ok test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep test.unit.obj.test_updater.TestObjectUpdater.test_object_sweep ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_2_skip_1 ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_defer_3_skip_1 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_object_stats_shard_container ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_some_limited ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_old_style_root_account_container_path ... ok test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unlimited ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpgau7z2a3/devices/sda1 (pid: 3656236) test ERROR: ERROR Pickle problem, quarantining /tmp/tmpgau7z2a3/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347499.00000: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/obj/updater.py", line 431, in _load_update return pickle.load(open(update_path, 'rb')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _pickle.UnpicklingError: invalid load key, '\xff'. test INFO: Object update sweep completed on /tmp/tmpgau7z2a3/devices/sda1 in 0.01s seconds:, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 0 failures, 1 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpxht012dp/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmpxht012dp/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717347499.38017 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpxht012dp/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717347499.37809 test INFO: Object update sweep completed on /tmp/tmpxht012dp/devices/sda1 in 0.21s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.21s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 9 skips, 9 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp0la3hdpn/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c1/o00 /tmp/tmp0la3hdpn/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717347499.59825 test DEBUG: Update sent for /a/c1/o03 /tmp/tmp0la3hdpn/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717347499.59898 test DEBUG: Update sent for /a/c1/o01 /tmp/tmp0la3hdpn/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717347499.59850 test INFO: Object update sweep completed on /tmp/tmp0la3hdpn/devices/sda1 in 0.10s seconds:, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.11s, 3 successes, 0 failures, 0 quarantines, 3 unlinks, 0 errors, 0 redirects, 1 skips, 2 deferrals, 1 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpev2zntwu/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c1/o04 /tmp/tmpev2zntwu/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717347499.71341 test DEBUG: Update sent for /a/c1/o00 /tmp/tmpev2zntwu/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717347499.71247 test DEBUG: Update sent for /a/c1/o03 /tmp/tmpev2zntwu/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717347499.71319 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpev2zntwu/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717347499.71297 test INFO: Object update sweep completed on /tmp/tmpev2zntwu/devices/sda1 in 0.01s seconds:, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 1 skips, 3 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpdhd_b_o1/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c1/o01 /tmp/tmpdhd_b_o1/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717347499.72953 test DEBUG: Update sent for /a/c1/o00 /tmp/tmpdhd_b_o1/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717347499.72922 test INFO: Object update sweep completed on /tmp/tmpdhd_b_o1/devices/sda1 in 0.11s seconds:, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.11s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 2 skips, 2 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmp7kyogz4y/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c1/o02 /tmp/tmp7kyogz4y/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717347499.84958 test DEBUG: Update sent for /a/c1/o06 /tmp/tmp7kyogz4y/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717347499.85047 test DEBUG: Update sent for /a/c1/o07 /tmp/tmp7kyogz4y/devices/sda1/async_pending/fc8/26ac197975ac1169d403eeb7ae8bbfc8-1717347499.85069 test DEBUG: Update sent for /a/c1/o03 /tmp/tmp7kyogz4y/devices/sda1/async_pending/ff2/d12c716df8024e3a557661979780dff2-1717347499.84980 test DEBUG: Update sent for /a/c1/o05 /tmp/tmp7kyogz4y/devices/sda1/async_pending/dc4/017ff741f591e7220e0836a0c4fcedc4-1717347499.85024 test DEBUG: Update sent for /a/c1/o00 /tmp/tmp7kyogz4y/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717347499.84907 test DEBUG: Update sent for /a/c1/o08 /tmp/tmp7kyogz4y/devices/sda1/async_pending/5b8/4e594cf882aac2f7f8f2f4cdade285b8-1717347499.85284 test DEBUG: Update sent for /.shards_a/c2_shard/o09 /tmp/tmp7kyogz4y/devices/sda1/async_pending/ca9/43aefe16fd826546f72b52db0520cca9-1717347499.85376 test DEBUG: Update sent for /a/c1/o01 /tmp/tmp7kyogz4y/devices/sda1/async_pending/22b/9f78f96341102120ea7474897595a22b-1717347499.84934 test DEBUG: Update sent for /a/c1/o09 /tmp/tmp7kyogz4y/devices/sda1/async_pending/bee/db77f283b4b1bb5e5b6ce501ca337bee-1717347499.85311 test DEBUG: Update sent for /a/c1/o04 /tmp/tmp7kyogz4y/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717347499.85002 test INFO: Object update sweep completed on /tmp/tmp7kyogz4y/devices/sda1 in 0.21s seconds:, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.21s, 11 successes, 0 failures, 0 quarantines, 11 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpwt784pxp/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c1/o04 /tmp/tmpwt784pxp/devices/sda1/async_pending/016/7bf5c8c93f8043f6705026176f67c016-1717347500.07494 test DEBUG: Update sent for /a/c1/o05 /tmp/tmpwt784pxp/devices/sda1/async_pending/dc4/017ff741f591e7220e0836a0c4fcedc4-1717347500.07515 test DEBUG: Update sent for /a/c1/o06 /tmp/tmpwt784pxp/devices/sda1/async_pending/5b6/29e303e1cb80d29563757ae87f5c55b6-1717347500.07542 test DEBUG: Update sent for /a/c1/o00 /tmp/tmpwt784pxp/devices/sda1/async_pending/a20/8110018052c71b62f4f93b1cd45fda20-1717347500.07398 test DEBUG: Update sent for /a/c1/o02 /tmp/tmpwt784pxp/devices/sda1/async_pending/5d9/74b39c6818abfa3cdadce7038f4a15d9-1717347500.07446 test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals test.unit.obj.test_updater.TestObjectUpdater.test_per_container_rate_limit_unsent_deferrals ... ok test.unit.obj.test_updater.TestObjectUpdater.test_run_once test.unit.obj.test_updater.TestObjectUpdater.test_run_once ... ok test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted test.unit.obj.test_updater.TestObjectUpdater.test_run_once_with_disk_unmounted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_db_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_path ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_errors ... ok test INFO: Object update sweep completed on /tmp/tmpwt784pxp/devices/sda1 in 0.01s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 2 skips, 4 deferrals, 2 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test ERROR: ERROR with remote server 127.0.0.1:67890/sda1: Connection refused test DEBUG: Update failed for /a/c/o /tmp/tmpfev1xhzp/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.10920 test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test DEBUG: Error code 500 is returned from remote server 127.0.0.1:34925/sda1 test DEBUG: Error code 500 is returned from remote server 127.0.0.1:34925/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpfev1xhzp/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.10920 test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.02s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test DEBUG: Error code 404 is returned from remote server 127.0.0.1:34925/sda1 test DEBUG: Update failed for /a/c/o /tmp/tmpfev1xhzp/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.10920 test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.02s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test INFO: Timeout waiting on remote server 127.0.0.1:34925/sda1: 99 seconds test DEBUG: Update failed for /a/c/o /tmp/tmpfev1xhzp/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.10920 test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test INFO: Timeout connecting to remote server 127.0.0.1:34925/sda1: 9 seconds test DEBUG: Update failed for /a/c/o /tmp/tmpfev1xhzp/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.10920 test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.01s seconds:, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test INFO: Object update sweep starting on /tmp/tmpfev1xhzp/devices/sda1 (pid: 3656236) test DEBUG: Update sent for /a/c/o /tmp/tmpfev1xhzp/devices/sda1/async_pending/a83/06fbf0b514e5199dfc4e00f42eb5ea83-1717347500.10920 test INFO: Object update sweep completed on /tmp/tmpfev1xhzp/devices/sda1 in 0.01s seconds:, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update single-threaded sweep completed: 0.01s, 1 successes, 0 failures, 0 quarantines, 1 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test INFO: Begin object update single threaded sweep test WARNING: Skipping: test INFO: Object update single-threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/object.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/object.recon' test INFO: Object update sweep starting on /tmp/tmpmp1bnhf3/devices/sda1 (pid: 3656236) test INFO: Object update sweep progress on /tmp/tmpmp1bnhf3/devices/sda1: 16.00s, 2 successes, 0 failures, 0 quarantines, 2 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update sweep progress on /tmp/tmpmp1bnhf3/devices/sda1: 32.00s, 4 successes, 0 failures, 0 quarantines, 4 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test INFO: Object update sweep completed on /tmp/tmpmp1bnhf3/devices/sda1 in 44.00s seconds:, 5 successes, 0 failures, 0 quarantines, 5 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info ... ok test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies test.unit.obj.test_updater.TestObjectUpdater.test_sweep_logs_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_find_shard_ranges_with_misplaced_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object ... ok test.unit.proxy.test_mem_server.TestAccountController.test_DELETE test.unit.proxy.test_mem_server.TestAccountController.test_DELETE ... ok test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string test.unit.proxy.test_mem_server.TestAccountController.test_DELETE_with_query_string ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_brokers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_sharding_states ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET test.unit.proxy.test_mem_server.TestAccountController.test_GET ... ok test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_GET_autocreate ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_info_stale_read_ok ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD test.unit.proxy.test_mem_server.TestAccountController.test_HEAD ... ok test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_HEAD_autocreate ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_db_state ... ok test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS test.unit.proxy.test_mem_server.TestAccountController.test_OPTIONS ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_items_since ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta test.unit.proxy.test_mem_server.TestAccountController.test_POST_autocreate_with_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_bad_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata test.unit.proxy.test_mem_server.TestAccountController.test_POST_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT test.unit.proxy.test_mem_server.TestAccountController.test_PUT ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_bad_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions test.unit.proxy.test_mem_server.TestAccountController.test_PUT_connect_exceptions ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length test.unit.proxy.test_mem_server.TestAccountController.test_PUT_max_account_name_length ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata test.unit.proxy.test_mem_server.TestAccountController.test_PUT_metadata ... ok test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status test.unit.proxy.test_mem_server.TestAccountController.test_PUT_status ... ok test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json test.unit.proxy.test_mem_server.TestAccountController.test_auto_create_account_listing_response_is_json ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused test.unit.proxy.test_mem_server.TestAccountController.test_connection_refused ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_is_deleted ... ok test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error test.unit.proxy.test_mem_server.TestAccountController.test_other_socket_error ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header test.unit.proxy.test_mem_server.TestAccountController.test_response_head_accept_ranges_header ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_root_container_fill_gap ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_no_stale_reads ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_namespaces_state_filtering ... ok test INFO: Object update sweep starting on /tmp/tmps0q0u0ci/devices/sda1 (pid: 3656236) test INFO: Object update sweep completed on /tmp/tmps0q0u0ci/devices/sda1 in 0.13s seconds:, 6 successes, 0 failures, 0 quarantines, 6 unlinks, 0 errors, 0 redirects, 0 skips, 0 deferrals, 0 drains (pid: 3656236) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmps7_5032t/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmps7_5032t/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmps7_5032t/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmps7_5032t/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0146 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0191 "-" 3656236 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0255 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0144 "-" 3656236 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "HEAD /sda1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx0efafba1af8c499f9392f-00665ca4ac" "proxy-server 3656236" 0.0022 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:57091/sdb1/0/a/c" "tx0efafba1af8c499f9392f-00665ca4ac" "container-server 3656236" 0.0030 "-" 3656236 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:39551/sda1/0/a/c" "tx0efafba1af8c499f9392f-00665ca4ac" "container-server 3656236" 0.0024 "-" 3656236 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx0efafba1af8c499f9392f-00665ca4ac" "proxy-server 3656236" 0.0412 "-" 3656236 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "tx0efafba1af8c499f9392f-00665ca4ac" "proxy-server 3656236" 0.0631 "-" 3656236 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/20 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx0efafba1af8c499f9392f-00665ca4ac - 0.0778 - - 1717347500.823168993 1717347500.900991678 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "HEAD /sdb1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx6977034df66b493f9c376-00665ca4ac" "proxy-server 3656236" 0.0021 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:39551/sda1/2/a1/c1" "tx6977034df66b493f9c376-00665ca4ac" "container-server 3656236" 0.0020 "-" 3656236 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:57091/sdb1/2/a1/c1" "tx6977034df66b493f9c376-00665ca4ac" "container-server 3656236" 0.0019 "-" 3656236 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx6977034df66b493f9c376-00665ca4ac" "proxy-server 3656236" 0.0435 "-" 3656236 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx6977034df66b493f9c376-00665ca4ac" "proxy-server 3656236" 0.0260 "-" 3656236 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/20 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx6977034df66b493f9c376-00665ca4ac - 0.0574 - - 1717347500.904016018 1717347500.961452723 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:20 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe0294adb0e6742409c6d7-00665ca4ac" "proxy-server 3656236" 0.0178 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:57091/sdb1/0/a/c1" "txe0294adb0e6742409c6d7-00665ca4ac" "container-server 3656236" 0.0019 "-" 3656236 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:39551/sda1/0/a/c1" "txe0294adb0e6742409c6d7-00665ca4ac" "container-server 3656236" 0.0174 "-" 3656236 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe0294adb0e6742409c6d7-00665ca4ac" "proxy-server 3656236" 0.2129 "-" 3656236 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe0294adb0e6742409c6d7-00665ca4ac" "proxy-server 3656236" 0.0522 "-" 3656236 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/21 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txe0294adb0e6742409c6d7-00665ca4ac - 0.2426 - - 1717347500.963992596 1717347501.206619024 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa71a93deb9e14b3bb7758-00665ca4ad" "proxy-server 3656236" 0.0121 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:39551/sda1/2/a/c2" "txa71a93deb9e14b3bb7758-00665ca4ad" "container-server 3656236" 0.0018 "-" 3656236 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:57091/sdb1/2/a/c2" "txa71a93deb9e14b3bb7758-00665ca4ad" "container-server 3656236" 0.0087 "-" 3656236 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa71a93deb9e14b3bb7758-00665ca4ad" "proxy-server 3656236" 0.0302 "-" 3656236 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:21 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txa71a93deb9e14b3bb7758-00665ca4ad" "proxy-server 3656236" 0.0491 "-" 3656236 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/21 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txa71a93deb9e14b3bb7758-00665ca4ad - 0.0725 - - 1717347501.208852291 1717347501.281345129 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmpgsw8d0m7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpgsw8d0m7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmpgsw8d0m7/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmpgsw8d0m7/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a" 201 - "-" "test" "-" 0.0196 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a" 201 - "-" "test" "-" 0.0185 "-" 3656236 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a1" 201 - "-" "test" "-" 0.0170 "-" 3656236 - test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_sharding_states ... ok test.unit.proxy.test_server.TestContainerController.test_CORS_valid test.unit.proxy.test_server.TestContainerController.test_CORS_valid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_info_stale_read_ok ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE test.unit.proxy.test_server.TestContainerController.test_DELETE ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_DELETE_backed_x_timestamp_header ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_DELETE_x_account_headers_with_more_account_replicas ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_items_since ... ok acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a1" 201 - "-" "test" "-" 0.0212 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdaef861b5f4a4957a7133-00665ca4ae" "proxy-server 3656236" 0.0023 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://127.0.0.1:57303/sdb1/0/a/c" "txdaef861b5f4a4957a7133-00665ca4ae" "container-server 3656236" 0.0020 "-" 3656236 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://127.0.0.1:54497/sda1/0/a/c" "txdaef861b5f4a4957a7133-00665ca4ae" "container-server 3656236" 0.0019 "-" 3656236 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txdaef861b5f4a4957a7133-00665ca4ae" "proxy-server 3656236" 0.0273 "-" 3656236 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a/c" 201 - "PUT http://localhost/v1/a/c" "txdaef861b5f4a4957a7133-00665ca4ae" "proxy-server 3656236" 0.0493 "-" 3656236 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/22 PUT /v1/a/c HTTP/1.0 201 - - t - - - txdaef861b5f4a4957a7133-00665ca4ae - 0.0636 - - 1717347502.090159178 1717347502.153757095 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "HEAD /sda1/0/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx0f03b9274487416e8821e-00665ca4ae" "proxy-server 3656236" 0.0022 "-" 3656236 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a1/c1" 201 - "PUT http://127.0.0.1:54497/sda1/2/a1/c1" "tx0f03b9274487416e8821e-00665ca4ae" "container-server 3656236" 0.0039 "-" 3656236 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a1/c1" 201 - "PUT http://127.0.0.1:57303/sdb1/2/a1/c1" "tx0f03b9274487416e8821e-00665ca4ae" "container-server 3656236" 0.0017 "-" 3656236 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx0f03b9274487416e8821e-00665ca4ae" "proxy-server 3656236" 0.0481 "-" 3656236 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx0f03b9274487416e8821e-00665ca4ae" "proxy-server 3656236" 0.0295 "-" 3656236 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/22 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx0f03b9274487416e8821e-00665ca4ae - 0.0629 - - 1717347502.156165838 1717347502.219018459 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx29f02dd12cc145e0b7dbf-00665ca4ae" "proxy-server 3656236" 0.0160 "-" 3656236 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://127.0.0.1:54497/sda1/0/a/c1" "tx29f02dd12cc145e0b7dbf-00665ca4ae" "container-server 3656236" 0.0182 "-" 3656236 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://127.0.0.1:57303/sdb1/0/a/c1" "tx29f02dd12cc145e0b7dbf-00665ca4ae" "container-server 3656236" 0.0017 "-" 3656236 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx29f02dd12cc145e0b7dbf-00665ca4ae" "proxy-server 3656236" 0.0604 "-" 3656236 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx29f02dd12cc145e0b7dbf-00665ca4ae" "proxy-server 3656236" 0.0427 "-" 3656236 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/22 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx29f02dd12cc145e0b7dbf-00665ca4ae - 0.0886 - - 1717347502.221475124 1717347502.310055017 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "HEAD /sdb1/0/a" 204 - "HEAD http://localhost/v1/a?format=json" "txdb3d0eec885e46149f5fe-00665ca4ae" "proxy-server 3656236" 0.0155 "-" 3656236 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/0/a/c2" 201 - "PUT http://127.0.0.1:57303/sdb1/2/a/c2" "txdb3d0eec885e46149f5fe-00665ca4ae" "container-server 3656236" 0.0019 "-" 3656236 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/0/a/c2" 201 - "PUT http://127.0.0.1:54497/sda1/2/a/c2" "txdb3d0eec885e46149f5fe-00665ca4ae" "container-server 3656236" 0.0132 "-" 3656236 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sdb1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txdb3d0eec885e46149f5fe-00665ca4ae" "proxy-server 3656236" 0.0373 "-" 3656236 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:58:22 +0000] "PUT /sda1/2/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txdb3d0eec885e46149f5fe-00665ca4ae" "proxy-server 3656236" 0.0593 "-" 3656236 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/58/22 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txdb3d0eec885e46149f5fe-00665ca4ae - 0.0863 - - 1717347502.312453747 1717347502.398774624 2 test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: tx4dc07664381049c9a086e-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx6e6b526d3dfa4954a6e5f-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx42d48c6b640d4d888226b-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:2000/sda (txn: tx42d48c6b640d4d888226b-00665ca4ae) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: tx42d48c6b640d4d888226b-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txa0ef1c9622c447f1ae39c-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txa0ef1c9622c447f1ae39c-00665ca4ae) test ERROR: Container GET returning 503 for (204, 503, 503) (txn: txa0ef1c9622c447f1ae39c-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx1ccd0a92e16b487396f62-00665ca4ae) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx1ccd0a92e16b487396f62-00665ca4ae) test ERROR: ERROR 503 Trying to DELETE /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx38db2ad2e5d04807808d4-00665ca4ae) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx38db2ad2e5d04807808d4-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent test.unit.proxy.test_server.TestContainerController.test_GET_auto_create_prefix_account_non_existent ... ok test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize test.unit.proxy.test_server.TestContainerController.test_GET_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_GET_no_content test.unit.proxy.test_server.TestContainerController.test_GET_no_content ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_GET test.unit.proxy.test_server.TestContainerController.test_HEAD_GET ... ok test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize test.unit.proxy.test_server.TestContainerController.test_HEAD_calls_authorize ... ok test.unit.proxy.test_server.TestContainerController.test_OPTIONS test.unit.proxy.test_server.TestContainerController.test_OPTIONS ... ok test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin test.unit.proxy.test_server.TestContainerController.test_OPTIONS_get_info_drops_origin ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_objects_since_row ... ok test.unit.proxy.test_server.TestContainerController.test_POST test.unit.proxy.test_server.TestContainerController.test_POST ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txeb7cf41ff27d47b2940b2-00665ca4ae) test INFO: Client disconnected on read of '/a/c' (txn: txeb7cf41ff27d47b2940b2-00665ca4ae) test INFO: Client disconnected on read of '/a/c' (txn: txebbf6ef9694d4ef1b79b8-00665ca4ae) test INFO: Client disconnected on read of '/a/c' (txn: txebbf6ef9694d4ef1b79b8-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx5c6a07b0a2cd4276a7992-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx3331cf43d255428fbc35e-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx6a54760fc85b42dab2a25-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx6a54760fc85b42dab2a25-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx6a54760fc85b42dab2a25-00665ca4ae) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: tx6a54760fc85b42dab2a25-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txae2cdacf675b45018cf4d-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txae2cdacf675b45018cf4d-00665ca4ae) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txae2cdacf675b45018cf4d-00665ca4ae) test ERROR: Container GET returning 503 for [500, 500, 500] (txn: txae2cdacf675b45018cf4d-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/a: Network unreachable (txn: tx2b6e9bb00d024995827c9-00665ca4ae) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/a: Network unreachable (txn: tx2b6e9bb00d024995827c9-00665ca4ae) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/a: Network unreachable (txn: tx2b6e9bb00d024995827c9-00665ca4ae) test ERROR: Account HEAD returning 503 for [] (txn: tx2b6e9bb00d024995827c9-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.0:2000/sda (txn: tx7cf18318a7b74773a516f-00665ca4ae) test ERROR: ERROR 500 Trying to POST /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx2dce5af888c141d48a249-00665ca4ae) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: tx2dce5af888c141d48a249-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx5ebd779af1114fd484633-00665ca4ae) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx70cc0575118a40bc9fb1c-00665ca4ae) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx0ea9a461d7344ea09bc71-00665ca4ae) test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata test.unit.proxy.test_server.TestContainerController.test_POST_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_POST_calls_clean_acl ... ok test.unit.proxy.test_server.TestContainerController.test_POST_metadata test.unit.proxy.test_server.TestContainerController.test_POST_metadata ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_root_container_fill_gap ... ok test.unit.proxy.test_server.TestContainerController.test_PUT test.unit.proxy.test_server.TestContainerController.test_PUT ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_utf8 ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta test.unit.proxy.test_server.TestContainerController.test_PUT_autocreate_account_with_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_own_shard_range ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header test.unit.proxy.test_server.TestContainerController.test_PUT_backed_x_timestamp_header ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_namespaces_state_filtering ... ok test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx3ae2f124ea204e2ca3b15-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txb6d3f167c1cf41e3b142e-00665ca4ae) test ERROR: ERROR 500 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf27ca4bd1c474e34bb750-00665ca4ae) test ERROR: Container GET returning 503 for (204, 404, 503) (txn: txf27ca4bd1c474e34bb750-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: Account HEAD returning 503 for [503, 503, 503] (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test WARNING: Could not autocreate account '/account' (txn: tx62b19aafc1aa413f9a949-00665ca4ae) test INFO: autocreate account '/account' (txn: txaba73b1b3e0e4e82970b3-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test ERROR: Account HEAD returning 503 for [503, 503] (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.2:1002/sdc (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.0:1000/sda (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test WARNING: Could not autocreate account '/account' (txn: tx642daa9dc8124d8fb1dcb-00665ca4ae) test INFO: autocreate account '/account' (txn: txa7e7db2181aa4d8fa65dc-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: txbdac58940e734662b9553-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: txbdac58940e734662b9553-00665ca4ae) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txbdac58940e734662b9553-00665ca4ae) test INFO: autocreate account '/account' (txn: tx8c792e98810a4c6b94594-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: tx013b3152f1804b629e025-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /account From Container Server 10.0.0.1:1001/sdb (txn: tx013b3152f1804b629e025-00665ca4ae) test INFO: autocreate account '/account' (txn: tx013b3152f1804b629e025-00665ca4ae) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.2:1002/sdc (txn: tx7708f09978404bed8f3c5-00665ca4ae) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:2000/sda (txn: tx7708f09978404bed8f3c5-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/☃' (txn: tx05cf2c7510cb47949698a-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: autocreate account '/account' (txn: tx5ed1990d50aa4cd8bd0e1-00665ca4ae) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_bad_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl test.unit.proxy.test_server.TestContainerController.test_PUT_calls_clean_acl ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions test.unit.proxy.test_server.TestContainerController.test_PUT_connect_exceptions ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_policy_info ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length test.unit.proxy.test_server.TestContainerController.test_PUT_max_container_name_length ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account test.unit.proxy.test_server.TestContainerController.test_PUT_max_containers_per_account ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_replication_info ... ok test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx490ae39f6ee14f25b9792-00665ca4af) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9a08e2e2010f4be1bc500-00665ca4af) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txd76f6ce97af244baa0eae-00665ca4af) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx67f1fccbbb354d5094788-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx0d303303e09e4b47acc7d-00665ca4af) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx17c0a73fc29c4278bf8cd-00665ca4af) test ERROR: ERROR with Container server 10.0.0.2:2002/sdc re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx0ac258938a8e4fd6a5a16-00665ca4af) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx0ac258938a8e4fd6a5a16-00665ca4af) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: tx0ac258938a8e4fd6a5a16-00665ca4af) test ERROR: ERROR with Container server 10.0.0.1:2001/sdb re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txca8a3de84f20467594629-00665ca4af) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txca8a3de84f20467594629-00665ca4af) test ERROR: Container GET returning 503 for (201, 503, 503) (txn: txca8a3de84f20467594629-00665ca4af) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: tx831d9832c1404599a72aa-00665ca4af) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx831d9832c1404599a72aa-00665ca4af) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx831d9832c1404599a72aa-00665ca4af) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx831d9832c1404599a72aa-00665ca4af) test ERROR: ERROR with Container server 10.0.0.0:2000/sda re: Trying to PUT /a/c/: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1003, in connect raise HTTPException() http.client.HTTPException (txn: txcd681d09afd74167acfa0-00665ca4af) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txcd681d09afd74167acfa0-00665ca4af) test ERROR: ERROR 503 Trying to PUT /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txcd681d09afd74167acfa0-00665ca4af) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txcd681d09afd74167acfa0-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestContainerController.test_PUT_metadata test.unit.proxy.test_server.TestContainerController.test_PUT_metadata ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers test.unit.proxy.test_server.TestContainerController.test_PUT_policy_headers ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_fewer_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas test.unit.proxy.test_server.TestContainerController.test_PUT_x_account_headers_with_more_account_replicas ... ok test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 test.unit.proxy.test_server.TestContainerController.test_acc_missing_returns_404 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_report_info ... ok test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_authorized_requests_when_account_not_found ... ok test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_convert_index_to_name ... ok test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index test.unit.proxy.test_server.TestContainerController.test_convert_policy_to_index ... ok test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name test.unit.proxy.test_server.TestContainerController.test_error_convert_index_to_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_range_rows_with_limit ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.0:1000/sda (txn: txdf15bf4b230a4efbb37b1-00665ca4af) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx1119db0bc2104815a0048-00665ca4af) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx1119db0bc2104815a0048-00665ca4af) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx1119db0bc2104815a0048-00665ca4af) test ERROR: Account HEAD returning 503 for [] (txn: tx1119db0bc2104815a0048-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txd1b513e4f1e445aa946ea-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txd1b513e4f1e445aa946ea-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txd1b513e4f1e445aa946ea-00665ca4af) test ERROR: Account HEAD returning 503 for [] (txn: txd1b513e4f1e445aa946ea-00665ca4af) test ERROR: ERROR 503 b'' Trying to HEAD /account From Account Server 10.0.0.1:1001/sdb (txn: txfb4ed56b025c47c9a8176-00665ca4af) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9726a4169aba46478c403-00665ca4af) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx9726a4169aba46478c403-00665ca4af) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx9726a4169aba46478c403-00665ca4af) test ERROR: Account HEAD returning 503 for [] (txn: tx9726a4169aba46478c403-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:1000/sda (txn: txe1963b05d8d24fc1b7ed7-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: txe1963b05d8d24fc1b7ed7-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: txe1963b05d8d24fc1b7ed7-00665ca4af) test ERROR: Account HEAD returning 503 for [] (txn: txe1963b05d8d24fc1b7ed7-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx415af41e9c9249b3b898b-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a' to policy (txn: tx66bcbd97ad37423b8b716-00665ca4af) test ERROR: Could not translate X-Backend-Storage-Policy-Index ('-1') from '/a/c' to policy (txn: tx66bcbd97ad37423b8b716-00665ca4af) test INFO: Client disconnected on read of '/a/c' (txn: tx66bcbd97ad37423b8b716-00665ca4af) test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_objects_since_row ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txda21a0e3c32d493781963-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txb42d8cbd19e24bc1a0e99-00665ca4af) test INFO: Client disconnected on read of '/a/c' (txn: txb42d8cbd19e24bc1a0e99-00665ca4af) test INFO: Client disconnected on read of '/a/c/' (txn: txb42d8cbd19e24bc1a0e99-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txa4f5393826054b7cbaad8-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txa4f5393826054b7cbaad8-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa4f5393826054b7cbaad8-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txa4f5393826054b7cbaad8-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txca09da17c4ba41df9ae44-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txca09da17c4ba41df9ae44-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txca09da17c4ba41df9ae44-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txca09da17c4ba41df9ae44-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txa7badc513e2b48189e3ed-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txa7badc513e2b48189e3ed-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txa7badc513e2b48189e3ed-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txa7badc513e2b48189e3ed-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txeb35fbca1c0d43b8907ac-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txeb35fbca1c0d43b8907ac-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txeb35fbca1c0d43b8907ac-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txeb35fbca1c0d43b8907ac-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txf36d927c93fe4c83adac9-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf36d927c93fe4c83adac9-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf36d927c93fe4c83adac9-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txf36d927c93fe4c83adac9-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: txde64f028a2434d098beb0-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: txde64f028a2434d098beb0-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: txde64f028a2434d098beb0-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txde64f028a2434d098beb0-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: txf8b11da9c9eb486e93c9d-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: txf8b11da9c9eb486e93c9d-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: txf8b11da9c9eb486e93c9d-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: txf8b11da9c9eb486e93c9d-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.0:2000/sda (txn: tx0bc4ae3cb3b9403c8846d-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx0bc4ae3cb3b9403c8846d-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx0bc4ae3cb3b9403c8846d-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx0bc4ae3cb3b9403c8846d-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:2000/sda (txn: tx3bb7845b5ca3430ca9688-00665ca4af) test ERROR: Node will be error limited for 60.00s: 10.0.0.0:2000/sda (txn: tx3bb7845b5ca3430ca9688-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx3bb7845b5ca3430ca9688-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx3bb7845b5ca3430ca9688-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx3bb7845b5ca3430ca9688-00665ca4af) test ERROR: Container GET returning 503 for [503, 503, 503] (txn: tx3bb7845b5ca3430ca9688-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx55e6d459816d4fadb742c-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.1:2001/sdb (txn: tx55e6d459816d4fadb742c-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c/ From Container Server 10.0.0.2:2002/sdc (txn: tx55e6d459816d4fadb742c-00665ca4af) test ERROR: Container GET returning 503 for [503, 503] (txn: tx55e6d459816d4fadb742c-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx732b17093b1d488bb29dc-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.1:2001/sdb (txn: tx732b17093b1d488bb29dc-00665ca4af) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:2001/sdb (txn: tx732b17093b1d488bb29dc-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx732b17093b1d488bb29dc-00665ca4af) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.2:2002/sdc (txn: tx732b17093b1d488bb29dc-00665ca4af) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:2002/sdc (txn: tx732b17093b1d488bb29dc-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx732b17093b1d488bb29dc-00665ca4af) test ERROR: Container GET returning 503 for [503, 503] (txn: tx732b17093b1d488bb29dc-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx50d7ed54041c4a3aae273-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx50d7ed54041c4a3aae273-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx50d7ed54041c4a3aae273-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx50d7ed54041c4a3aae273-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx0e173cc47c17406cb2471-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx0e173cc47c17406cb2471-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx0e173cc47c17406cb2471-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx0e173cc47c17406cb2471-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx2a4fd47a1ec947839204a-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx2a4fd47a1ec947839204a-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx2a4fd47a1ec947839204a-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx2a4fd47a1ec947839204a-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txadea19aac8074ebdaacc0-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txadea19aac8074ebdaacc0-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txadea19aac8074ebdaacc0-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: txadea19aac8074ebdaacc0-00665ca4af) test.unit.proxy.test_server.TestContainerController.test_error_limiting test.unit.proxy.test_server.TestContainerController.test_error_limiting ... ok test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found test.unit.proxy.test_server.TestContainerController.test_no_convert_index_to_name_when_container_not_found ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges ... ok test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx50c7f1e48b2c41129bd6a-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx50c7f1e48b2c41129bd6a-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx50c7f1e48b2c41129bd6a-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx50c7f1e48b2c41129bd6a-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx039a89c910ef406091941-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx039a89c910ef406091941-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx039a89c910ef406091941-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx039a89c910ef406091941-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx49e1ef413704476a9f982-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx49e1ef413704476a9f982-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx49e1ef413704476a9f982-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx49e1ef413704476a9f982-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx6d872497357349688f034-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx6d872497357349688f034-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx6d872497357349688f034-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx6d872497357349688f034-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx629cda4c1c534c8080e6b-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx629cda4c1c534c8080e6b-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx629cda4c1c534c8080e6b-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx629cda4c1c534c8080e6b-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx05c013c77a0248e1a2b11-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx05c013c77a0248e1a2b11-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx05c013c77a0248e1a2b11-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: tx05c013c77a0248e1a2b11-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txd51dff4669534204aba69-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txd51dff4669534204aba69-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txd51dff4669534204aba69-00665ca4af) test ERROR: Container GET returning 503 for [] (txn: txd51dff4669534204aba69-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txdb8b17a0cc3a449bb4434-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txdb8b17a0cc3a449bb4434-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txdb8b17a0cc3a449bb4434-00665ca4af) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txdb8b17a0cc3a449bb4434-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx505e3eff6bc54e0eb8337-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx505e3eff6bc54e0eb8337-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx505e3eff6bc54e0eb8337-00665ca4af) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx505e3eff6bc54e0eb8337-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: tx8d1497deb04641f897d41-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: tx8d1497deb04641f897d41-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: tx8d1497deb04641f897d41-00665ca4af) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: tx8d1497deb04641f897d41-00665ca4af) test DEBUG: Node is error limited: 10.0.0.0:2000/sda (txn: txb3dff7b08b164c98b4fd9-00665ca4af) test DEBUG: Node is error limited: 10.0.0.1:2001/sdb (txn: txb3dff7b08b164c98b4fd9-00665ca4af) test DEBUG: Node is error limited: 10.0.0.2:2002/sdc (txn: txb3dff7b08b164c98b4fd9-00665ca4af) test ERROR: Container GET returning 503 for (503, 503, 503) (txn: txb3dff7b08b164c98b4fd9-00665ca4af) test INFO: Client disconnected on read of '/a/c' (txn: tx629361f495074b2cb48f8-00665ca4af) test INFO: Client disconnected on read of '/a/c/' (txn: tx629361f495074b2cb48f8-00665ca4af) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txaf1f45e084874515bd904-00665ca4af) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txaf1f45e084874515bd904-00665ca4af) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: txaf1f45e084874515bd904-00665ca4af) test ERROR: Account HEAD returning 503 for [] (txn: txaf1f45e084874515bd904-00665ca4af) test ERROR: ERROR with Account server 10.0.0.0:1000/sda re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx660b1026de0d44809041e-00665ca4af) test ERROR: ERROR with Account server 10.0.0.1:1001/sdb re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx660b1026de0d44809041e-00665ca4af) test ERROR: ERROR with Account server 10.0.0.2:1002/sdc re: Trying to HEAD /v1/account: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1556, in _make_node_request possible_source = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 845, in getresponse raise Exception('test') Exception: test (txn: tx660b1026de0d44809041e-00665ca4af) test ERROR: Account HEAD returning 503 for [] (txn: tx660b1026de0d44809041e-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container test.unit.proxy.test_server.TestContainerController.test_node_read_timeout_no_retry_to_container ... ok test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_get_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header test.unit.proxy.test_server.TestContainerController.test_response_head_accept_ranges_header ... ok test.unit.proxy.test_server.TestContainerController.test_transfer_headers test.unit.proxy.test_server.TestContainerController.test_transfer_headers ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_own_shard_range ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_includes ... ok test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found test.unit.proxy.test_server.TestContainerController.test_unauthorized_requests_when_account_not_found ... ok test.unit.proxy.test_server.TestECObjectController.test_GET_ec test.unit.proxy.test_server.TestECObjectController.test_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 test.unit.proxy.test_server.TestECObjectController.test_GET_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big test.unit.proxy.test_server.TestECObjectController.test_GET_ec_big ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted test.unit.proxy.test_server.TestECObjectController.test_GET_ec_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling test.unit.proxy.test_server.TestECObjectController.test_GET_ec_failure_handling ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline test.unit.proxy.test_server.TestECObjectController.test_GET_pipeline ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 test.unit.proxy.test_server.TestECObjectController.test_HEAD_ec_404 ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec test.unit.proxy.test_server.TestECObjectController.test_PUT_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_archive_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_fragment_quorum_bad_request ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_if_none_match ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_multiple_segments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch test.unit.proxy.test_server.TestECObjectController.test_PUT_ec_object_etag_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec test.unit.proxy.test_server.TestECObjectController.test_conditional_GET_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect test.unit.proxy.test_server.TestECObjectController.test_ec_client_put_disconnect ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec test.unit.proxy.test_server.TestECObjectController.test_reload_ring_ec ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_policy_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_replication_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_report_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_multiple_policies ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx160beeb07068489f8dd98-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx0280078f403549d5a2662-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: tx7a1ede3549b84615bd2a0-00665ca4af) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_range_rows_with_limit ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_includes ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_sharding_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_ranges_with_shrinking_overlaps ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_get_shard_usage ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_multiple_policies ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_has_other_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_initialize_container_broker_in_default ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_old_enough_to_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_is_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_last_modified_time ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_legacy_pending_files ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_non_slash ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_order_and_reverse ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_list_objects_iter_with_reserved_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_load_old_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_make_tuple_for_pickle ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_is_green ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_overwrite_unicode ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_items_post_overwrite_out_of_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_save_and_load_record_from_pending_file ... ok Signal account-reaper pid: 3 signal: 15 Signal account-reaper pid: 4 signal: 15 Signal account-reaper pid: 3 signal: 15 test ERROR: Problem finding shard upper in '/tmp/tmphjqr6x6d/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmphjqr6x6d/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpvmtw17bl/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpvmtw17bl/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_range_single_record ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_initiated_and_required ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_state_is_not_reclaimable ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRangeTombstonesColumn.test_sync_merging ... ok test.unit.container.test_updater.TestContainerUpdater.test_conf_params test.unit.container.test_updater.TestContainerUpdater.test_conf_params ... ok test.unit.container.test_updater.TestContainerUpdater.test_creation test.unit.container.test_updater.TestContainerUpdater.test_creation ... ok test.unit.container.test_updater.TestContainerUpdater.test_error_in_process test.unit.container.test_updater.TestContainerUpdater.test_error_in_process ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_with_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception test.unit.container.test_updater.TestContainerUpdater.test_listdir_without_exception ... ok test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container test.unit.container.test_updater.TestContainerUpdater.test_old_style_shard_container ... ok test ERROR: Failed to unlink '/tmp/tmp1jznpgft/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmp1jznpgft/containers/part/suffix/hash/container_1717347516.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmp1jznpgft/containers/part/suffix/hash/container_1717347516.00000.db': ['/tmp/tmp1jznpgft/containers/part/suffix/hash/container.db', '/tmp/tmp1jznpgft/containers/part/suffix/hash/container_1717347517.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmp1jznpgft/containers/part/suffix/hash/container_1717347517.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test INFO: Begin container update single threaded sweep test ERROR: Error processing container /tmp/tmp0vltry_6/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db: Boom!: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/updater.py", line 225, in container_sweep self.process_container(dbfile) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Boom! test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: ERROR: Failed to get paths to drive partitions: permission_denied test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpfbo9a550/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpfbo9a550/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpfbo9a550/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test.unit.container.test_updater.TestContainerUpdater.test_run_once test.unit.container.test_updater.TestContainerUpdater.test_run_once ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_device_unmounted ... ok test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout test.unit.container.test_updater.TestContainerUpdater.test_run_once_with_get_info_timeout ... ok test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmpxpjhavug/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /a/c /tmp/tmpxpjhavug/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/c /tmp/tmpxpjhavug/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test WARNING: /tmp/tmpey0_56mn/tmp_test_container_updater/devices/sda1 is not mounted test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Failed to get container info (Lock timeout: 10 seconds: /tmp/tmpg_8lsop1/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db); skipping. test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test INFO: Container update single threaded sweep completed: 0.00s, 0 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpiac6c20f/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.01s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test ERROR: ERROR account update failed with 127.0.0.1:67890/sda1 (will retry later):: Connection refused test DEBUG: Update report failed for /.shards_a/c /tmp/tmpiac6c20f/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 0 successes, 1 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /.shards_a/c /tmp/tmpiac6c20f/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.02s, 1 successes, 0 failures, 0 with no changes test.unit.container.test_updater.TestContainerUpdater.test_shard_container test.unit.container.test_updater.TestContainerUpdater.test_shard_container ... ok test.unit.container.test_updater.TestContainerUpdater.test_unicode test.unit.container.test_updater.TestContainerUpdater.test_unicode ... ok test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_can_read_old_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_extract_policy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_async_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_data_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_part_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_get_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_pickle_async_update_tmp_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_quarantine_renamer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_different_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_makedirs_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_object_dir_exists_but_not_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_os_link_race ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_same_file_exists ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_relink_paths_target_path_does_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata test.unit.obj.test_diskfile.TestDiskFileModuleMethods.test_write_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_build_reconstruction_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_collect_parts_mkdirs_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_partition ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_max_objects_per_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_delete_reverted_nondurable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_part_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_partners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_get_response ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_handoffs_only ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_ignores_status_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_iter_nodes_for_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_max_objects_per_revert_only_for_revert_jobs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_no_delete_failed_revert ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_client_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_insufficient_storage ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_success ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_process_job_all_timeout ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstruct_check_ring ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skipped_partpower_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_reconstructor_skips_bogus_partition_dirs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_removes_zbf ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_run_once_override_devices ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_non_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_past_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window test.unit.obj.test_reconstructor.TestGlobalSetupObjectReconstructorLegacyDurable.test_sync_old_nondurable_before_committed_zero_commit_window ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests test.unit.proxy.controllers.test_container.TestContainerController.test_GET_bad_requests ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_container_cache_cleared_after_PUT ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached test.unit.proxy.controllers.test_container.TestContainerController.test_container_info_got_cached ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database test.unit.proxy.controllers.test_container.TestContainerController.test_handoff_has_deleted_database ... ok test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test INFO: Begin container update single threaded sweep test DEBUG: Update report sent for /a/Ω /tmp/tmpp5aiaf_6/tmp_test_container_updater/devices/sda1/containers/subdir/hash.db test INFO: Container update single threaded sweep completed: 0.03s, 1 successes, 0 failures, 0 with no changes test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx803b04871c084534b0010-00665ca4bf) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx803b04871c084534b0010-00665ca4bf) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx803b04871c084534b0010-00665ca4bf) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: tx803b04871c084534b0010-00665ca4bf) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx803b04871c084534b0010-00665ca4bf) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx803b04871c084534b0010-00665ca4bf) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx803b04871c084534b0010-00665ca4bf) test INFO: Client disconnected on read of '/a/c' (txn: tx803b04871c084534b0010-00665ca4bf) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txa122d78ef8da439ca4a95-00665ca4bf) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txa122d78ef8da439ca4a95-00665ca4bf) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txa122d78ef8da439ca4a95-00665ca4bf) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda (txn: tx0cc460c6f307493497c51-00665ca4c0) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb (txn: txd51a8e602bcf49c99771e-00665ca4c0) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx4b5a8e7cb53e4e7295ae2-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Timeout (Nones) (txn: txe059cf41bb944e1aabd3a-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: tx576520f185934a4dacd9d-00665ca4c0) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx1efe0785c5db46808d4b6-00665ca4c0) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx1efe0785c5db46808d4b6-00665ca4c0) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda (txn: txb6bbda5795c84a71afe1a-00665ca4c0) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx5a5413547f4f49fa9e07a-00665ca4c0) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc (txn: txb44da419a8b447bba3ba7-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to DELETE /a/c: Timeout (Nones) (txn: tx253d46e715aa4d31a8221-00665ca4c0) test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors test.unit.proxy.controllers.test_container.TestContainerController.test_node_errors ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin test.unit.proxy.controllers.test_container.TestContainerController.test_reseller_admin ... ok test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txe623fae2a1ad472498a8e-00665ca4c0) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx33cc3826df1342fa91522-00665ca4c0) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx33cc3826df1342fa91522-00665ca4c0) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda (txn: txaefc84ae14e04586a7398-00665ca4c0) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb (txn: txcc4c2b8771044072b49e2-00665ca4c0) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc (txn: txbbc02b80eba7458dbb5f9-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to POST /a/c: Timeout (Nones) (txn: txeb7b90c4055f4002a32ed-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status Exception: kaboom! (txn: txb38bbf36e1c14bd6a2296-00665ca4c0) test ERROR: Node will be error limited for 60.00s: 10.0.0.2:1002/sdc, error: ERROR Insufficient Storage (txn: tx13a81f86a24e419e890e1-00665ca4c0) test DEBUG: Node is error limited: 10.0.0.2:1002/sdc (txn: tx13a81f86a24e419e890e1-00665ca4c0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txa1423193177f4e8fba7c2-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: tx9c92a5f643804a17bab15-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: txa90cdf5fc24141f88f9e1-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: txa90cdf5fc24141f88f9e1-00665ca4c0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_DELETE ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to DELETE /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR 503 Trying to DELETE /a/c From Container Server 10.0.0.0:1000/sda test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_POST ... ok test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (204, 404, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (204, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR 503 Trying to POST /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: Container GET returning 503 for (201, 404, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (201, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test ERROR: Container GET returning 503 for (404, 503, 503) test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.1:1001/sdb test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_response_code_for_PUT ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.0:1000/sda test ERROR: ERROR 503 Trying to PUT /a/c From Container Server 10.0.0.2:1002/sdc test ERROR: Container GET returning 503 for (503, 503, 503) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 1548, in _make_node_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 706, in __init__ raise status OSError (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: Container GET returning 503 for [] (txn: txe8ce4a40146c4266baf4d-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx6468419742284eba9eb09-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx6468419742284eba9eb09-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.1:1001/sdb (txn: tx6468419742284eba9eb09-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.3:1003/sda (txn: tx6468419742284eba9eb09-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.4:1004/sda (txn: tx6468419742284eba9eb09-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.5:1005/sda (txn: tx6468419742284eba9eb09-00665ca4c0) test ERROR: Container GET returning 503 for [500, 500, 500, 500, 500, 500] (txn: tx6468419742284eba9eb09-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: tx607eba3744704f0baa515-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: txee8eed83351e480295ff5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf9a7709d11934601af26d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf9a7709d11934601af26d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txf9a7709d11934601af26d-00665ca4c0) test ERROR: Container GET returning 503 for [] (txn: txf9a7709d11934601af26d-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: txf9a7709d11934601af26d-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: Container GET returning 503 for [] (txn: tx014800e99e7945e98f6a5-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txbfe4ca06f1f84f54bc16a-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txbfe4ca06f1f84f54bc16a-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txbfe4ca06f1f84f54bc16a-00665ca4c0) test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET test.unit.proxy.controllers.test_container.TestContainerController.test_response_codes_for_GET ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner test.unit.proxy.controllers.test_container.TestContainerController.test_swift_owner ... ok test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_POST ... ok test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txbfe4ca06f1f84f54bc16a-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txbfe4ca06f1f84f54bc16a-00665ca4c0) test ERROR: Container GET returning 503 for [] (txn: txbfe4ca06f1f84f54bc16a-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txa0bd0986c94f43499d13e-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to GET /v1/a/c: Timeout (Nones) (txn: txa0bd0986c94f43499d13e-00665ca4c0) test ERROR: Container GET returning 503 for [] (txn: txa0bd0986c94f43499d13e-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to GET /v1/a/c: Timeout (Nones) (txn: tx76aa363cc307411b99736-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.2:1002/sdc (txn: tx439e6173468b4062b4a39-00665ca4c0) test ERROR: ERROR 500 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx439e6173468b4062b4a39-00665ca4c0) test ERROR: Container GET returning 503 for [500, 500] (txn: tx439e6173468b4062b4a39-00665ca4c0) test ERROR: ERROR 503 b'' Trying to GET /a/c From Container Server 10.0.0.0:1000/sda (txn: tx0293857dc9394762917a3-00665ca4c0) test ERROR: Node will be error limited for 60.00s: 10.0.0.1:1001/sdb, error: ERROR Insufficient Storage (txn: tx9c69e17aaf864af8a59b4-00665ca4c0) test DEBUG: Node is error limited: 10.0.0.1:1001/sdb (txn: tx9c69e17aaf864af8a59b4-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: tx9c69e17aaf864af8a59b4-00665ca4c0) test ERROR: Container GET returning 503 for [] (txn: tx8f99a00a64704aa0a0cc6-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: tx8f99a00a64704aa0a0cc6-00665ca4c0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Client disconnected on read of '/a/c' (txn: txfd4d694fd3404ed9b3175-00665ca4c0) test INFO: Client disconnected on read of '/a/c' (txn: txfd4d694fd3404ed9b3175-00665ca4c0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca3980c0c8174fbd94800-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.2:1002/sdc re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca3980c0c8174fbd94800-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca3980c0c8174fbd94800-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca3980c0c8174fbd94800-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to POST /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: txca3980c0c8174fbd94800-00665ca4c0) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: txca3980c0c8174fbd94800-00665ca4c0) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: ERROR with Container server 10.0.0.0:1000/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx81520c9bfcae4458b3382-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.1:1001/sdb re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx81520c9bfcae4458b3382-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.3:1003/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx81520c9bfcae4458b3382-00665ca4c0) test ERROR: ERROR with Container server 10.0.0.4:1004/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx81520c9bfcae4458b3382-00665ca4c0) test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT test.unit.proxy.controllers.test_container.TestContainerController.test_sys_meta_headers_PUT ... ok test ERROR: ERROR with Container server 10.0.0.5:1005/sda re: Trying to PUT /a/c: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/proxy/controllers/base.py", line 2078, in _make_request conn = http_connect( ^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 977, in connect i, status = next(conn_id_and_code_iter) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration (txn: tx81520c9bfcae4458b3382-00665ca4c0) test ERROR: Container GET returning 503 for (200, 503, 503) (txn: tx81520c9bfcae4458b3382-00665ca4c0) test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_object_table_migration ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_merge_shard_ranges_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_newid ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_old_style_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_root_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_object_stats_shard_container ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_old_style_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_overlap_shard_range_order ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_pending_file_name ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_policy_stat_tracking ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_misplaced_object_does_not_effect_container_stats ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_encoded_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_put_object_multiple_explicit_timestamps_using_memory ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_save_and_load_record_from_pending_file ... ok test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test DEBUG: Calling S3Api Middleware test DEBUG: checking permission: bucket object HEAD {'Host': 'localhost:80', 'Authorization': 'AWS test:other:hmac', 'Date': 'Sun, 02 Jun 2024 16:58:02 -0000', 'User-Agent': 'Mozzarella Foxfire'} test DEBUG: test:testertest:testertest:full_controltest:full_controlFULL_CONTROLtest:readtest:readREADtest:writetest:writeWRITEtest:read_acptest:read_acpREAD_ACPtest:write_acptest:write_acpWRITE_ACP test DEBUG: permission denined: 403 Forbidden test:other READ test ERROR: Problem finding shard upper in '/tmp/tmpjsl3i23w/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmpjsl3i23w/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmpb2xybqbv/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmpb2xybqbv/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_deadlock ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_shard_ranges ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reclaim_with_duplicate_names ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reconciler_sync ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharded_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_deleted ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_remove_objects ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_resolve_shard_range_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_storage_policy_index_empty ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_set_x_syncs ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_reverse_prefix_delim ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_root_account_container_path ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_save_and_load_record_from_pending_file ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_initiated_and_required ... ok test INFO: - - 01/Jan/1970/02/46/41 DELETE /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/41 OPTIONS /v1/a/ HTTP/1.0 299 - - - 11 3 - - - 1.1700 - - 10000.000000000 10001.170000000 - test INFO: - - 01/Jan/1970/02/46/42 GET - HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET / HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /bad HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /baddy/mc_badderson HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1 HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/02/46/42 GET /v1/ HTTP/1.0 123 - - - 7 13 - - - 2.7183 - - 10000.000000000 10002.718281828 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 - test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 - test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/ HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/ HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 0 test INFO: - - 01/Jan/1970/00/00/20 GET /v1/a/c/o/p/p2 HTTP/1.0 321 - - - 4 7 - - - 2.7183 - - 18.000000000 20.718281828 -1 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 0 test INFO: - - 01/Jan/1970/00/01/05 PUT /v1/a/c/o/p/p2 HTTP/1.0 314 - - - 6 8 - - - 7.3321 - - 58.200000000 65.532100000 -1 test INFO: - - 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.152329683 1717347486.152388811 - test INFO: - - 02/Jun/2024/16/58/06 GET /foo HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.152837038 1717347486.152890921 - test INFO: - - 02/Jun/2024/16/58/06 GET /foo/bar HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.153313637 1717347486.153366566 - test INFO: - - 02/Jun/2024/16/58/06 GET /v1 HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.153789520 1717347486.153842211 - test INFO: - - 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - 17 - - - 0.0001 SOS - 1717347486.157922268 1717347486.158029318 - test INFO: - - 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - - - - - 0.0001 - - 1717347486.159392834 1717347486.159443617 - test INFO: - - 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - 3 - - - 0.0001 - - 1717347486.160692692 1717347486.160752535 - test INFO: - - 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - 12 - - - 0.0000 - - 1717347486.161952019 1717347486.161997795 - test INFO: - - 02/Jun/2024/16/58/06 PUT /v1/a/c/o HTTP/1.0 200 - - - - 8 - - - 0.0000 - - 1717347486.171242237 1717347486.171287298 1 test INFO: - - 02/Jun/2024/16/58/06 GET /v1/a/c HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.172438860 1717347486.172526836 1 test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.173837900 1717347486.173898697 - test INFO: 4.5.6.7 1.2.3.4 02/Jun/2024/16/58/06 GET / HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.174916506 1717347486.174974918 - test INFO: - - 02/Jun/2024/16/58/06 GET /v1/something HTTP/1.0 200 - - - - 8 - - - 0.0001 - - 1717347486.176272392 1717347486.176364660 - test INFO: - - 02/Jun/2024/16/58/06 GET / HTTP/1.0 499 - - - - 4 - - - 0.0001 - - 1717347486.179816484 1717347486.179883242 - test INFO: - - 02/Jun/2024/16/58/06 POST /v1/a/c HTTP/1.0 200 - - - 11 8 - - Host:%20localhost:80 0.0000 - - 1717347486.181108475 1717347486.181138277 - test INFO: - - 02/Jun/2024/16/58/06 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717347486.182472467 1717347486.182506800 0 test INFO: - - 02/Jun/2024/16/58/06 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717347486.183681250 1717347486.183715582 -1 test INFO: - - 02/Jun/2024/16/58/06 PUT /v1/a/c/o/foo HTTP/1.0 200 - - - 10 8 - - Host:%20localhost:80 0.0000 - - 1717347486.185061932 1717347486.185096025 - test INFO: Begin container audit "once" mode test DEBUG: Audit passed for /tmp/tmp6d2n0em3/sda/containers/0/0/0/test.db test INFO: Container audit "once" mode completed: 0.00s test ERROR: Problem finding shard upper in '/tmp/tmp5igjazcm/test_container.db': : LockTimeout (Nones) test ERROR: Problem finding shard upper in '/tmp/tmp5igjazcm/test_container.db': : Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2507, in find_shard_ranges next_shard_upper = sub_broker._get_next_shard_range_upper( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect sqlite3.OperationalError test DEBUG: Unlinked retiring db '/tmp/tmp2a9__uuk/containers/part/suffix/hash/container.db' test WARNING: Unexpected db files will be ignored: ['/tmp/tmp2a9__uuk/containers/part/suffix/hash/container.db'] Container 'a/c' cannot be set to sharded state while in unsharded state Container 'a/c' cannot be set to sharding state: missing epoch test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_state_is_not_reclaimable ... ok Container 'a/c' cannot be set to sharding state while in sharding state Container 'a/c' cannot be set to sharded state while in sharded state test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sharding_sysmeta ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_db_states ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeSPI.test_sync_merging ... ok test.unit.obj.test_server.TestTpoolSize.test_default_config test.unit.obj.test_server.TestTpoolSize.test_default_config ... ok test.unit.obj.test_server.TestTpoolSize.test_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_empty ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_no_explicit_setting ... ok test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting test.unit.obj.test_server.TestTpoolSize.test_servers_per_port_with_explicit_setting ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharded_state_errors ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespace_caching ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_404_response ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_bad_response_body ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_cache_unused_recheck_listing_shard_ranges ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_incomplete_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_non_sharded_states ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_for_object_listing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_cache_write_with_cached_container_info ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_deleted ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_no_memcache_available ... ok test ERROR: Failed to unlink '/tmp/tmpopxj0yic/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpopxj0yic/containers/part/suffix/hash/container_1717347541.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpopxj0yic/containers/part/suffix/hash/container_1717347541.00000.db': ['/tmp/tmpopxj0yic/containers/part/suffix/hash/container.db', '/tmp/tmpopxj0yic/containers/part/suffix/hash/container_1717347542.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpopxj0yic/containers/part/suffix/hash/container_1717347542.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3f98233d81f04776b93cf-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx1418f88076fb45fbb029d-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txaabb5ab3cb6e4ee9a53aa-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txa27fad1872d1469cb14a8-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1ac2b1bc2d3447f588f0e-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx4d1b4be035ab43319e9e6-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx9073dcb4264f49a5b11ac-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx3eb6950be22d491e90a78-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txb5a552fc36f6468cbfcef-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx0fd729dd71514a88b9f73-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx001f7e5587b0419fa1bf3-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txd69b5baf668b4094b9b59-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx05c2fa61b6a04302a63a4-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txb0861745dfc5455db0ce9-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx4860a6470e5a4d478434c-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx6f14ce1aae1e4fea84b3a-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx1b399e96fd5a4bc2aebba-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txc62a16847a08492ea5917-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx94f987d215b24447901d3-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx500f1c8758c34cf0a921d-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx9469a9df21d8441d8ddfc-00665ca4d8) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf30923ce4a8249fb83b0e-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx03c7dc9fe40d4ce5b48cd-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txe519221f4fdc4388a00bf-00665ca4d8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: txbca645eaca7846afb098d-00665ca4d8) test ERROR: Failed to get namespaces from /v1/a/c?states=listing&format=json: invalid data: KeyError('name') (txn: tx1fe59153054c4b5f8581e-00665ca4d8) test ERROR: Problem with listing response from /v1/a/c?states=listing&format=json: ValueError('not a list') (txn: tx3d0a5d38d7b34223966b4-00665ca4d8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_empty_list ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_read_from_cache_error ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_GET_namespaces_write_to_cache_with_x_newest ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_cache_clearing ... ok test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi test.unit.proxy.controllers.test_container.TestGetPathNamespaceCachingLegacy.test_get_from_shards_add_root_spi ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked test.unit.proxy.controllers.test_obj.TestECFragGetter.test_fragment_size_resuming_chunked ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_init_node_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part ... ok test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: tx8ad47b78f33e47c4ae128-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&format=json (txn: txe8bf8385217b48eab4e6f-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&reverse=true&format=json (txn: tx93c42d025c8d46ddbeefe-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&format=json (txn: tx48c8e489257947ed9dcbe-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&marker=jam&end_marker=kale&format=json (txn: tx381038a66ecf4836a069d-00665ca4d8) test DEBUG: Found 3 shards in cache for /v1/a/c?states=listing&includes=egg&format=json (txn: tx06c612c1b0ed4e8eae0ad-00665ca4d8) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx5b60bcdcf429443dbaed8-00665ca4d9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx0e36472f8edd4b21a22bf-00665ca4d9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx7b85b70f0acd4ae499aef-00665ca4d9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txd0dd53e3b07e4587b8879-00665ca4d9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: txf1749d75369b4b5485311-00665ca4d9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx161b0eb9f5b1470cb3ab3-00665ca4d9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Skipping shard cache lookup (x-newest) for /v1/a/c?states=listing&format=json (txn: tx816e537d50bd406892010-00665ca4d9) test INFO: Caching listing namespaces for shard-listing-v2/a/c (3 namespaces) (txn: tx816e537d50bd406892010-00665ca4d9) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] test DEBUG: GET listing from 3 shards for: /v1/a/c test DEBUG: Getting listing part 0 from shard Namespace(name='.shards_a/c_ham', lower='', upper='ham') .shards_a/c_ham with {'X-Backend-Record-Type': 'auto'} test ERROR: Aborting listing from shards due to bad response: [204] proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server ERROR: Trying to read EC fragment during GET (retrying) 1.2.3.4:6200/sda (txn: txn1) (client_ip: 127.0.0.2) proxy-server WARNING: Client disconnected on read of EC frag 'a/c/o' (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_insufficient_bytes ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_read_timeout ... ok test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size test.unit.proxy.controllers.test_obj.TestECFragGetter.test_iter_bytes_from_response_part_small_fragment_size ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_sharding_state_errors ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_storage_policy_index_empty ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_set_x_syncs ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok swift: Client disconnected on read of '/a' (txn: tx6ae296c7df454a68bd55a-00665ca4da) swift: Client disconnected on read of '/a' (txn: tx6ae296c7df454a68bd55a-00665ca4da) test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_mem_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 1.0, 'concurrent_ec_extra_requests': 0}, app) (txn: txn1) (client_ip: 127.0.0.2) proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) (client_ip: 127.0.0.2) cont1 DEBUG: Could not load '/tmp/tmp7b2cihe3/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp7b2cihe3/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) cont2 DEBUG: Could not load '/tmp/tmp7b2cihe3/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp7b2cihe3/tmp_test_proxy_server_chunked/container-sync-realms.conf' (client_ip: 127.0.0.2) cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. (client_ip: 127.0.0.2) acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.4193 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0109 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0107 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0110 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb02c079deda64ea89ff01-00665ca4d9" "proxy-server 3656221" 0.0019 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:59579/sdb1/1/a/c" "txb02c079deda64ea89ff01-00665ca4d9" "container-server 3656221" 0.0019 "-" 3656221 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:45195/sda1/1/a/c" "txb02c079deda64ea89ff01-00665ca4d9" "container-server 3656221" 0.0020 "-" 3656221 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txb02c079deda64ea89ff01-00665ca4d9" "proxy-server 3656221" 0.0239 "-" 3656221 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "txb02c079deda64ea89ff01-00665ca4d9" "proxy-server 3656221" 0.1014 "-" 3656221 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/05 PUT /v1/a/c HTTP/1.0 201 - - t - - - txb02c079deda64ea89ff01-00665ca4d9 - 0.1137 - - 1717347545.703269958 1717347545.816955328 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "txbfbd0b87af9b409c8e401-00665ca4d9" "proxy-server 3656221" 0.0020 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:59579/sdb1/1/a1/c1" "txbfbd0b87af9b409c8e401-00665ca4d9" "container-server 3656221" 0.0031 "-" 3656221 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:45195/sda1/1/a1/c1" "txbfbd0b87af9b409c8e401-00665ca4d9" "container-server 3656221" 0.0023 "-" 3656221 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbfbd0b87af9b409c8e401-00665ca4d9" "proxy-server 3656221" 0.0231 "-" 3656221 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "txbfbd0b87af9b409c8e401-00665ca4d9" "proxy-server 3656221" 0.0383 "-" 3656221 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/05 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - txbfbd0b87af9b409c8e401-00665ca4d9 - 0.0510 - - 1717347545.819085598 1717347545.870117903 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx76f6ff62906949f588b67-00665ca4d9" "proxy-server 3656221" 0.0107 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:59579/sdb1/2/a/c1" "tx76f6ff62906949f588b67-00665ca4d9" "container-server 3656221" 0.0049 "-" 3656221 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:45195/sda1/2/a/c1" "tx76f6ff62906949f588b67-00665ca4d9" "container-server 3656221" 0.0082 "-" 3656221 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx76f6ff62906949f588b67-00665ca4d9" "proxy-server 3656221" 0.0349 "-" 3656221 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "tx76f6ff62906949f588b67-00665ca4d9" "proxy-server 3656221" 0.0567 "-" 3656221 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/05 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - tx76f6ff62906949f588b67-00665ca4d9 - 0.0855 - - 1717347545.873048306 1717347545.958596945 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:05 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txb5df3cbce86c431bbf237-00665ca4d9" "proxy-server 3656221" 0.0097 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:45195/sda1/1/a/c2" "txb5df3cbce86c431bbf237-00665ca4d9" "container-server 3656221" 0.0023 "-" 3656221 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:59579/sdb1/1/a/c2" "txb5df3cbce86c431bbf237-00665ca4d9" "container-server 3656221" 0.0081 "-" 3656221 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txb5df3cbce86c431bbf237-00665ca4d9" "proxy-server 3656221" 0.0316 "-" 3656221 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txb5df3cbce86c431bbf237-00665ca4d9" "proxy-server 3656221" 0.0492 "-" 3656221 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/06 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txb5df3cbce86c431bbf237-00665ca4d9 - 0.0717 - - 1717347545.961448431 1717347546.033120155 2 proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_initiated_and_required ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_unknown ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_accept_xml ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_bad_accept ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_invalid_utf8 ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_json ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_format_xml ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_state_is_not_reclaimable ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_GET_autocreate_utf8 ... ok swift: Client disconnected on read of '/a' (txn: txcba597f5ce4948c79117e-00665ca4da) swift: Client disconnected on read of '/a' (txn: txcba597f5ce4948c79117e-00665ca4da) test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acl_header_access ... ok test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation test.unit.proxy.test_server.TestAccountControllerFakeGetResponse.test_account_acls_through_delegation ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sharding_sysmeta ... ok cont1 DEBUG: Could not load '/tmp/tmp2t38ihfk/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp2t38ihfk/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmp2t38ihfk/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmp2t38ihfk/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0113 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0185 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0274 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0142 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx62f78540878b4be5a659e-00665ca4da" "proxy-server 3656221" 0.0024 "-" 3656221 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:49765/sdb1/1/a/c" "tx62f78540878b4be5a659e-00665ca4da" "container-server 3656221" 0.0023 "-" 3656221 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:46045/sda1/1/a/c" "tx62f78540878b4be5a659e-00665ca4da" "container-server 3656221" 0.0034 "-" 3656221 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx62f78540878b4be5a659e-00665ca4da" "proxy-server 3656221" 0.0247 "-" 3656221 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx62f78540878b4be5a659e-00665ca4da" "proxy-server 3656221" 0.0428 "-" 3656221 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/06 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx62f78540878b4be5a659e-00665ca4da - 0.0571 - - 1717347546.393076181 1717347546.450187683 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx635a97f308df42b4a68d5-00665ca4da" "proxy-server 3656221" 0.0025 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:46045/sda1/1/a1/c1" "tx635a97f308df42b4a68d5-00665ca4da" "container-server 3656221" 0.0022 "-" 3656221 0 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:49765/sdb1/1/a1/c1" "tx635a97f308df42b4a68d5-00665ca4da" "container-server 3656221" 0.0021 "-" 3656221 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx635a97f308df42b4a68d5-00665ca4da" "proxy-server 3656221" 0.0377 "-" 3656221 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx635a97f308df42b4a68d5-00665ca4da" "proxy-server 3656221" 0.0257 "-" 3656221 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/06 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx635a97f308df42b4a68d5-00665ca4da - 0.0534 - - 1717347546.452879667 1717347546.506318569 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txe7cd4abc99af444ba77a4-00665ca4da" "proxy-server 3656221" 0.0095 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:49765/sdb1/2/a/c1" "txe7cd4abc99af444ba77a4-00665ca4da" "container-server 3656221" 0.0034 "-" 3656221 1 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:46045/sda1/2/a/c1" "txe7cd4abc99af444ba77a4-00665ca4da" "container-server 3656221" 0.0085 "-" 3656221 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe7cd4abc99af444ba77a4-00665ca4da" "proxy-server 3656221" 0.0415 "-" 3656221 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txe7cd4abc99af444ba77a4-00665ca4da" "proxy-server 3656221" 0.0299 "-" 3656221 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/06 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txe7cd4abc99af444ba77a4-00665ca4da - 0.0646 - - 1717347546.508947134 1717347546.573577404 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx4d94202340e841d7812cf-00665ca4da" "proxy-server 3656221" 0.0088 "-" 3656221 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:49765/sdb1/1/a/c2" "tx4d94202340e841d7812cf-00665ca4da" "container-server 3656221" 0.0018 "-" 3656221 2 acct1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:46045/sda1/1/a/c2" "tx4d94202340e841d7812cf-00665ca4da" "container-server 3656221" 0.0240 "-" 3656221 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx4d94202340e841d7812cf-00665ca4da" "proxy-server 3656221" 0.0551 "-" 3656221 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:16:59:06 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "tx4d94202340e841d7812cf-00665ca4da" "proxy-server 3656221" 0.0434 "-" 3656221 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/16/59/06 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - tx4d94202340e841d7812cf-00665ca4da - 0.0751 - - 1717347546.575869322 1717347546.650986910 2 test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_storage_policy_property ... ok test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging test.unit.container.test_backend.TestContainerBrokerBeforeShardRanges.test_sync_merging ... ok test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy test.unit.container.test_reconciler.TestReconciler.test_can_reconcile_policy ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency test.unit.container.test_reconciler.TestReconciler.test_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_concurrency_config test.unit.container.test_reconciler.TestReconciler.test_concurrency_config ... ok test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers test.unit.container.test_reconciler.TestReconciler.test_delete_old_empty_queue_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers test.unit.container.test_reconciler.TestReconciler.test_error_in_iter_containers ... ok test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi test.unit.container.test_reconciler.TestReconciler.test_fail_to_move_if_ppi ... ok test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name test.unit.container.test_reconciler.TestReconciler.test_init_internal_client_log_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name ... ok test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward test.unit.container.test_reconciler.TestReconciler.test_invalid_queue_name_marches_onward ... ok test ERROR: Failed to unlink '/tmp/tmpzlc2f55i/containers/part/suffix/hash/container.db': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2244, in set_sharded_state os.unlink(retiring_file) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect OSError: 1 test DEBUG: Unlinked retiring db '/tmp/tmpzlc2f55i/containers/part/suffix/hash/container_1717347545.00000.db' test WARNING: Still have multiple db files after unlinking '/tmp/tmpzlc2f55i/containers/part/suffix/hash/container_1717347545.00000.db': ['/tmp/tmpzlc2f55i/containers/part/suffix/hash/container.db', '/tmp/tmpzlc2f55i/containers/part/suffix/hash/container_1717347546.00000.db'] test WARNING: Refusing to delete db file for 'a/c': no fresher db file found in ['/tmp/tmpzlc2f55i/containers/part/suffix/hash/container_1717347546.00000.db']. test ERROR: Failed to set the ROWID of the fresh database for a/c: test ERROR: Failed to sync the container_stat table/view with the fresh database for a/c: test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716681600 test DEBUG: looking for objects in 1716681600 test DEBUG: checking container 1716678000 test DEBUG: looking for objects in 1716678000 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'something', 'last_modified': '1970-01-01T00:00:00.000000', 'hash': '0.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test ERROR: Error listing containers in account .misplaced_objects (Unexpected response: 503 Service Unavailable) test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: ppi_skip: Container policy (0) in the middle of a part power increase (PPI) test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'ppi_skip': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': 'bogus', 'last_modified': '1970-01-01T01:00:18.841870', 'hash': '3618.841870', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test INFO: Reconciler Stats: {'invalid_record': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test ERROR: invalid_record: invalid queue record: {'bytes': 0, 'name': '00000bogus', 'last_modified': '1970-01-01T01:00:00.000000', 'hash': '3600.000000', 'content_type': 'application/x-put'} Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 835, in reconcile queue_item = parse_raw_obj(raw_obj) ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 280, in parse_raw_obj policy_index, obj_name = raw_obj_name.split(':', 1) ^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 2, got 1) test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'invalid_record': 1, 'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1717340400 test DEBUG: looking for objects in 1717340400 test DEBUG: checking container 1717336800 test DEBUG: looking for objects in 1717336800 test DEBUG: checking container 1717333200 test DEBUG: looking for objects in 1717333200 test DEBUG: checking container 1717329600 test DEBUG: looking for objects in 1717329600 test DEBUG: checking container 1717326000 test DEBUG: looking for objects in 1717326000 test DEBUG: checking container 1717322400 test DEBUG: looking for objects in 1717322400 test DEBUG: checking container 1717318800 test DEBUG: looking for objects in 1717318800 test DEBUG: checking container 1717315200 test DEBUG: looking for objects in 1717315200 test DEBUG: checking container 1717311600 test DEBUG: looking for objects in 1717311600 test DEBUG: checking container 1716739200 test DEBUG: looking for objects in 1716739200 test DEBUG: checking container 1716735600 test DEBUG: looking for objects in 1716735600 test DEBUG: checking container 1716732000 test DEBUG: looking for objects in 1716732000 test DEBUG: checking container 1716728400 test DEBUG: looking for objects in 1716728400 test DEBUG: checking container 1716724800 test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse test.unit.container.test_reconciler.TestReconciler.test_iter_over_old_containers_in_reverse ... ok test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process test.unit.container.test_reconciler.TestReconciler.test_multi_process_should_process ... ok test.unit.container.test_reconciler.TestReconciler.test_no_concurrency test.unit.container.test_reconciler.TestReconciler.test_no_concurrency ... ok test.unit.container.test_reconciler.TestReconciler.test_object_delete test.unit.container.test_reconciler.TestReconciler.test_object_delete ... ok test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop test.unit.container.test_reconciler.TestReconciler.test_object_enqueued_for_the_correct_dest_noop ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move test.unit.container.test_reconciler.TestReconciler.test_object_move ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_already_moved ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_newer_than_queue_entry ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! test DEBUG: looking for objects in 1716724800 test DEBUG: checking container 1716721200 test DEBUG: looking for objects in 1716721200 test DEBUG: checking container 1716717600 test DEBUG: looking for objects in 1716717600 test DEBUG: checking container 1716714000 test DEBUG: looking for objects in 1716714000 test DEBUG: checking container 1716710400 test DEBUG: looking for objects in 1716710400 test DEBUG: checking container 1716706800 test DEBUG: looking for objects in 1716706800 test INFO: Reconciler Stats: {} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test DEBUG: checking placement for '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 test INFO: success: '/AUTH_bob/c/o2' (3724.234560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o2' (3724.234560) from the queue (0000003724.23456_0000000000000001) test INFO: Reconciler Stats: {'success': 2, 'pop_queue': 2} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347548.865560) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347548.865560) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347548.865560) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347548.865560) from storage policy 1 test DEBUG: delete_attempt: '/AUTH_bob/c/o1' (1717347548.865560) in policy_index 1 will be deleted from policy_index 0 (1717347548.86556_0000000000000002) test INFO: delete_success: '/AUTH_bob/c/o1' (1717347548.865560) deleted from policy_index 0 (1717347548.86556_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (1717347548.865560) from policy_index 1 (1717347548.86556_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (1717347548.865560) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (1717347548.865560) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1717347548.865560) from the queue (1717347548.86556_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'delete_attempt': 1, 'delete_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: noop_object: '/AUTH_bob/c/o1' (3618.841870) container policy_index 1 matches queue policy index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'noop_object': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3679.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3679.201900) in destination policy_index 0 test DEBUG: found_object: '/AUTH_bob/c/o1' (3680.201900) in policy_index 0 is newer than queue (3679.201900) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3679.201900) from policy_index 1 (0000003679.20190_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3679.201900) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3679.201900) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3679.201900) from the queue (0000003679.20190_0000000000000001) test INFO: Reconciler Stats: {'found_object': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383930) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 should be in policy_index 0 test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_dest_object_older_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_cleanup ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight test.unit.container.test_reconciler.TestReconciler.test_object_move_fails_preflight ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_ancient ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent test.unit.container.test_reconciler.TestReconciler.test_object_move_no_such_object_no_tombstone_recent ... ok test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383930) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (36123.383930) moved from policy_index 1 to policy_index 0 (0000036123.38393_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 (0000036123.38393_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (36123.383930) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/36000/1:/AUTH_bob/c/o1' (36123.383930) from the queue (0000036123.38393_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.123460) in policy_index 1 will be moved to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.123460) moved from policy_index 1 to policy_index 0 (0000003600.12346_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_failed: '/AUTH_bob/c/o1' (3600.123456) was not cleaned up in storage_policy 1 (Unexpected response: 503 Service Unavailable) test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: unavailable_destination: '/AUTH_bob/c/o1' (3600.123456) unable to determine the destination timestamp, if any test INFO: retry: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_destination': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 1716681600 test DEBUG: looking for objects in 1716681600 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1716682268.986190) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1716682268.986190) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1716682268.986190) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1716682268.986190) from storage policy 1 test CRITICAL: lost_source: '/AUTH_jeb/c/o1' (1716682268.98619) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_jeb/c/o1' (1716682268.986190) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/1716681600/1:/AUTH_jeb/c/o1' (1716682268.986190) from the queue (1716682268.98619_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_jeb/c/o1' (1717347548.996803) in policy_index 1 test DEBUG: checking for '/AUTH_jeb/c/o1' (1717347548.996803) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_jeb/c/o1' (1717347548.996803) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_jeb/c/o1' (1717347548.996803) from storage policy 1 test WARNING: unavailable_source: '/AUTH_jeb/c/o1' (1717347548.99680) in policy_index 1 responded 404 (0000000000.00000) test INFO: retry: '/AUTH_jeb/c/o1' (1717347548.996803) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town test.unit.container.test_reconciler.TestReconciler.test_object_move_put_blows_up_crazy_town ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails test.unit.container.test_reconciler.TestReconciler.test_object_move_put_fails ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_is_forever_gone ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_newer_than_queue_entry ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry test.unit.container.test_reconciler.TestReconciler.test_object_move_src_object_older_than_queue_entry ... ok Traceback (most recent call last): File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 471, in fire_timers timer() File "/usr/lib/python3/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test ERROR: unhandled_error: unable to upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 635, in ensure_object_in_right_location self.swift.upload_object( File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 844, in upload_object self.handle_request('PUT', path, headers, acceptable_statuses, fobj, File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 264, in handle_request resp = self.make_request(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 1923, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'unhandled_error': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 36000 test DEBUG: looking for objects in 36000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (36123.383925) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (36123.383925) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (36123.383930) in policy_index 1 will be moved to policy_index 0 (0000036123.38393_0000000000000002) test WARNING: copy_failed: upload '/AUTH_bob/c/o1' (36123.383930) from policy_index 1 to policy_index 0 returned Unexpected response: 503 Service Unavailable test INFO: retry: '/AUTH_bob/c/o1' (36123.383925) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_failed': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1716739199.000000) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1716739199.000000) from storage policy 1 test CRITICAL: lost_source: '/AUTH_bob/c/o1' (1716739199.00000) was not available in policy_index 1 and has expired test INFO: success: '/AUTH_bob/c/o1' (1716739199.000000) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (1716739199.000000) from the queue (1716739199.00000_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'lost_source': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (3600.123456) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (3600.123456) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3600.234570) in policy_index 1 will be moved to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3600.234570) moved from policy_index 1 to policy_index 0 (0000003600.23457_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3600.123456) from policy_index 1 (0000003600.12346_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3600.123456) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c/o1' (3600.123456) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c/o1' (3600.123456) from the queue (0000003600.12346_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347549.057159) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347549.057159) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347549.057159) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347549.057159) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717347549.05716) in policy_index 1 responded 200 (1717347548.05716) test INFO: retry: '/AUTH_bob/c/o1' (1717347549.057159) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 test DEBUG: checking for '/AUTH_bob/c/o1' (3618.841870) in destination policy_index 1 test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 should be in policy_index 1 test DEBUG: fetching '/AUTH_bob/c/o1' (3618.841870) from storage policy 0 test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction test.unit.container.test_reconciler.TestReconciler.test_object_move_the_other_direction ... ok test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces test.unit.container.test_reconciler.TestReconciler.test_object_move_with_unicode_and_spaces ... ok test.unit.container.test_reconciler.TestReconciler.test_processes_config test.unit.container.test_reconciler.TestReconciler.test_processes_config ... ok test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name test.unit.container.test_reconciler.TestReconciler.test_queue_name_with_policy_index_delimiter_in_name ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_server_error ... ok test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone test.unit.container.test_reconciler.TestReconciler.test_src_object_unavailable_with_slightly_newer_tombstone ... ok test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy test.unit.container.test_reconciler.TestReconciler.test_unable_to_direct_get_oldest_storage_policy ... ok test DEBUG: copy_attempt: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 will be moved to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c/o1' (3618.841870) moved from policy_index 0 to policy_index 1 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c/o1' (3618.841870) from policy_index 0 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c/o1' (3618.841870) was successfully removed from policy_index 0 test INFO: success: '/AUTH_bob/c/o1' (3618.841870) in policy_index 0 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/0:/AUTH_bob/c/o1' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c ت/o1 ت' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c ت/o1 ت' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c ت/o1 ت' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c ت/o1 ت' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c ت/o1 ت' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c ت/o1 ت' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in destination policy_index 0 test DEBUG: misplaced_object: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from storage policy 1 test DEBUG: copy_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 will be moved to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: copy_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) moved from policy_index 1 to policy_index 0 (0000003618.84187_0000000000000002) test DEBUG: cleanup_attempt: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from policy_index 1 (0000003618.84187_0000000000000001) will be deleted test DEBUG: cleanup_success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) was successfully removed from policy_index 1 test INFO: success: '/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) in policy_index 1 was handled successfully test DEBUG: pop_queue: remove '/.misplaced_objects/3600/1:/AUTH_bob/c:sneaky/o1:sneaky' (3618.841870) from the queue (0000003618.84187_0000000000000001) test INFO: Reconciler Stats: {'misplaced_object': 1, 'copy_attempt': 1, 'copy_success': 1, 'cleanup_attempt': 1, 'cleanup_success': 1, 'success': 1, 'pop_queue': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347549.104076) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347549.104076) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347549.104076) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347549.104076) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717347549.10408) in policy_index 1 responded 503 (0000000000.00000) test INFO: retry: '/AUTH_bob/c/o1' (1717347549.104076) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: checking placement for '/AUTH_bob/c/o1' (1717347549.112184) in policy_index 1 test DEBUG: checking for '/AUTH_bob/c/o1' (1717347549.112184) in destination policy_index 0 test DEBUG: looking for containers in .misplaced_objects test DEBUG: misplaced_object: '/AUTH_bob/c/o1' (1717347549.112184) in policy_index 1 should be in policy_index 0 test DEBUG: fetching '/AUTH_bob/c/o1' (1717347549.112184) from storage policy 1 test WARNING: unavailable_source: '/AUTH_bob/c/o1' (1717347549.11218) in policy_index 1 responded 404 (1717347549.11218_0000000000000002) test INFO: retry: '/AUTH_bob/c/o1' (1717347549.112184) in policy_index 1 must be retried test INFO: Reconciler Stats: {'misplaced_object': 1, 'unavailable_source': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test DEBUG: checking container 3600 test DEBUG: looking for objects in 3600 test DEBUG: checking placement for '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 test DEBUG: unavailable_container: '/AUTH_bob/c/o1' (3618.841870) unable to determine the destination policy_index test INFO: retry: '/AUTH_bob/c/o1' (3618.841870) in policy_index 1 must be retried test INFO: Reconciler Stats: {'unavailable_container': 1, 'retry': 1} test DEBUG: pulling items from the queue test DEBUG: checking container 1717344000 test DEBUG: looking for objects in 1717344000 test DEBUG: looking for containers in .misplaced_objects test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile test.unit.container.test_reconciler.TestReconciler.test_unhandled_exception_in_reconcile ... ok test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts test.unit.container.test_sharder.TestSharder.test_audit_cleave_contexts ... ok test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container test.unit.container.test_sharder.TestSharder.test_audit_deleted_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container ... ok test ERROR: Unhandled Exception trying to reconcile: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 852, in run_once self.reconcile() File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 831, in reconcile for container in self._iter_containers(): File "/build/reproducible-path/swift-2.33.0/swift/container/reconciler.py", line 748, in _iter_containers one_page = list(itertools.islice( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 331, in _iter_items resp = self.make_request( ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 261, in make_request six.reraise(exc_type, exc_value, exc_traceback) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/build/reproducible-path/swift-2.33.0/swift/common/internal_client.py", line 231, in make_request resp = spawn(req.get_response, self.app).wait() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 224, in wait return self._exit_event.wait() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/event.py", line 124, in wait result = hub.switch() ^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 310, in switch return self.greenlet.switch() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/eventlet/greenthread.py", line 264, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1165, in get_response status, headers, app_iter = self.call_application(application) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/swob.py", line 1149, in call_application app_iter = application(self.environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 86, in __call__ return self.storage_policy[policy_index].__call__( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/common/middleware/helpers.py", line 324, in __call__ resp = resp_class( ^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_reconciler.py", line 2128, in blow_up raise Exception('kaboom!') Exception: kaboom! test INFO: Reconciler Stats: {} sharder-test WARNING: Reclaimable db stuck waiting for shrinking, path: a/c, db: /tmp/tmp8u6ydlg6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717347550.00000, (0, 0) as of 1717347550.00000, active as of 1717347550.00000> to ShardRange<'k' to 't' as of 1717347551.00000, (0, 0) as of 1717347551.00000, active as of 1717347551.00000>, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347555.00000, (0, 0) as of 1717347555.00000, shrinking as of 1717347552.00000> to ShardRange<'k' to 't' as of 1717347555.00000, (0, 0) as of 1717347555.00000, sharding as of 1717347556.00000>, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpvg4p12ku/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347550.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/057/40e3dc542884345ef9d28bc544619057/40e3dc542884345ef9d28bc544619057.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347552.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/426/c3285f5d31c2ac33591cc5e74489d426/c3285f5d31c2ac33591cc5e74489d426.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347551.00000, (0, 0) as of 1717347551.00000, found as of 1717347551.00000> to ShardRange<'k' to 't' as of 1717347552.00000, (0, 0) as of 1717347552.00000, created as of 1717347552.00000>, path: .shards_a/shard_c_1717347552.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/426/c3285f5d31c2ac33591cc5e74489d426/c3285f5d31c2ac33591cc5e74489d426.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347554.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/dbc/c25e14406212ad28776c27c097f87dbc/c25e14406212ad28776c27c097f87dbc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347553.00000, (0, 0) as of 1717347553.00000, found as of 1717347553.00000> to ShardRange<'k' to 't' as of 1717347554.00000, (0, 0) as of 1717347554.00000, cleaved as of 1717347554.00000>, path: .shards_a/shard_c_1717347554.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/dbc/c25e14406212ad28776c27c097f87dbc/c25e14406212ad28776c27c097f87dbc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347556.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/695/94d76f8e7fa9eb2483132aff757a8695/94d76f8e7fa9eb2483132aff757a8695.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347555.00000, (0, 0) as of 1717347555.00000, found as of 1717347555.00000> to ShardRange<'k' to 't' as of 1717347556.00000, (0, 0) as of 1717347556.00000, active as of 1717347556.00000>, path: .shards_a/shard_c_1717347556.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/695/94d76f8e7fa9eb2483132aff757a8695/94d76f8e7fa9eb2483132aff757a8695.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347558.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/07a/a392568e4c9676ee33b90e3e97fbf07a/a392568e4c9676ee33b90e3e97fbf07a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347557.00000, (0, 0) as of 1717347557.00000, found as of 1717347557.00000> to ShardRange<'k' to 't' as of 1717347558.00000, (0, 0) as of 1717347558.00000, shrinking as of 1717347558.00000>, path: .shards_a/shard_c_1717347558.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/07a/a392568e4c9676ee33b90e3e97fbf07a/a392568e4c9676ee33b90e3e97fbf07a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347558.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/07a/a392568e4c9676ee33b90e3e97fbf07a/a392568e4c9676ee33b90e3e97fbf07a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347560.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c44/a56e8d901279d99b823c1c15cf377c44/a56e8d901279d99b823c1c15cf377c44.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347559.00000, (0, 0) as of 1717347559.00000, found as of 1717347559.00000> to ShardRange<'k' to 't' as of 1717347560.00000, (0, 0) as of 1717347560.00000, sharding as of 1717347560.00000>, path: .shards_a/shard_c_1717347560.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c44/a56e8d901279d99b823c1c15cf377c44/a56e8d901279d99b823c1c15cf377c44.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347560.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c44/a56e8d901279d99b823c1c15cf377c44/a56e8d901279d99b823c1c15cf377c44.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347562.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/65f/d86da1e9668f0385df9cc823cbbe965f/d86da1e9668f0385df9cc823cbbe965f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347561.00000, (0, 0) as of 1717347561.00000, found as of 1717347561.00000> to ShardRange<'k' to 't' as of 1717347562.00000, (0, 0) as of 1717347562.00000, sharded as of 1717347562.00000>, path: .shards_a/shard_c_1717347562.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/65f/d86da1e9668f0385df9cc823cbbe965f/d86da1e9668f0385df9cc823cbbe965f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347562.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/65f/d86da1e9668f0385df9cc823cbbe965f/d86da1e9668f0385df9cc823cbbe965f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347564.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/dbc/b85b721a2bf827dd4dd699715528edbc/b85b721a2bf827dd4dd699715528edbc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347563.00000, (0, 0) as of 1717347563.00000, found as of 1717347563.00000> to ShardRange<'k' to 't' as of 1717347564.00000, (0, 0) as of 1717347564.00000, shrunk as of 1717347564.00000>, path: .shards_a/shard_c_1717347564.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/dbc/b85b721a2bf827dd4dd699715528edbc/b85b721a2bf827dd4dd699715528edbc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347564.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/dbc/b85b721a2bf827dd4dd699715528edbc/b85b721a2bf827dd4dd699715528edbc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347566.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/50d/f030fc9c67cb78fee47724a398e8d50d/f030fc9c67cb78fee47724a398e8d50d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347565.00000, (0, 0) as of 1717347565.00000, created as of 1717347565.00000> to ShardRange<'k' to 't' as of 1717347566.00000, (0, 0) as of 1717347566.00000, found as of 1717347566.00000>, path: .shards_a/shard_c_1717347566.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/50d/f030fc9c67cb78fee47724a398e8d50d/f030fc9c67cb78fee47724a398e8d50d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347568.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ef0/f71a100a22f811bbda9c97d9f73bdef0/f71a100a22f811bbda9c97d9f73bdef0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347570.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/66f/3ea1508833ca34888ee59667a769c66f/3ea1508833ca34888ee59667a769c66f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347569.00000, (0, 0) as of 1717347569.00000, created as of 1717347569.00000> to ShardRange<'k' to 't' as of 1717347570.00000, (0, 0) as of 1717347570.00000, cleaved as of 1717347570.00000>, path: .shards_a/shard_c_1717347570.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/66f/3ea1508833ca34888ee59667a769c66f/3ea1508833ca34888ee59667a769c66f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347572.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ff4/713561672bb080d260f330fbb602dff4/713561672bb080d260f330fbb602dff4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347571.00000, (0, 0) as of 1717347571.00000, created as of 1717347571.00000> to ShardRange<'k' to 't' as of 1717347572.00000, (0, 0) as of 1717347572.00000, active as of 1717347572.00000>, path: .shards_a/shard_c_1717347572.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ff4/713561672bb080d260f330fbb602dff4/713561672bb080d260f330fbb602dff4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347574.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/469/07dd0e0d3b34ab0626b9e9c274c4b469/07dd0e0d3b34ab0626b9e9c274c4b469.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347573.00000, (0, 0) as of 1717347573.00000, created as of 1717347573.00000> to ShardRange<'k' to 't' as of 1717347574.00000, (0, 0) as of 1717347574.00000, shrinking as of 1717347574.00000>, path: .shards_a/shard_c_1717347574.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/469/07dd0e0d3b34ab0626b9e9c274c4b469/07dd0e0d3b34ab0626b9e9c274c4b469.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347574.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/469/07dd0e0d3b34ab0626b9e9c274c4b469/07dd0e0d3b34ab0626b9e9c274c4b469.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347576.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/5bc/42c16e333b034088143999ff506305bc/42c16e333b034088143999ff506305bc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347575.00000, (0, 0) as of 1717347575.00000, created as of 1717347575.00000> to ShardRange<'k' to 't' as of 1717347576.00000, (0, 0) as of 1717347576.00000, sharding as of 1717347576.00000>, path: .shards_a/shard_c_1717347576.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/5bc/42c16e333b034088143999ff506305bc/42c16e333b034088143999ff506305bc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347576.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/5bc/42c16e333b034088143999ff506305bc/42c16e333b034088143999ff506305bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347578.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/b1c/4e4191824acd4731333030325abe7b1c/4e4191824acd4731333030325abe7b1c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347577.00000, (0, 0) as of 1717347577.00000, created as of 1717347577.00000> to ShardRange<'k' to 't' as of 1717347578.00000, (0, 0) as of 1717347578.00000, sharded as of 1717347578.00000>, path: .shards_a/shard_c_1717347578.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/b1c/4e4191824acd4731333030325abe7b1c/4e4191824acd4731333030325abe7b1c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347578.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/b1c/4e4191824acd4731333030325abe7b1c/4e4191824acd4731333030325abe7b1c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347580.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2b7/4036a7e8891da2063afc2b4db2a662b7/4036a7e8891da2063afc2b4db2a662b7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347579.00000, (0, 0) as of 1717347579.00000, created as of 1717347579.00000> to ShardRange<'k' to 't' as of 1717347580.00000, (0, 0) as of 1717347580.00000, shrunk as of 1717347580.00000>, path: .shards_a/shard_c_1717347580.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2b7/4036a7e8891da2063afc2b4db2a662b7/4036a7e8891da2063afc2b4db2a662b7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347580.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2b7/4036a7e8891da2063afc2b4db2a662b7/4036a7e8891da2063afc2b4db2a662b7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347582.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c96/9c35ebf2c2da9a49336dccc3a6359c96/9c35ebf2c2da9a49336dccc3a6359c96.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347581.00000, (0, 0) as of 1717347581.00000, cleaved as of 1717347581.00000> to ShardRange<'k' to 't' as of 1717347582.00000, (0, 0) as of 1717347582.00000, found as of 1717347582.00000>, path: .shards_a/shard_c_1717347582.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c96/9c35ebf2c2da9a49336dccc3a6359c96/9c35ebf2c2da9a49336dccc3a6359c96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347584.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/84c/a7591461facd36eb69c99cee2c86784c/a7591461facd36eb69c99cee2c86784c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347583.00000, (0, 0) as of 1717347583.00000, cleaved as of 1717347583.00000> to ShardRange<'k' to 't' as of 1717347584.00000, (0, 0) as of 1717347584.00000, created as of 1717347584.00000>, path: .shards_a/shard_c_1717347584.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/84c/a7591461facd36eb69c99cee2c86784c/a7591461facd36eb69c99cee2c86784c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347586.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c90/d3cac2eace99463e1ee8678c0c2edc90/d3cac2eace99463e1ee8678c0c2edc90.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347588.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/0ee/70f553867d7924fab8394588b10c10ee/70f553867d7924fab8394588b10c10ee.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347587.00000, (0, 0) as of 1717347587.00000, cleaved as of 1717347587.00000> to ShardRange<'k' to 't' as of 1717347588.00000, (0, 0) as of 1717347588.00000, active as of 1717347588.00000>, path: .shards_a/shard_c_1717347588.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/0ee/70f553867d7924fab8394588b10c10ee/70f553867d7924fab8394588b10c10ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347590.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9fb/b05de8fabece49a9efb290faebccd9fb/b05de8fabece49a9efb290faebccd9fb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347589.00000, (0, 0) as of 1717347589.00000, cleaved as of 1717347589.00000> to ShardRange<'k' to 't' as of 1717347590.00000, (0, 0) as of 1717347590.00000, shrinking as of 1717347590.00000>, path: .shards_a/shard_c_1717347590.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9fb/b05de8fabece49a9efb290faebccd9fb/b05de8fabece49a9efb290faebccd9fb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347590.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9fb/b05de8fabece49a9efb290faebccd9fb/b05de8fabece49a9efb290faebccd9fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347592.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/26d/9bc117fa0b6d6f708f944e0a49ace26d/9bc117fa0b6d6f708f944e0a49ace26d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347591.00000, (0, 0) as of 1717347591.00000, cleaved as of 1717347591.00000> to ShardRange<'k' to 't' as of 1717347592.00000, (0, 0) as of 1717347592.00000, sharding as of 1717347592.00000>, path: .shards_a/shard_c_1717347592.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/26d/9bc117fa0b6d6f708f944e0a49ace26d/9bc117fa0b6d6f708f944e0a49ace26d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347592.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/26d/9bc117fa0b6d6f708f944e0a49ace26d/9bc117fa0b6d6f708f944e0a49ace26d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347594.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/729/a7b4c4357436e90778b65b0de8d2a729/a7b4c4357436e90778b65b0de8d2a729.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347593.00000, (0, 0) as of 1717347593.00000, cleaved as of 1717347593.00000> to ShardRange<'k' to 't' as of 1717347594.00000, (0, 0) as of 1717347594.00000, sharded as of 1717347594.00000>, path: .shards_a/shard_c_1717347594.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/729/a7b4c4357436e90778b65b0de8d2a729/a7b4c4357436e90778b65b0de8d2a729.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347594.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/729/a7b4c4357436e90778b65b0de8d2a729/a7b4c4357436e90778b65b0de8d2a729.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347596.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d3b/480689c569ecc2b36effcec24d25dd3b/480689c569ecc2b36effcec24d25dd3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347595.00000, (0, 0) as of 1717347595.00000, cleaved as of 1717347595.00000> to ShardRange<'k' to 't' as of 1717347596.00000, (0, 0) as of 1717347596.00000, shrunk as of 1717347596.00000>, path: .shards_a/shard_c_1717347596.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d3b/480689c569ecc2b36effcec24d25dd3b/480689c569ecc2b36effcec24d25dd3b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347596.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d3b/480689c569ecc2b36effcec24d25dd3b/480689c569ecc2b36effcec24d25dd3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347598.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/538/ed951779bae1d8879da21e18cbb6d538/ed951779bae1d8879da21e18cbb6d538.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347597.00000, (0, 0) as of 1717347597.00000, active as of 1717347597.00000> to ShardRange<'k' to 't' as of 1717347598.00000, (0, 0) as of 1717347598.00000, found as of 1717347598.00000>, path: .shards_a/shard_c_1717347598.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/538/ed951779bae1d8879da21e18cbb6d538/ed951779bae1d8879da21e18cbb6d538.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347600.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/b02/cb8813844798d018fb6a75a5cd266b02/cb8813844798d018fb6a75a5cd266b02.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347599.00000, (0, 0) as of 1717347599.00000, active as of 1717347599.00000> to ShardRange<'k' to 't' as of 1717347600.00000, (0, 0) as of 1717347600.00000, created as of 1717347600.00000>, path: .shards_a/shard_c_1717347600.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/b02/cb8813844798d018fb6a75a5cd266b02/cb8813844798d018fb6a75a5cd266b02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347602.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/da4/40ba35370c26094f14b4a673493e2da4/40ba35370c26094f14b4a673493e2da4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347601.00000, (0, 0) as of 1717347601.00000, active as of 1717347601.00000> to ShardRange<'k' to 't' as of 1717347602.00000, (0, 0) as of 1717347602.00000, cleaved as of 1717347602.00000>, path: .shards_a/shard_c_1717347602.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/da4/40ba35370c26094f14b4a673493e2da4/40ba35370c26094f14b4a673493e2da4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347604.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/50b/6cf3babe52a190d3bc41512207fff50b/6cf3babe52a190d3bc41512207fff50b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347606.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/912/bac54f4ccd40aea7d16f45db0b37f912/bac54f4ccd40aea7d16f45db0b37f912.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347605.00000, (0, 0) as of 1717347605.00000, active as of 1717347605.00000> to ShardRange<'k' to 't' as of 1717347606.00000, (0, 0) as of 1717347606.00000, shrinking as of 1717347606.00000>, path: .shards_a/shard_c_1717347606.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/912/bac54f4ccd40aea7d16f45db0b37f912/bac54f4ccd40aea7d16f45db0b37f912.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347606.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/912/bac54f4ccd40aea7d16f45db0b37f912/bac54f4ccd40aea7d16f45db0b37f912.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347608.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/730/2192113353efc0b141e01898e0368730/2192113353efc0b141e01898e0368730.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347607.00000, (0, 0) as of 1717347607.00000, active as of 1717347607.00000> to ShardRange<'k' to 't' as of 1717347608.00000, (0, 0) as of 1717347608.00000, sharding as of 1717347608.00000>, path: .shards_a/shard_c_1717347608.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/730/2192113353efc0b141e01898e0368730/2192113353efc0b141e01898e0368730.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347608.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/730/2192113353efc0b141e01898e0368730/2192113353efc0b141e01898e0368730.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347610.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/bfc/e6f9e2b8479867c543b80ee727448bfc/e6f9e2b8479867c543b80ee727448bfc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347609.00000, (0, 0) as of 1717347609.00000, active as of 1717347609.00000> to ShardRange<'k' to 't' as of 1717347610.00000, (0, 0) as of 1717347610.00000, sharded as of 1717347610.00000>, path: .shards_a/shard_c_1717347610.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/bfc/e6f9e2b8479867c543b80ee727448bfc/e6f9e2b8479867c543b80ee727448bfc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347610.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/bfc/e6f9e2b8479867c543b80ee727448bfc/e6f9e2b8479867c543b80ee727448bfc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347612.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/554/857484de50bb64537346fc4482491554/857484de50bb64537346fc4482491554.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347611.00000, (0, 0) as of 1717347611.00000, active as of 1717347611.00000> to ShardRange<'k' to 't' as of 1717347612.00000, (0, 0) as of 1717347612.00000, shrunk as of 1717347612.00000>, path: .shards_a/shard_c_1717347612.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/554/857484de50bb64537346fc4482491554/857484de50bb64537346fc4482491554.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347612.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/554/857484de50bb64537346fc4482491554/857484de50bb64537346fc4482491554.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347614.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/288/772b41b595459e0fad474cb66cc83288/772b41b595459e0fad474cb66cc83288.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347613.00000, (0, 0) as of 1717347613.00000, shrinking as of 1717347613.00000> to ShardRange<'k' to 't' as of 1717347614.00000, (0, 0) as of 1717347614.00000, found as of 1717347614.00000>, path: .shards_a/shard_c_1717347614.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/288/772b41b595459e0fad474cb66cc83288/772b41b595459e0fad474cb66cc83288.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347616.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d09/5b172d789fcae78d96e28231ebf61d09/5b172d789fcae78d96e28231ebf61d09.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347615.00000, (0, 0) as of 1717347615.00000, shrinking as of 1717347615.00000> to ShardRange<'k' to 't' as of 1717347616.00000, (0, 0) as of 1717347616.00000, created as of 1717347616.00000>, path: .shards_a/shard_c_1717347616.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d09/5b172d789fcae78d96e28231ebf61d09/5b172d789fcae78d96e28231ebf61d09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347618.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/975/d8d889042ecf29323a1f19221a2ca975/d8d889042ecf29323a1f19221a2ca975.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347617.00000, (0, 0) as of 1717347617.00000, shrinking as of 1717347617.00000> to ShardRange<'k' to 't' as of 1717347618.00000, (0, 0) as of 1717347618.00000, cleaved as of 1717347618.00000>, path: .shards_a/shard_c_1717347618.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/975/d8d889042ecf29323a1f19221a2ca975/d8d889042ecf29323a1f19221a2ca975.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347620.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347619.00000, (0, 0) as of 1717347619.00000, shrinking as of 1717347619.00000> to ShardRange<'k' to 't' as of 1717347620.00000, (0, 0) as of 1717347620.00000, active as of 1717347620.00000>, path: .shards_a/shard_c_1717347620.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347622.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/053/a4e09fd8c8ae6bae6244b48a3bbf0053/a4e09fd8c8ae6bae6244b48a3bbf0053.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347622.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/053/a4e09fd8c8ae6bae6244b48a3bbf0053/a4e09fd8c8ae6bae6244b48a3bbf0053.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347624.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4fb/f0194b74a2cc438488b7bab88c7c84fb/f0194b74a2cc438488b7bab88c7c84fb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347623.00000, (0, 0) as of 1717347623.00000, shrinking as of 1717347623.00000> to ShardRange<'k' to 't' as of 1717347624.00000, (0, 0) as of 1717347624.00000, sharding as of 1717347624.00000>, path: .shards_a/shard_c_1717347624.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4fb/f0194b74a2cc438488b7bab88c7c84fb/f0194b74a2cc438488b7bab88c7c84fb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347624.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4fb/f0194b74a2cc438488b7bab88c7c84fb/f0194b74a2cc438488b7bab88c7c84fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347626.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/7cc/44083eb73e0b2a9ecfe13c79685d67cc/44083eb73e0b2a9ecfe13c79685d67cc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347625.00000, (0, 0) as of 1717347625.00000, shrinking as of 1717347625.00000> to ShardRange<'k' to 't' as of 1717347626.00000, (0, 0) as of 1717347626.00000, sharded as of 1717347626.00000>, path: .shards_a/shard_c_1717347626.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/7cc/44083eb73e0b2a9ecfe13c79685d67cc/44083eb73e0b2a9ecfe13c79685d67cc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347626.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/7cc/44083eb73e0b2a9ecfe13c79685d67cc/44083eb73e0b2a9ecfe13c79685d67cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347628.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/0e9/6b924c534b5577f457d8cb4a8afa80e9/6b924c534b5577f457d8cb4a8afa80e9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347627.00000, (0, 0) as of 1717347627.00000, shrinking as of 1717347627.00000> to ShardRange<'k' to 't' as of 1717347628.00000, (0, 0) as of 1717347628.00000, shrunk as of 1717347628.00000>, path: .shards_a/shard_c_1717347628.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/0e9/6b924c534b5577f457d8cb4a8afa80e9/6b924c534b5577f457d8cb4a8afa80e9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347628.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/0e9/6b924c534b5577f457d8cb4a8afa80e9/6b924c534b5577f457d8cb4a8afa80e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347630.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/8f7/a3655c7da85a5b4b0846e60a8b13a8f7/a3655c7da85a5b4b0846e60a8b13a8f7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347629.00000, (0, 0) as of 1717347629.00000, sharding as of 1717347629.00000> to ShardRange<'k' to 't' as of 1717347630.00000, (0, 0) as of 1717347630.00000, found as of 1717347630.00000>, path: .shards_a/shard_c_1717347630.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/8f7/a3655c7da85a5b4b0846e60a8b13a8f7/a3655c7da85a5b4b0846e60a8b13a8f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347632.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/167/d15b78536e91bf1514e8847bc6809167/d15b78536e91bf1514e8847bc6809167.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347631.00000, (0, 0) as of 1717347631.00000, sharding as of 1717347631.00000> to ShardRange<'k' to 't' as of 1717347632.00000, (0, 0) as of 1717347632.00000, created as of 1717347632.00000>, path: .shards_a/shard_c_1717347632.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/167/d15b78536e91bf1514e8847bc6809167/d15b78536e91bf1514e8847bc6809167.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347634.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4f7/59813a133ee4d28bf3292c9fd1a074f7/59813a133ee4d28bf3292c9fd1a074f7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347633.00000, (0, 0) as of 1717347633.00000, sharding as of 1717347633.00000> to ShardRange<'k' to 't' as of 1717347634.00000, (0, 0) as of 1717347634.00000, cleaved as of 1717347634.00000>, path: .shards_a/shard_c_1717347634.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4f7/59813a133ee4d28bf3292c9fd1a074f7/59813a133ee4d28bf3292c9fd1a074f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347636.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/a30/8d8f0f1e5cebd170889c9634ae569a30/8d8f0f1e5cebd170889c9634ae569a30.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347635.00000, (0, 0) as of 1717347635.00000, sharding as of 1717347635.00000> to ShardRange<'k' to 't' as of 1717347636.00000, (0, 0) as of 1717347636.00000, active as of 1717347636.00000>, path: .shards_a/shard_c_1717347636.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/a30/8d8f0f1e5cebd170889c9634ae569a30/8d8f0f1e5cebd170889c9634ae569a30.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347638.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4dc/00c9534fdabeefdcabf68e2943f7f4dc/00c9534fdabeefdcabf68e2943f7f4dc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347637.00000, (0, 0) as of 1717347637.00000, sharding as of 1717347637.00000> to ShardRange<'k' to 't' as of 1717347638.00000, (0, 0) as of 1717347638.00000, shrinking as of 1717347638.00000>, path: .shards_a/shard_c_1717347638.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4dc/00c9534fdabeefdcabf68e2943f7f4dc/00c9534fdabeefdcabf68e2943f7f4dc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347638.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/4dc/00c9534fdabeefdcabf68e2943f7f4dc/00c9534fdabeefdcabf68e2943f7f4dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347640.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/60a/39531825230642a57978f47a5874660a/39531825230642a57978f47a5874660a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347640.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/60a/39531825230642a57978f47a5874660a/39531825230642a57978f47a5874660a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347642.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/07c/a49ec302fcc8bec304fbbf13ee39b07c/a49ec302fcc8bec304fbbf13ee39b07c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347641.00000, (0, 0) as of 1717347641.00000, sharding as of 1717347641.00000> to ShardRange<'k' to 't' as of 1717347642.00000, (0, 0) as of 1717347642.00000, sharded as of 1717347642.00000>, path: .shards_a/shard_c_1717347642.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/07c/a49ec302fcc8bec304fbbf13ee39b07c/a49ec302fcc8bec304fbbf13ee39b07c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347642.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/07c/a49ec302fcc8bec304fbbf13ee39b07c/a49ec302fcc8bec304fbbf13ee39b07c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347644.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c26/c2719e4be550e8f6bba30053f8b0dc26/c2719e4be550e8f6bba30053f8b0dc26.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347643.00000, (0, 0) as of 1717347643.00000, sharding as of 1717347643.00000> to ShardRange<'k' to 't' as of 1717347644.00000, (0, 0) as of 1717347644.00000, shrunk as of 1717347644.00000>, path: .shards_a/shard_c_1717347644.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c26/c2719e4be550e8f6bba30053f8b0dc26/c2719e4be550e8f6bba30053f8b0dc26.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347644.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c26/c2719e4be550e8f6bba30053f8b0dc26/c2719e4be550e8f6bba30053f8b0dc26.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347646.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/038/07dfc1d5715794bbc39721b607477038/07dfc1d5715794bbc39721b607477038.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347645.00000, (0, 0) as of 1717347645.00000, sharded as of 1717347645.00000> to ShardRange<'k' to 't' as of 1717347646.00000, (0, 0) as of 1717347646.00000, found as of 1717347646.00000>, path: .shards_a/shard_c_1717347646.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/038/07dfc1d5715794bbc39721b607477038/07dfc1d5715794bbc39721b607477038.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347648.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c42/9124a4aeb227914548aa009d3f504c42/9124a4aeb227914548aa009d3f504c42.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347647.00000, (0, 0) as of 1717347647.00000, sharded as of 1717347647.00000> to ShardRange<'k' to 't' as of 1717347648.00000, (0, 0) as of 1717347648.00000, created as of 1717347648.00000>, path: .shards_a/shard_c_1717347648.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/c42/9124a4aeb227914548aa009d3f504c42/9124a4aeb227914548aa009d3f504c42.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347650.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/173/fe560fbc02d686927dca800a9dbc3173/fe560fbc02d686927dca800a9dbc3173.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347649.00000, (0, 0) as of 1717347649.00000, sharded as of 1717347649.00000> to ShardRange<'k' to 't' as of 1717347650.00000, (0, 0) as of 1717347650.00000, cleaved as of 1717347650.00000>, path: .shards_a/shard_c_1717347650.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/173/fe560fbc02d686927dca800a9dbc3173/fe560fbc02d686927dca800a9dbc3173.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347652.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/fa6/d57b70349d40b1fb1d531ddd1e514fa6/d57b70349d40b1fb1d531ddd1e514fa6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347651.00000, (0, 0) as of 1717347651.00000, sharded as of 1717347651.00000> to ShardRange<'k' to 't' as of 1717347652.00000, (0, 0) as of 1717347652.00000, active as of 1717347652.00000>, path: .shards_a/shard_c_1717347652.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/fa6/d57b70349d40b1fb1d531ddd1e514fa6/d57b70349d40b1fb1d531ddd1e514fa6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347654.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6b4/c9afbccbfe36ae619835c127424ed6b4/c9afbccbfe36ae619835c127424ed6b4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347653.00000, (0, 0) as of 1717347653.00000, sharded as of 1717347653.00000> to ShardRange<'k' to 't' as of 1717347654.00000, (0, 0) as of 1717347654.00000, shrinking as of 1717347654.00000>, path: .shards_a/shard_c_1717347654.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6b4/c9afbccbfe36ae619835c127424ed6b4/c9afbccbfe36ae619835c127424ed6b4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347654.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6b4/c9afbccbfe36ae619835c127424ed6b4/c9afbccbfe36ae619835c127424ed6b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347656.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2a8/074cd982b7ad263001d12006e2a772a8/074cd982b7ad263001d12006e2a772a8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347655.00000, (0, 0) as of 1717347655.00000, sharded as of 1717347655.00000> to ShardRange<'k' to 't' as of 1717347656.00000, (0, 0) as of 1717347656.00000, sharding as of 1717347656.00000>, path: .shards_a/shard_c_1717347656.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2a8/074cd982b7ad263001d12006e2a772a8/074cd982b7ad263001d12006e2a772a8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347656.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2a8/074cd982b7ad263001d12006e2a772a8/074cd982b7ad263001d12006e2a772a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347658.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/142/72654e67a130da0b4c9994fa357d1142/72654e67a130da0b4c9994fa357d1142.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347658.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/142/72654e67a130da0b4c9994fa357d1142/72654e67a130da0b4c9994fa357d1142.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347660.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/cd2/1889c5322918afbc174706402e9d0cd2/1889c5322918afbc174706402e9d0cd2.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347659.00000, (0, 0) as of 1717347659.00000, sharded as of 1717347659.00000> to ShardRange<'k' to 't' as of 1717347660.00000, (0, 0) as of 1717347660.00000, shrunk as of 1717347660.00000>, path: .shards_a/shard_c_1717347660.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/cd2/1889c5322918afbc174706402e9d0cd2/1889c5322918afbc174706402e9d0cd2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347660.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/cd2/1889c5322918afbc174706402e9d0cd2/1889c5322918afbc174706402e9d0cd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347662.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f62/3c282437f4a35c0c8442b800f9bebf62/3c282437f4a35c0c8442b800f9bebf62.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347661.00000, (0, 0) as of 1717347661.00000, shrunk as of 1717347661.00000> to ShardRange<'k' to 't' as of 1717347662.00000, (0, 0) as of 1717347662.00000, found as of 1717347662.00000>, path: .shards_a/shard_c_1717347662.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f62/3c282437f4a35c0c8442b800f9bebf62/3c282437f4a35c0c8442b800f9bebf62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347664.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/7c3/e730357db3a99bf8e85a5edf45c3f7c3/e730357db3a99bf8e85a5edf45c3f7c3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347663.00000, (0, 0) as of 1717347663.00000, shrunk as of 1717347663.00000> to ShardRange<'k' to 't' as of 1717347664.00000, (0, 0) as of 1717347664.00000, created as of 1717347664.00000>, path: .shards_a/shard_c_1717347664.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/7c3/e730357db3a99bf8e85a5edf45c3f7c3/e730357db3a99bf8e85a5edf45c3f7c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347666.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/14e/298fa02c11aeb357735c26e5b2cbc14e/298fa02c11aeb357735c26e5b2cbc14e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347665.00000, (0, 0) as of 1717347665.00000, shrunk as of 1717347665.00000> to ShardRange<'k' to 't' as of 1717347666.00000, (0, 0) as of 1717347666.00000, cleaved as of 1717347666.00000>, path: .shards_a/shard_c_1717347666.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/14e/298fa02c11aeb357735c26e5b2cbc14e/298fa02c11aeb357735c26e5b2cbc14e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347668.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/892/5cb60017e4b5671717e514d1c6744892/5cb60017e4b5671717e514d1c6744892.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347667.00000, (0, 0) as of 1717347667.00000, shrunk as of 1717347667.00000> to ShardRange<'k' to 't' as of 1717347668.00000, (0, 0) as of 1717347668.00000, active as of 1717347668.00000>, path: .shards_a/shard_c_1717347668.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/892/5cb60017e4b5671717e514d1c6744892/5cb60017e4b5671717e514d1c6744892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347670.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/37b/8c09e68091cd469f053bfd06e509237b/8c09e68091cd469f053bfd06e509237b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347669.00000, (0, 0) as of 1717347669.00000, shrunk as of 1717347669.00000> to ShardRange<'k' to 't' as of 1717347670.00000, (0, 0) as of 1717347670.00000, shrinking as of 1717347670.00000>, path: .shards_a/shard_c_1717347670.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/37b/8c09e68091cd469f053bfd06e509237b/8c09e68091cd469f053bfd06e509237b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347670.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/37b/8c09e68091cd469f053bfd06e509237b/8c09e68091cd469f053bfd06e509237b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347672.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/91f/dca688e5b450c45ad7e0b8cc5bdfd91f/dca688e5b450c45ad7e0b8cc5bdfd91f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347671.00000, (0, 0) as of 1717347671.00000, shrunk as of 1717347671.00000> to ShardRange<'k' to 't' as of 1717347672.00000, (0, 0) as of 1717347672.00000, sharding as of 1717347672.00000>, path: .shards_a/shard_c_1717347672.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/91f/dca688e5b450c45ad7e0b8cc5bdfd91f/dca688e5b450c45ad7e0b8cc5bdfd91f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347672.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/91f/dca688e5b450c45ad7e0b8cc5bdfd91f/dca688e5b450c45ad7e0b8cc5bdfd91f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347674.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6e6/a0ac6329c280c463624b6d8e558a06e6/a0ac6329c280c463624b6d8e558a06e6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347673.00000, (0, 0) as of 1717347673.00000, shrunk as of 1717347673.00000> to ShardRange<'k' to 't' as of 1717347674.00000, (0, 0) as of 1717347674.00000, sharded as of 1717347674.00000>, path: .shards_a/shard_c_1717347674.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6e6/a0ac6329c280c463624b6d8e558a06e6/a0ac6329c280c463624b6d8e558a06e6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347674.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6e6/a0ac6329c280c463624b6d8e558a06e6/a0ac6329c280c463624b6d8e558a06e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347676.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/3bc/2f5681d343374c08c138b5a9c47f63bc/2f5681d343374c08c138b5a9c47f63bc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347676.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/3bc/2f5681d343374c08c138b5a9c47f63bc/2f5681d343374c08c138b5a9c47f63bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347677.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/806/a2ae0f687d4f3ea924675705f8cc1806/a2ae0f687d4f3ea924675705f8cc1806.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347679.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/287/d04db9bf3ccac5addde1761632090287/d04db9bf3ccac5addde1761632090287.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347681.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/984/f4a851d136ea3fb70382262bac778984/f4a851d136ea3fb70382262bac778984.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347683.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e8c/da4e0e0325a7f00da40f4745f3f0ce8c/da4e0e0325a7f00da40f4745f3f0ce8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347685.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d34/5258307d779b5ac1d07b8d6040a2bd34/5258307d779b5ac1d07b8d6040a2bd34.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347687.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/3f6/8ad20b4a4129245db03b967149b8d3f6/8ad20b4a4129245db03b967149b8d3f6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347689.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/dcd/f29578b93dda1a13df2c9475d7b28dcd/f29578b93dda1a13df2c9475d7b28dcd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347691.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/61e/b0aacf91ea1401ad2d728368aa35f61e/b0aacf91ea1401ad2d728368aa35f61e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347693.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/15b/d12225e411e66742aa8cdf258078115b/d12225e411e66742aa8cdf258078115b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347695.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/532/52956f13c3ed5f2f9bd0928199c77532/52956f13c3ed5f2f9bd0928199c77532.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347697.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/783/eb360b501ebd66d0ac6ef38760e40783/eb360b501ebd66d0ac6ef38760e40783.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347699.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ee6/88a24c4ca91fc6ee40ceac03ae63fee6/88a24c4ca91fc6ee40ceac03ae63fee6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347701.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/541/89e441df910ae09679d6c259d1223541/89e441df910ae09679d6c259d1223541.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347703.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/1e6/447dfad4870f604b7ef9da37dd5341e6/447dfad4870f604b7ef9da37dd5341e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347705.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/641/7fb7a4aeea0107d5a213f7f332eea641/7fb7a4aeea0107d5a213f7f332eea641.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347707.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/77e/e2ae7477d24c595a81a662f8049f777e/e2ae7477d24c595a81a662f8049f777e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347709.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2bf/ac798cc03a8462e14b05271c0e26f2bf/ac798cc03a8462e14b05271c0e26f2bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347711.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/fec/01bf42e316fb45675e57f0b8073ebfec/01bf42e316fb45675e57f0b8073ebfec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347713.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ca8/6df4a68a8a7734566efb70095e24bca8/6df4a68a8a7734566efb70095e24bca8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347715.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/6d4/e2f329bf224e83d2845c07579d5ac6d4/e2f329bf224e83d2845c07579d5ac6d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347717.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/0ae/7663691181efb0a475df6832ecaf70ae/7663691181efb0a475df6832ecaf70ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347719.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9c1/d3f8f1c74636075987cb6dee406b89c1/d3f8f1c74636075987cb6dee406b89c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347721.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f5c/51b325346429387282c9b8ae5398bf5c/51b325346429387282c9b8ae5398bf5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347723.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/59b/2c01f66ed60c4ff4a8c7e3cdb6ea559b/2c01f66ed60c4ff4a8c7e3cdb6ea559b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347725.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/cc2/1d87c710aaa6069dbe6dcb10f9783cc2/1d87c710aaa6069dbe6dcb10f9783cc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347727.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/df7/b15201b52f7dedef31520f2b88b06df7/b15201b52f7dedef31520f2b88b06df7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347729.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/713/b44793ca64fed6ec80ab85bccc259713/b44793ca64fed6ec80ab85bccc259713.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347731.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/72d/84340448ece42bf8b763d9d22329c72d/84340448ece42bf8b763d9d22329c72d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347733.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/247/2f72086e9567be59fabae7b0f24fc247/2f72086e9567be59fabae7b0f24fc247.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347735.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/fed/9f27e843d6a46ca1498c3c8586ddffed/9f27e843d6a46ca1498c3c8586ddffed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347737.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9cf/4fd1e46222678d500a338950dbe429cf/4fd1e46222678d500a338950dbe429cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347739.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ab8/81e8f3dbb37962063734c2788e86cab8/81e8f3dbb37962063734c2788e86cab8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347741.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/87c/a8d69f38e6efed3092e68f6ec3a7787c/a8d69f38e6efed3092e68f6ec3a7787c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347741.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/87c/a8d69f38e6efed3092e68f6ec3a7787c/a8d69f38e6efed3092e68f6ec3a7787c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347743.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/76d/732d5acac6605750c4407bee124cf76d/732d5acac6605750c4407bee124cf76d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347743.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/76d/732d5acac6605750c4407bee124cf76d/732d5acac6605750c4407bee124cf76d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347745.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/70b/cbc55291c015632d748b6bfb26be570b/cbc55291c015632d748b6bfb26be570b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347745.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/70b/cbc55291c015632d748b6bfb26be570b/cbc55291c015632d748b6bfb26be570b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347747.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/3e4/168af284255765a320e4e9019cf1d3e4/168af284255765a320e4e9019cf1d3e4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347747.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/3e4/168af284255765a320e4e9019cf1d3e4/168af284255765a320e4e9019cf1d3e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347749.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/612/907f0f2b4f9eafcc461de12596f06612/907f0f2b4f9eafcc461de12596f06612.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347749.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/612/907f0f2b4f9eafcc461de12596f06612/907f0f2b4f9eafcc461de12596f06612.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347751.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/73a/2f075fec0558d09cfe1464b79172273a/2f075fec0558d09cfe1464b79172273a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347751.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/73a/2f075fec0558d09cfe1464b79172273a/2f075fec0558d09cfe1464b79172273a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347753.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/38a/a512729c82f210b0b637683f9185038a/a512729c82f210b0b637683f9185038a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347753.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/38a/a512729c82f210b0b637683f9185038a/a512729c82f210b0b637683f9185038a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347755.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/8ae/8e125fe70da828d36913a687b50bb8ae/8e125fe70da828d36913a687b50bb8ae.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347755.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/8ae/8e125fe70da828d36913a687b50bb8ae/8e125fe70da828d36913a687b50bb8ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347757.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/516/4d88add553ff68178db81a35ff8c4516/4d88add553ff68178db81a35ff8c4516.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347757.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/516/4d88add553ff68178db81a35ff8c4516/4d88add553ff68178db81a35ff8c4516.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347759.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/291/c523317f40985535da16fb0a6f708291/c523317f40985535da16fb0a6f708291.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347759.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/291/c523317f40985535da16fb0a6f708291/c523317f40985535da16fb0a6f708291.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347761.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d1e/517c182950428b8026e9d67a96075d1e/517c182950428b8026e9d67a96075d1e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347761.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/d1e/517c182950428b8026e9d67a96075d1e/517c182950428b8026e9d67a96075d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347763.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e9c/85555a3f841f726690cb6593d473de9c/85555a3f841f726690cb6593d473de9c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347763.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e9c/85555a3f841f726690cb6593d473de9c/85555a3f841f726690cb6593d473de9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347765.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/539/b0b9b89c6c0f66ec21d3ba9a8f35d539/b0b9b89c6c0f66ec21d3ba9a8f35d539.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347765.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/539/b0b9b89c6c0f66ec21d3ba9a8f35d539/b0b9b89c6c0f66ec21d3ba9a8f35d539.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347767.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/a4b/b05a488a705698f2be5b6c65b933fa4b/b05a488a705698f2be5b6c65b933fa4b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347767.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/a4b/b05a488a705698f2be5b6c65b933fa4b/b05a488a705698f2be5b6c65b933fa4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347769.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/515/deab18cf0e11f99039f4b827dabe5515/deab18cf0e11f99039f4b827dabe5515.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347769.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/515/deab18cf0e11f99039f4b827dabe5515/deab18cf0e11f99039f4b827dabe5515.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347771.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/84f/2d2b9e30412dcdef3b53e4b90b76884f/2d2b9e30412dcdef3b53e4b90b76884f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347771.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/84f/2d2b9e30412dcdef3b53e4b90b76884f/2d2b9e30412dcdef3b53e4b90b76884f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347773.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f3c/b3c55d6c835ed1d8cf9229f4de748f3c/b3c55d6c835ed1d8cf9229f4de748f3c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347773.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f3c/b3c55d6c835ed1d8cf9229f4de748f3c/b3c55d6c835ed1d8cf9229f4de748f3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347775.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ed0/3da0b906614f1d69f381719f74d90ed0/3da0b906614f1d69f381719f74d90ed0.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_container_merge_other_ranges ... ok sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347775.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/ed0/3da0b906614f1d69f381719f74d90ed0/3da0b906614f1d69f381719f74d90ed0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347777.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/59d/367f0316445c5e3693983d52e6efb59d/367f0316445c5e3693983d52e6efb59d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347777.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/59d/367f0316445c5e3693983d52e6efb59d/367f0316445c5e3693983d52e6efb59d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347779.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2db/3ad8759f8926220e8a6c4432900b32db/3ad8759f8926220e8a6c4432900b32db.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347779.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/2db/3ad8759f8926220e8a6c4432900b32db/3ad8759f8926220e8a6c4432900b32db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347781.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/cc8/7a8d5f11809425d249947b168de76cc8/7a8d5f11809425d249947b168de76cc8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347781.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/cc8/7a8d5f11809425d249947b168de76cc8/7a8d5f11809425d249947b168de76cc8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347783.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/948/6b8eae1c3a767c32b320af8adadd0948/6b8eae1c3a767c32b320af8adadd0948.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347783.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/948/6b8eae1c3a767c32b320af8adadd0948/6b8eae1c3a767c32b320af8adadd0948.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347785.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/707/3168133b50d23ba8db24474e6ed3d707/3168133b50d23ba8db24474e6ed3d707.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347785.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/707/3168133b50d23ba8db24474e6ed3d707/3168133b50d23ba8db24474e6ed3d707.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347787.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/444/72ac5a4e1cbcc5493501c9f905bdc444/72ac5a4e1cbcc5493501c9f905bdc444.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347787.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/444/72ac5a4e1cbcc5493501c9f905bdc444/72ac5a4e1cbcc5493501c9f905bdc444.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347789.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f40/876f77696ad83bfab45dbd5517ca5f40/876f77696ad83bfab45dbd5517ca5f40.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347789.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/f40/876f77696ad83bfab45dbd5517ca5f40/876f77696ad83bfab45dbd5517ca5f40.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347791.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9cb/8ab7640f6b9912fc871e8e57866969cb/8ab7640f6b9912fc871e8e57866969cb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347791.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/9cb/8ab7640f6b9912fc871e8e57866969cb/8ab7640f6b9912fc871e8e57866969cb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347793.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/051/bdae4541400ecdc86d45d8799e555051/bdae4541400ecdc86d45d8799e555051.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347793.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/051/bdae4541400ecdc86d45d8799e555051/bdae4541400ecdc86d45d8799e555051.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347795.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/efe/7043d202c2894ad9d1088fc5363b1efe/7043d202c2894ad9d1088fc5363b1efe.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347795.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/efe/7043d202c2894ad9d1088fc5363b1efe/7043d202c2894ad9d1088fc5363b1efe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347797.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/100/ffae73052fb179814a521ca2e4ece100/ffae73052fb179814a521ca2e4ece100.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347797.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/100/ffae73052fb179814a521ca2e4ece100/ffae73052fb179814a521ca2e4ece100.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347799.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e0b/72afd535a23f5501b187d58f2ce3ee0b/72afd535a23f5501b187d58f2ce3ee0b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347799.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e0b/72afd535a23f5501b187d58f2ce3ee0b/72afd535a23f5501b187d58f2ce3ee0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347801.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e9f/a073666f141375aafe32fc85c709ee9f/a073666f141375aafe32fc85c709ee9f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347801.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/e9f/a073666f141375aafe32fc85c709ee9f/a073666f141375aafe32fc85c709ee9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347803.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/69d/04a857082e13d5650487c5d4fb5f269d/04a857082e13d5650487c5d4fb5f269d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347803.00000, db: /tmp/tmpsgzk0pbx/sda/containers/0/69d/04a857082e13d5650487c5d4fb5f269d/04a857082e13d5650487c5d4fb5f269d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347560.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a70/793dd519b079c475c0c10570e6bf7a70/793dd519b079c475c0c10570e6bf7a70.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347565.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/849/274c480fab9b1f18b78b4570fa04a849/274c480fab9b1f18b78b4570fa04a849.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347570.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/98c/44632839a234c5c8889f04b7cfbe098c/44632839a234c5c8889f04b7cfbe098c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347575.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9c0/3c9103365b9f07c97ada0f0bcdd9b9c0/3c9103365b9f07c97ada0f0bcdd9b9c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347580.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f56/9c89e88ed1382d8157763a98ac59bf56/9c89e88ed1382d8157763a98ac59bf56.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347585.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/161/739fcb3c30c343a0dd6823bb2df35161/739fcb3c30c343a0dd6823bb2df35161.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347590.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/658/9b50e5c8b4812ff4627ea04754907658/9b50e5c8b4812ff4627ea04754907658.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347595.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/927/cfb45dd245e2aafe51f405678f957927/cfb45dd245e2aafe51f405678f957927.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347600.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/02c/c963245de37fe34ad4083a0687d2702c/c963245de37fe34ad4083a0687d2702c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347605.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/34a/99fed0ed4ce2b49a1153aa26d07fb34a/99fed0ed4ce2b49a1153aa26d07fb34a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347610.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/363/a7f6204dd0f22ac8597f755aeed90363/a7f6204dd0f22ac8597f755aeed90363.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347615.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/eee/615f85f1d7a071cefbd4654669b7beee/615f85f1d7a071cefbd4654669b7beee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347620.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/058/5cdb99d2d569d9da3bc419ff1dfc6058/5cdb99d2d569d9da3bc419ff1dfc6058.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347625.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/77f/aa1d6813e07cc32fc435d0608019277f/aa1d6813e07cc32fc435d0608019277f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347630.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/006/adc08935a25d855056391774cf81a006/adc08935a25d855056391774cf81a006.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347635.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/bc2/846d5e69d8b9bf77558085c3f5cc0bc2/846d5e69d8b9bf77558085c3f5cc0bc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347650.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/939/a65f6f7661617c4312bcbbcf34c63939/a65f6f7661617c4312bcbbcf34c63939.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347655.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b50/6af70ffeca5372ca4cd9d87b84614b50/6af70ffeca5372ca4cd9d87b84614b50.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347665.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/2dc/56231bb56535d7ae0dd86961366c52dc/56231bb56535d7ae0dd86961366c52dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347670.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/35b/fcd26c3abb10b783d89d36fdf44e135b/fcd26c3abb10b783d89d36fdf44e135b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347680.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/8bb/aae77f028aa9a9780ee8fc5297c6c8bb/aae77f028aa9a9780ee8fc5297c6c8bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347685.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/70f/2e05998d4d14388abcf50dba1aaf370f/2e05998d4d14388abcf50dba1aaf370f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347690.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/354/f379bcf67d17b83e8612a2b17eb0c354/f379bcf67d17b83e8612a2b17eb0c354.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347695.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/7fc/960d0403382d4f530771e01fb325c7fc/960d0403382d4f530771e01fb325c7fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347710.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b73/a440a74f1454af0ddb6aa057909f0b73/a440a74f1454af0ddb6aa057909f0b73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347715.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/aec/d611b19ffc806aa25c87a098e4465aec/d611b19ffc806aa25c87a098e4465aec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347725.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9f3/4d5c42343ad62e7bfb8d38e83e6349f3/4d5c42343ad62e7bfb8d38e83e6349f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347730.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c84/a1c5cdafd5f74fb95eb78d4e96445c84/a1c5cdafd5f74fb95eb78d4e96445c84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347735.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b6f/66be2eb4b3cf41d35caa1c96a69b1b6f/66be2eb4b3cf41d35caa1c96a69b1b6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347740.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/45c/9c8ec32ccf58393b11dc3cd4e0e7e45c/9c8ec32ccf58393b11dc3cd4e0e7e45c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347745.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/dda/27a9ca1c7ae225c426acfa89d211fdda/27a9ca1c7ae225c426acfa89d211fdda.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347750.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c40/3533c4527a93b20163c3e3ad474d2c40/3533c4527a93b20163c3e3ad474d2c40.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347755.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f97/a845b58185a0bf9de204f95f5d076f97/a845b58185a0bf9de204f95f5d076f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347770.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/96c/54f948a1165aaf139573e9079fa8b96c/54f948a1165aaf139573e9079fa8b96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347775.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/57e/c71a3503c848b63d82c2502f0583457e/c71a3503c848b63d82c2502f0583457e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347785.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/5aa/a948b8c8c554e6f77656406f38f145aa/a948b8c8c554e6f77656406f38f145aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347790.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/965/c55e12e6ea051eec2ce7e93485fa8965/c55e12e6ea051eec2ce7e93485fa8965.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347795.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d9b/050560945ef5bbab351b2b51d557ad9b/050560945ef5bbab351b2b51d557ad9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347800.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d60/29c4e4c38b2a9b66d6dc06a3a31c5d60/29c4e4c38b2a9b66d6dc06a3a31c5d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347805.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/957/bd01253df32bee6088b3c10b2c231957/bd01253df32bee6088b3c10b2c231957.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347810.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e2f/c575f6bb053862175e0b67f38ba58e2f/c575f6bb053862175e0b67f38ba58e2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347815.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/1d5/f0e6aeb96f0c36c9151e7628306601d5/f0e6aeb96f0c36c9151e7628306601d5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347820.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/5fd/006fcabd0221977584acf3840c2005fd/006fcabd0221977584acf3840c2005fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347825.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/8de/73f855b7cb3477d481a0c22e6505a8de/73f855b7cb3477d481a0c22e6505a8de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347830.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c6a/e0e3afd71788c838aad06292c0db6c6a/e0e3afd71788c838aad06292c0db6c6a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347835.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/992/011065492122d68ab4286cefef06f992/011065492122d68ab4286cefef06f992.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347840.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ba7/1be2ce69ea2bd1dc593afb7bb33fdba7/1be2ce69ea2bd1dc593afb7bb33fdba7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347845.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/4c7/bee591a6cdc9e5f3a62c798dfe91e4c7/bee591a6cdc9e5f3a62c798dfe91e4c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347850.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b83/5aca788f2fbfa8c9c9f2e4d7bf817b83/5aca788f2fbfa8c9c9f2e4d7bf817b83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347855.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d1a/01313479a199746172fd553e96c37d1a/01313479a199746172fd553e96c37d1a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347860.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/1f1/525911717c691d9742a0d63954d751f1/525911717c691d9742a0d63954d751f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347865.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/083/0c3c57f47368c3acbf8f2c59c55bd083/0c3c57f47368c3acbf8f2c59c55bd083.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347870.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/691/a39a45f99fb6ad1281550b1c36ed7691/a39a45f99fb6ad1281550b1c36ed7691.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347875.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d97/381cbda8f21b520fd6fa8af128774d97/381cbda8f21b520fd6fa8af128774d97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347880.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b3e/670079b1c2086bc65d0a8c9c0e61fb3e/670079b1c2086bc65d0a8c9c0e61fb3e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347885.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ccb/4eda514f33b17a9f427637c5b7392ccb/4eda514f33b17a9f427637c5b7392ccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347890.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/eae/d24ceadcb229e1c8ccaa073a62934eae/d24ceadcb229e1c8ccaa073a62934eae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347895.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/8a7/474dd4fa2d6f8c8a5c8595d1be2a18a7/474dd4fa2d6f8c8a5c8595d1be2a18a7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347900.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/233/afd45040e81a45be47f71516659dc233/afd45040e81a45be47f71516659dc233.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347905.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a27/0f882f1c2c943e0a369d3e5eb5cb3a27/0f882f1c2c943e0a369d3e5eb5cb3a27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347910.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/192/7ba0c40a17511607b306399c5e5f0192/7ba0c40a17511607b306399c5e5f0192.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347915.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d1d/a7a4505712d9f26b6a929a3e753f4d1d/a7a4505712d9f26b6a929a3e753f4d1d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347920.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/3fe/580e9c5d3741cad4261f2df0237143fe/580e9c5d3741cad4261f2df0237143fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347925.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/3c8/dadd317843955d4a8078835095dc53c8/dadd317843955d4a8078835095dc53c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347930.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/250/77556b341f9024a065dc6d7c4237f250/77556b341f9024a065dc6d7c4237f250.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347935.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e31/37dea06684b0588036a2d1fb21310e31/37dea06684b0588036a2d1fb21310e31.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347940.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/eef/3854af72dd9d24247430f1e7621dbeef/3854af72dd9d24247430f1e7621dbeef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347945.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/044/a33b6292a60d0f796e2cb49059b9e044/a33b6292a60d0f796e2cb49059b9e044.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347950.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/29a/477acbcddd059e7fad03b1cc3cf8429a/477acbcddd059e7fad03b1cc3cf8429a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347955.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/149/0606b83a54a6eff2c6bd118aba5fe149/0606b83a54a6eff2c6bd118aba5fe149.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347960.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/96c/1b9ff9b0c59e9547a4750f0d431bb96c/1b9ff9b0c59e9547a4750f0d431bb96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347965.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/cb7/fd0e749512ecb60cee0051ccf8d3fcb7/fd0e749512ecb60cee0051ccf8d3fcb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347970.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/256/a5519f5f0d0cb84115cc3b6c8a191256/a5519f5f0d0cb84115cc3b6c8a191256.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347975.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d02/0d6ea296208ee085c870b76a8f3cbd02/0d6ea296208ee085c870b76a8f3cbd02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347980.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f97/2a80ffebc7c69a504f975770c6693f97/2a80ffebc7c69a504f975770c6693f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347985.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f10/1b00a5c88b65373d5e637161327b2f10/1b00a5c88b65373d5e637161327b2f10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347990.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/31b/2376b644dbfe9ba7fd100ae89204c31b/2376b644dbfe9ba7fd100ae89204c31b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347995.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/db1/3d73127ef8d2da0bbc046caeecae3db1/3d73127ef8d2da0bbc046caeecae3db1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348000.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ba5/b598bcb80f3103b28c49f79766e04ba5/b598bcb80f3103b28c49f79766e04ba5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348005.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/912/595904597e1790fbdfd9e32790dc6912/595904597e1790fbdfd9e32790dc6912.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348010.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/34b/0d8f3b88d3cfe5cbd2aeaad2b5aa834b/0d8f3b88d3cfe5cbd2aeaad2b5aa834b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348015.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/dd1/8aca460283f0bb4f76698c0e35ba3dd1/8aca460283f0bb4f76698c0e35ba3dd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348020.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9ce/a425f11f2290689b2f5878d7cfe5e9ce/a425f11f2290689b2f5878d7cfe5e9ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348025.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/5ee/d2b979a99eec3622d4c66df3bc6ed5ee/d2b979a99eec3622d4c66df3bc6ed5ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348030.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a2a/b48ba937fa34b94b3829e71beb61da2a/b48ba937fa34b94b3829e71beb61da2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348035.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fa1/02e3cf74f37ce0e83b0aa44e8acc5fa1/02e3cf74f37ce0e83b0aa44e8acc5fa1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348040.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f97/2a2166a062677ec01259d644dab03f97/2a2166a062677ec01259d644dab03f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348045.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/49a/2f0b29f01bda657a965865bafd44249a/2f0b29f01bda657a965865bafd44249a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348050.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ea2/94fbd703423313426921aff051b5fea2/94fbd703423313426921aff051b5fea2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348055.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ea7/fc171f72480fd73453e103c463a8eea7/fc171f72480fd73453e103c463a8eea7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348060.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b7d/bc7cfaf8392671e45668c28e4202fb7d/bc7cfaf8392671e45668c28e4202fb7d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348065.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/710/7fc15467f16b6c761e8b49d881f2f710/7fc15467f16b6c761e8b49d881f2f710.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348070.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9f1/1722d9f4636e014c07aabaa20b82a9f1/1722d9f4636e014c07aabaa20b82a9f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348075.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/11a/48d0037f06bce7151eea6da51d27b11a/48d0037f06bce7151eea6da51d27b11a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348080.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/1ca/32658bb760b2094363eccf14bb1ef1ca/32658bb760b2094363eccf14bb1ef1ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348085.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/958/9c71fc26ef1336fa8d1b46f2660a5958/9c71fc26ef1336fa8d1b46f2660a5958.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348090.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/1ad/cf5efb62101cfaad6c008a01af77c1ad/cf5efb62101cfaad6c008a01af77c1ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348095.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/11e/6e9363bde772f4e5af2c91f10691b11e/6e9363bde772f4e5af2c91f10691b11e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348100.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e50/16c14f0b3d21999f1e1378d49943ce50/16c14f0b3d21999f1e1378d49943ce50.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348105.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/2a9/52c958e2b95fda3f78ea5771c335b2a9/52c958e2b95fda3f78ea5771c335b2a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348110.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/bcd/7d466eac11c27e27b2321ad4eb291bcd/7d466eac11c27e27b2321ad4eb291bcd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348115.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/3bb/61579d0ddb3167076c2a5ce91a6df3bb/61579d0ddb3167076c2a5ce91a6df3bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348120.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/984/ce3bf1de18e53a815a6efebe6da41984/ce3bf1de18e53a815a6efebe6da41984.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348125.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/dc2/de9b7a20006c5694d6ca75f65a52adc2/de9b7a20006c5694d6ca75f65a52adc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348130.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/cba/1fede09e63384a435cfd9ef3ea3c9cba/1fede09e63384a435cfd9ef3ea3c9cba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348135.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fea/84f9202a504c9a0b81f505bb00fb0fea/84f9202a504c9a0b81f505bb00fb0fea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348140.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9bd/af9286ec9c837fdebdc64b369f38d9bd/af9286ec9c837fdebdc64b369f38d9bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348145.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/6ba/d138ea09bfed69872e1b973dbcf566ba/d138ea09bfed69872e1b973dbcf566ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348150.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9e2/42c08c17d37df6696e31d2aba92f19e2/42c08c17d37df6696e31d2aba92f19e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348155.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f28/526a19766b268d93e739292f1fa2ff28/526a19766b268d93e739292f1fa2ff28.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348160.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/763/8da02c52ce3916899e377db52f0b7763/8da02c52ce3916899e377db52f0b7763.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348165.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/91f/1307f57b4fc06a773c92e26bfeedf91f/1307f57b4fc06a773c92e26bfeedf91f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348170.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/77e/adad2e594313509933ecb58c142bd77e/adad2e594313509933ecb58c142bd77e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348175.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/734/6bd8627252ffca7ca4080c3996ca5734/6bd8627252ffca7ca4080c3996ca5734.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348180.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/4ce/40b92ace61f6a035a00ad75c8505a4ce/40b92ace61f6a035a00ad75c8505a4ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348185.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/eed/6d612780c9230166851a6ed7ef51feed/6d612780c9230166851a6ed7ef51feed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348190.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/4fe/0a1382ba0e0bc02835dbed3b68a494fe/0a1382ba0e0bc02835dbed3b68a494fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348195.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/381/3792c9b2a749e4e955adbe5e20cc6381/3792c9b2a749e4e955adbe5e20cc6381.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348200.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/223/20f945c49eeb9156a91118f8f84b1223/20f945c49eeb9156a91118f8f84b1223.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348205.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/690/e71a1d3a6f49b2368310ef49c1803690/e71a1d3a6f49b2368310ef49c1803690.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348210.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c65/1f115b7058374289c9ae905f7b5f7c65/1f115b7058374289c9ae905f7b5f7c65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348215.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/120/20a27009a9346a8915c6202603f62120/20a27009a9346a8915c6202603f62120.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348220.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/675/b26440bb50e7d0e3245478962fefe675/b26440bb50e7d0e3245478962fefe675.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348225.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/384/0934c95d5a78fdc42930ee6e3d677384/0934c95d5a78fdc42930ee6e3d677384.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348230.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ccf/9dfbc58b7aafeb8800b972daaaa12ccf/9dfbc58b7aafeb8800b972daaaa12ccf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348235.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/37a/08f19c05663d8c08adea4ce5fa63e37a/08f19c05663d8c08adea4ce5fa63e37a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348240.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/3a8/db9e2f57ff2134686bcdea79528083a8/db9e2f57ff2134686bcdea79528083a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348245.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/8ed/e425beb7100568d77f593318dc1948ed/e425beb7100568d77f593318dc1948ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348250.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/05c/c27264c23dfdfc32e126812b5434305c/c27264c23dfdfc32e126812b5434305c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348255.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/002/434a0e0967310f955ad2727c79f3d002/434a0e0967310f955ad2727c79f3d002.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348260.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/966/be887be5d893ab70b836c94d770cb966/be887be5d893ab70b836c94d770cb966.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348265.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/efe/34cbb3484023489bbdb7c6e1bbb01efe/34cbb3484023489bbdb7c6e1bbb01efe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348270.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/5fa/7a658eee0c95553dfb98ba5d88c0d5fa/7a658eee0c95553dfb98ba5d88c0d5fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348275.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/482/dc3fc77d2644dea12dd9a51779489482/dc3fc77d2644dea12dd9a51779489482.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348280.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e00/271e1216e64e9f867453f84c350c1e00/271e1216e64e9f867453f84c350c1e00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348285.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/402/fe9e4d9268a518f70f30191ec69a4402/fe9e4d9268a518f70f30191ec69a4402.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348290.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fff/a2f7950eb02f2f5efd37098fe2da0fff/a2f7950eb02f2f5efd37098fe2da0fff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348295.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fbb/5ecea0974c7dbc77f7113841db3bcfbb/5ecea0974c7dbc77f7113841db3bcfbb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348300.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d3c/b766b321ae4bc28ebc5d917c38164d3c/b766b321ae4bc28ebc5d917c38164d3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348305.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e51/9df200507457cb65d18353b537ba6e51/9df200507457cb65d18353b537ba6e51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348310.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f4f/42e3c1dbd83f277538ef815a54febf4f/42e3c1dbd83f277538ef815a54febf4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348315.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/209/9a643e7f79a0f1df5930b289a22c3209/9a643e7f79a0f1df5930b289a22c3209.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348320.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/997/95b6ce24d3c65f9a1cd529ee6c503997/95b6ce24d3c65f9a1cd529ee6c503997.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348325.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/486/228c259ca2244b437095ce27f8147486/228c259ca2244b437095ce27f8147486.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348330.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/6ba/3da3ac137dc47c729415bf31e6cca6ba/3da3ac137dc47c729415bf31e6cca6ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348335.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d59/d7d6c6e8ff3754b4ab79a6d4decbdd59/d7d6c6e8ff3754b4ab79a6d4decbdd59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348340.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/2a7/e5e3e64c532bce2ace3a91d1713482a7/e5e3e64c532bce2ace3a91d1713482a7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348345.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b05/a77a8db3a798c72f319b47d89f9adb05/a77a8db3a798c72f319b47d89f9adb05.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348350.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c31/68c0c991d4380831728d65366a820c31/68c0c991d4380831728d65366a820c31.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348355.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/19f/9456567126af49e6910011715cbcf19f/9456567126af49e6910011715cbcf19f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348360.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/bd3/3ca31264c6219b18b0e1ed365913dbd3/3ca31264c6219b18b0e1ed365913dbd3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348365.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/03b/fbb81aef876b36d1a1d3c10c882c903b/fbb81aef876b36d1a1d3c10c882c903b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348370.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/16b/56137a1cb3bbcc190a0800a1afdba16b/56137a1cb3bbcc190a0800a1afdba16b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348375.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/3bc/95262600647a76561dd6c1a8eb3163bc/95262600647a76561dd6c1a8eb3163bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348380.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b75/610c64ca24fb70a45dac042f27ce5b75/610c64ca24fb70a45dac042f27ce5b75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348385.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/6ee/48e19fcc5565af486e287105c4eb46ee/48e19fcc5565af486e287105c4eb46ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348390.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/7a6/e2fe4734903a5e9a75f9712d3c5d97a6/e2fe4734903a5e9a75f9712d3c5d97a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348395.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c19/4876ee85117ada8520fadf6e7a196c19/4876ee85117ada8520fadf6e7a196c19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348400.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/91f/5416c8a52d221b1fd896399024da991f/5416c8a52d221b1fd896399024da991f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348405.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ee2/dc935c37a5f162ff6ea559e989b31ee2/dc935c37a5f162ff6ea559e989b31ee2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348410.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/52c/c6f129c5a152df97360693457a24952c/c6f129c5a152df97360693457a24952c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348415.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f94/43e1e952c853b20ebdf0c319cbb2af94/43e1e952c853b20ebdf0c319cbb2af94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348420.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/0ba/248fe19803695fd9d1de01a085bea0ba/248fe19803695fd9d1de01a085bea0ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348425.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/604/be776072afcc8f2b46bed52167d52604/be776072afcc8f2b46bed52167d52604.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348430.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/364/5ab4394ac26ef6903b236be1ad563364/5ab4394ac26ef6903b236be1ad563364.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348435.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/18d/af6334a2d7504ed955c3d5379573b18d/af6334a2d7504ed955c3d5379573b18d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348440.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e62/ab6405b9791b6c7d9bd3f613af9b3e62/ab6405b9791b6c7d9bd3f613af9b3e62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348445.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fe2/ccfbca61b13f2cade63e000c8567dfe2/ccfbca61b13f2cade63e000c8567dfe2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348450.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/336/5a3ee7b9f73a0b696e58f731e5cb7336/5a3ee7b9f73a0b696e58f731e5cb7336.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348455.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/7bd/d16950d0bcbc09a0e7047cff092d37bd/d16950d0bcbc09a0e7047cff092d37bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348460.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a02/38627e463622b2ab85682029e5408a02/38627e463622b2ab85682029e5408a02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348465.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c57/0b022911f76049e94b3b6e2a916a7c57/0b022911f76049e94b3b6e2a916a7c57.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348470.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/241/51b52d05863c42542eddbbf9e4da1241/51b52d05863c42542eddbbf9e4da1241.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348475.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/853/90d96cb85f4ac123cc95f71575f4e853/90d96cb85f4ac123cc95f71575f4e853.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348480.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f60/554b638b8590cbc9b07a6fef081b8f60/554b638b8590cbc9b07a6fef081b8f60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348485.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/2c2/fc884f55dcccf60b50e84656095972c2/fc884f55dcccf60b50e84656095972c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348490.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fb2/b1e52d98f196b13e69bfea07afbc7fb2/b1e52d98f196b13e69bfea07afbc7fb2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348495.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/536/79d9bb12faedb7010d613d0555f6c536/79d9bb12faedb7010d613d0555f6c536.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348500.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fe3/fdd891acfa799a04886bcbdd022a4fe3/fdd891acfa799a04886bcbdd022a4fe3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348505.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e4b/16448043b77b6de15c09fdb0fd0a7e4b/16448043b77b6de15c09fdb0fd0a7e4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348510.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f03/fbc3c2071e41733dadda0c1f7cffff03/fbc3c2071e41733dadda0c1f7cffff03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348515.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/4ea/fcc6d4872c7c48b5d7350ad6b5d114ea/fcc6d4872c7c48b5d7350ad6b5d114ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348520.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/42c/ec37b602416e36604fd77b410216542c/ec37b602416e36604fd77b410216542c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348525.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/94e/4fff2656d95c093683b65e3db482894e/4fff2656d95c093683b65e3db482894e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348530.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b0a/f37fdf17d2606364f7dc5deb05094b0a/f37fdf17d2606364f7dc5deb05094b0a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348535.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/903/e050ea15fc8416cd4ae74ab6831e4903/e050ea15fc8416cd4ae74ab6831e4903.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348540.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/19e/5bf73b1e190f535adcf051e3a422419e/5bf73b1e190f535adcf051e3a422419e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348545.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/4b8/19c07aa0510b1835822c3c86e14994b8/19c07aa0510b1835822c3c86e14994b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348550.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/6f6/7ab0fe2433331ad862ebc0a9123416f6/7ab0fe2433331ad862ebc0a9123416f6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348555.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d01/10863ab32a7c96b86d7c05fb15e81d01/10863ab32a7c96b86d7c05fb15e81d01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348560.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b16/f7d9bb22f1f2ffd0bf3a30158d064b16/f7d9bb22f1f2ffd0bf3a30158d064b16.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348565.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b45/a346476cc3c22dd655ca0115951a3b45/a346476cc3c22dd655ca0115951a3b45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348570.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/154/c46139c5ab113abe6618d6f349bdb154/c46139c5ab113abe6618d6f349bdb154.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348575.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/eb1/6643be26330dff914d200e3c2c381eb1/6643be26330dff914d200e3c2c381eb1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348580.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/5d9/ca048f445b3fa820476ccbb8e2ae15d9/ca048f445b3fa820476ccbb8e2ae15d9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348585.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/441/93022679b6a84ead2bb1f1d57e437441/93022679b6a84ead2bb1f1d57e437441.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348590.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/681/78cec865b6e7cc2409a8070a5a16d681/78cec865b6e7cc2409a8070a5a16d681.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348595.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/34d/bbe5240d11e9ea43b14ae2ce51ce034d/bbe5240d11e9ea43b14ae2ce51ce034d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348600.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c04/562b7721b50a8f2fb91633b6855bec04/562b7721b50a8f2fb91633b6855bec04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348605.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f4d/c2e12ed78dfc9379613108d21af18f4d/c2e12ed78dfc9379613108d21af18f4d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348610.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/af4/d213edd2fe5bb9ad715154b1a6e6caf4/d213edd2fe5bb9ad715154b1a6e6caf4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348615.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/74b/d0809f071782ba8f52463789f8ac874b/d0809f071782ba8f52463789f8ac874b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348620.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/324/50cf98cb92f1f3ddab2a5927afbd8324/50cf98cb92f1f3ddab2a5927afbd8324.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348625.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d9f/2001df02a9203b2e0dc3003ea5b22d9f/2001df02a9203b2e0dc3003ea5b22d9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348630.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9fa/656402c044a4950009b5f15a521119fa/656402c044a4950009b5f15a521119fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348635.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/fc6/70ed22e7ba784091bf9f44b57f4f6fc6/70ed22e7ba784091bf9f44b57f4f6fc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348640.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/3d3/92616908a5063620cff78c68c36313d3/92616908a5063620cff78c68c36313d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348645.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d39/032e5fc18015ae034e3525a7958a4d39/032e5fc18015ae034e3525a7958a4d39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348650.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b80/7e35283a9e978b1e874b7f9ec38a0b80/7e35283a9e978b1e874b7f9ec38a0b80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348655.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/074/1dcc897f4c98b73bb87ff2871e6fb074/1dcc897f4c98b73bb87ff2871e6fb074.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348660.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b99/fccb2ed5fe6bc269310cae9d1e37bb99/fccb2ed5fe6bc269310cae9d1e37bb99.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348665.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/622/3ae169413303727e56a377bb581a7622/3ae169413303727e56a377bb581a7622.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348670.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/57a/95e9515ebbbe808902a56b2349c7f57a/95e9515ebbbe808902a56b2349c7f57a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348675.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/119/23be2c8393330d0797eb21aeaaf09119/23be2c8393330d0797eb21aeaaf09119.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348680.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d1b/ee1368276b79cd6ff697b372ec013d1b/ee1368276b79cd6ff697b372ec013d1b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348685.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/acd/e750d7f7a0098c4577bc00b7e3fdaacd/e750d7f7a0098c4577bc00b7e3fdaacd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348690.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/ddb/25e5fe9c34a81b6883376cdbe6ed6ddb/25e5fe9c34a81b6883376cdbe6ed6ddb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348695.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/158/5296f4eb06cf79caf8accf442bb1f158/5296f4eb06cf79caf8accf442bb1f158.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348700.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/488/f74e83fac25a36ec34d5fd0e2fe91488/f74e83fac25a36ec34d5fd0e2fe91488.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348705.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f55/dfbd9847ef03a94cfe7df8ae5134af55/dfbd9847ef03a94cfe7df8ae5134af55.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348710.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d59/59e3df290e4775005b19f3c4af837d59/59e3df290e4775005b19f3c4af837d59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348715.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/c81/42f5ac27426d2a559a3b26fbaa553c81/42f5ac27426d2a559a3b26fbaa553c81.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348720.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e0c/fba3ff17603f4bd4dbda7585be18ae0c/fba3ff17603f4bd4dbda7585be18ae0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348725.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/220/0795f619d64804780637cb72d4a5b220/0795f619d64804780637cb72d4a5b220.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348730.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b5f/1f5852a8e4bb91dfdd4108cae7b33b5f/1f5852a8e4bb91dfdd4108cae7b33b5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348735.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/7a5/166c1ddb0ab9c1173f7753521abdf7a5/166c1ddb0ab9c1173f7753521abdf7a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348740.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/7d3/7532c8d7b25914e8518aa19f6c6d77d3/7532c8d7b25914e8518aa19f6c6d77d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348745.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/cd0/a06d5d00d689624133971d5bd0ca3cd0/a06d5d00d689624133971d5bd0ca3cd0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348750.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/932/571ab1d6d1986446a401595ba57fb932/571ab1d6d1986446a401595ba57fb932.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348755.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/bfd/4eea9c5516594734065b19ade255fbfd/4eea9c5516594734065b19ade255fbfd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348760.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/6ad/caa9dac73fe06a0279cf320e3426e6ad/caa9dac73fe06a0279cf320e3426e6ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348765.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/627/a2f4d191a9443f9cea74f725a7584627/a2f4d191a9443f9cea74f725a7584627.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348770.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/26c/74847d44fd47693f0e5eb7417769f26c/74847d44fd47693f0e5eb7417769f26c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348775.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d29/4568af56df0fe57cbf21debcee37dd29/4568af56df0fe57cbf21debcee37dd29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348780.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/745/099a33ac9118903d82aa58a508920745/099a33ac9118903d82aa58a508920745.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348785.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/86d/9e4e01ca49210ebf6780d40d095eb86d/9e4e01ca49210ebf6780d40d095eb86d.db test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_old_style_shard_root_ranges_not_merged_not_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container test.unit.container.test_sharder.TestSharder.test_audit_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch test.unit.container.test_sharder.TestSharder.test_audit_root_container_reset_epoch ... ok test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping test.unit.container.test_sharder.TestSharder.test_audit_root_container_with_parent_child_overlapping ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_active_root_range_merged_while_shrinking ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348790.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/958/6833a908e0a1f79f9781006e97a8d958/6833a908e0a1f79f9781006e97a8d958.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348795.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a65/b0248a7653269a0befbc0d843b7e5a65/b0248a7653269a0befbc0d843b7e5a65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348800.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a0c/7acef9801031d30a340fa514e8a32a0c/7acef9801031d30a340fa514e8a32a0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348805.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/d4e/277a9a7b46c13e2cb26599030ac2dd4e/277a9a7b46c13e2cb26599030ac2dd4e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348810.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/9c7/78990da72e23faec3a9095a2791399c7/78990da72e23faec3a9095a2791399c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348815.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b37/4b9e4bc15a0fa25921d2d45615241b37/4b9e4bc15a0fa25921d2d45615241b37.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348820.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/a54/8685c69fabba176be7b050a7b5a89a54/8685c69fabba176be7b050a7b5a89a54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348825.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/f77/f0519496f7719dfae28928b189bcff77/f0519496f7719dfae28928b189bcff77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348830.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/b02/79ba8abecdd686ff63ac361bcabc2b02/79ba8abecdd686ff63ac361bcabc2b02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348835.00000-0, db: /tmp/tmpdbhlxa37/sda/containers/0/e82/d9c2908eb6aa86f1e2d96943445d9e82/d9c2908eb6aa86f1e2d96943445d9e82.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'found': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'created': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'sharding': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: missing range(s): -a j-k z-, overlapping ranges in state 'active': k-t s-y, y-z y-z, path: a/c, db: /tmp/tmpg4aztw96/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Audit failed for root: own_shard_range reset to None should be 1717347568.00000, path: a/c, db: /tmp/tmploog_kko/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347568.00000.db sharder-test WARNING: Audit failed for root: overlapping ranges in state 'cleaved': a-f c-f, a-c a-f, path: a/c, db: /tmp/tmpathirmwa/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347569.00000-0, db: /tmp/tmpag9tkrax/sda/containers/0/c6d/bc3eae34cfaa31fe0fa220cbf110fc6d/bc3eae34cfaa31fe0fa220cbf110fc6d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347569.00000-0, db: /tmp/tmpag9tkrax/sda/containers/0/c6d/bc3eae34cfaa31fe0fa220cbf110fc6d/bc3eae34cfaa31fe0fa220cbf110fc6d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347572.00000-0, db: /tmp/tmpag9tkrax/sda/containers/0/8bf/9161eacaef5cbf66ff805619e7a6e8bf/9161eacaef5cbf66ff805619e7a6e8bf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347572.00000-0, db: /tmp/tmpag9tkrax/sda/containers/0/8bf/9161eacaef5cbf66ff805619e7a6e8bf/9161eacaef5cbf66ff805619e7a6e8bf.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: bad_account/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: account not in shards namespace '.shards_', path: bad_account/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit failed for shard: missing own shard range (skipping), path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'j' to 'k' as of 1717347569.00000, (0, 0) as of 1717347569.00000, active as of 1717347569.00000> to ShardRange<'k' to 't' as of 1717347570.00000, (0, 0) as of 1717347570.00000, active as of 1717347570.00000>, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container test.unit.container.test_sharder.TestSharder.test_audit_shard_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_ancestors_not_merged_while_sharding ... ok sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347574.00000, (0, 0) as of 1717347574.00000, shrinking as of 1717347571.00000> to ShardRange<'k' to 't' as of 1717347574.00000, (0, 0) as of 1717347574.00000, sharding as of 1717347575.00000>, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmp1j95bykw/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-00197613f17d5500dc5954b27a3d619e-1717347573.00000-2, db: /tmp/tmp1c3x017l/sda/containers/0/8ca/9329334670c38a1b329bd86d8dd5f8ca/9329334670c38a1b329bd86d8dd5f8ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347569.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/83a/1154e53266afd7ceb3ec30c67d85483a/1154e53266afd7ceb3ec30c67d85483a.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347569.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/83a/1154e53266afd7ceb3ec30c67d85483a/1154e53266afd7ceb3ec30c67d85483a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347578.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/eb0/fd261033aa030bb13e79e55eb56e5eb0/fd261033aa030bb13e79e55eb56e5eb0.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347578.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/eb0/fd261033aa030bb13e79e55eb56e5eb0/fd261033aa030bb13e79e55eb56e5eb0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347587.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/224/d6786b240e3202b549e64b7a19749224/d6786b240e3202b549e64b7a19749224.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347587.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/224/d6786b240e3202b549e64b7a19749224/d6786b240e3202b549e64b7a19749224.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347596.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/0d3/4b0ddac907fb1cb6fbc62111259390d3/4b0ddac907fb1cb6fbc62111259390d3.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347596.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/0d3/4b0ddac907fb1cb6fbc62111259390d3/4b0ddac907fb1cb6fbc62111259390d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347605.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/f23/7111470f496cfb42c8148ae3b4514f23/7111470f496cfb42c8148ae3b4514f23.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347605.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/f23/7111470f496cfb42c8148ae3b4514f23/7111470f496cfb42c8148ae3b4514f23.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347614.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/79b/db4cbddc3a1f3ca378b14e7aca84f79b/db4cbddc3a1f3ca378b14e7aca84f79b.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347614.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/79b/db4cbddc3a1f3ca378b14e7aca84f79b/db4cbddc3a1f3ca378b14e7aca84f79b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347623.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/b45/5b7d4d72d90cf9b66a77e95c38262b45/5b7d4d72d90cf9b66a77e95c38262b45.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347623.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/b45/5b7d4d72d90cf9b66a77e95c38262b45/5b7d4d72d90cf9b66a77e95c38262b45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347632.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/29d/71e3f55d8b57b8064c40d38ac349529d/71e3f55d8b57b8064c40d38ac349529d.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347632.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/29d/71e3f55d8b57b8064c40d38ac349529d/71e3f55d8b57b8064c40d38ac349529d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347641.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/1a7/014b1446be51ec82f79903fa3b88d1a7/014b1446be51ec82f79903fa3b88d1a7.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347641.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/1a7/014b1446be51ec82f79903fa3b88d1a7/014b1446be51ec82f79903fa3b88d1a7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347641.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/1a7/014b1446be51ec82f79903fa3b88d1a7/014b1446be51ec82f79903fa3b88d1a7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347650.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/273/a3b82dcb9a563b0d2c043f91dadc0273/a3b82dcb9a563b0d2c043f91dadc0273.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347650.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/273/a3b82dcb9a563b0d2c043f91dadc0273/a3b82dcb9a563b0d2c043f91dadc0273.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347650.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/273/a3b82dcb9a563b0d2c043f91dadc0273/a3b82dcb9a563b0d2c043f91dadc0273.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347659.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/902/906a5a1996d81931022a99a0796b0902/906a5a1996d81931022a99a0796b0902.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347659.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/902/906a5a1996d81931022a99a0796b0902/906a5a1996d81931022a99a0796b0902.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347659.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/902/906a5a1996d81931022a99a0796b0902/906a5a1996d81931022a99a0796b0902.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347668.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/145/5d65104415697d702e9278a7359e0145/5d65104415697d702e9278a7359e0145.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347668.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/145/5d65104415697d702e9278a7359e0145/5d65104415697d702e9278a7359e0145.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347668.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/145/5d65104415697d702e9278a7359e0145/5d65104415697d702e9278a7359e0145.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_merged_while_sharding ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_container_children_not_merged_once_sharded ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347677.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/a1e/af931d331b5c3b743f2a1f7b68d2fa1e/af931d331b5c3b743f2a1f7b68d2fa1e.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347677.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/a1e/af931d331b5c3b743f2a1f7b68d2fa1e/af931d331b5c3b743f2a1f7b68d2fa1e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347677.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/a1e/af931d331b5c3b743f2a1f7b68d2fa1e/af931d331b5c3b743f2a1f7b68d2fa1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347686.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/953/8302f704febc87f92c4c391b7654a953/8302f704febc87f92c4c391b7654a953.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347686.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/953/8302f704febc87f92c4c391b7654a953/8302f704febc87f92c4c391b7654a953.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347686.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/953/8302f704febc87f92c4c391b7654a953/8302f704febc87f92c4c391b7654a953.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347695.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/5cf/c0b49a942413493b0cc1d219285ea5cf/c0b49a942413493b0cc1d219285ea5cf.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347695.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/5cf/c0b49a942413493b0cc1d219285ea5cf/c0b49a942413493b0cc1d219285ea5cf.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347695.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/5cf/c0b49a942413493b0cc1d219285ea5cf/c0b49a942413493b0cc1d219285ea5cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347704.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/ca7/85ead6731daa937d39f639062c57fca7/85ead6731daa937d39f639062c57fca7.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347704.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/ca7/85ead6731daa937d39f639062c57fca7/85ead6731daa937d39f639062c57fca7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347704.00000-2, db: /tmp/tmpjr7z7g_a/sda/containers/0/ca7/85ead6731daa937d39f639062c57fca7/85ead6731daa937d39f639062c57fca7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717347570.00000, (0, 0) as of 1717347570.00000, active as of 1717347570.00000> to ShardRange<'a' to 'b' as of 1717347570.00000, (0, 0) as of 1717347570.00000, sharding as of 1717347575.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58_1717347570.00000.db sharder-test DEBUG: Updating 2 children shard ranges from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58_1717347570.00000.db sharder-test DEBUG: Updating 0 other shard range(s) from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58_1717347570.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58_1717347570.00000.db sharder-test INFO: Updated own shard range from ShardRange<'a' to 'b' as of 1717347570.00000, (0, 0) as of 1717347570.00000, sharding as of 1717347576.00000> to ShardRange<'a' to 'b' as of 1717347570.00000, (0, 0) as of 1717347570.00000, sharded as of 1717347579.00000>, path: .shards_a/root-63a9f0ea7bb98050796b649e85481845-1717347570.00000-2, db: /tmp/tmp7vo8oa16/sda/containers/0/f58/d4f4a3e3a8e0a724540ed8f1fd64af58/d4f4a3e3a8e0a724540ed8f1fd64af58_1717347570.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347570.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/66f/3ea1508833ca34888ee59667a769c66f/3ea1508833ca34888ee59667a769c66f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347572.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ff4/713561672bb080d260f330fbb602dff4/713561672bb080d260f330fbb602dff4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347571.00000, (0, 0) as of 1717347571.00000, found as of 1717347571.00000> to ShardRange<'k' to 't' as of 1717347572.00000, (0, 0) as of 1717347572.00000, created as of 1717347572.00000>, path: .shards_a/shard_c_1717347572.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ff4/713561672bb080d260f330fbb602dff4/713561672bb080d260f330fbb602dff4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347574.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/469/07dd0e0d3b34ab0626b9e9c274c4b469/07dd0e0d3b34ab0626b9e9c274c4b469.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347573.00000, (0, 0) as of 1717347573.00000, found as of 1717347573.00000> to ShardRange<'k' to 't' as of 1717347574.00000, (0, 0) as of 1717347574.00000, cleaved as of 1717347574.00000>, path: .shards_a/shard_c_1717347574.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/469/07dd0e0d3b34ab0626b9e9c274c4b469/07dd0e0d3b34ab0626b9e9c274c4b469.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347576.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/5bc/42c16e333b034088143999ff506305bc/42c16e333b034088143999ff506305bc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347575.00000, (0, 0) as of 1717347575.00000, found as of 1717347575.00000> to ShardRange<'k' to 't' as of 1717347576.00000, (0, 0) as of 1717347576.00000, active as of 1717347576.00000>, path: .shards_a/shard_c_1717347576.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/5bc/42c16e333b034088143999ff506305bc/42c16e333b034088143999ff506305bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347578.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b1c/4e4191824acd4731333030325abe7b1c/4e4191824acd4731333030325abe7b1c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347577.00000, (0, 0) as of 1717347577.00000, found as of 1717347577.00000> to ShardRange<'k' to 't' as of 1717347578.00000, (0, 0) as of 1717347578.00000, shrinking as of 1717347578.00000>, path: .shards_a/shard_c_1717347578.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b1c/4e4191824acd4731333030325abe7b1c/4e4191824acd4731333030325abe7b1c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347578.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b1c/4e4191824acd4731333030325abe7b1c/4e4191824acd4731333030325abe7b1c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347580.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2b7/4036a7e8891da2063afc2b4db2a662b7/4036a7e8891da2063afc2b4db2a662b7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347579.00000, (0, 0) as of 1717347579.00000, found as of 1717347579.00000> to ShardRange<'k' to 't' as of 1717347580.00000, (0, 0) as of 1717347580.00000, sharding as of 1717347580.00000>, path: .shards_a/shard_c_1717347580.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2b7/4036a7e8891da2063afc2b4db2a662b7/4036a7e8891da2063afc2b4db2a662b7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347580.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2b7/4036a7e8891da2063afc2b4db2a662b7/4036a7e8891da2063afc2b4db2a662b7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347582.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c96/9c35ebf2c2da9a49336dccc3a6359c96/9c35ebf2c2da9a49336dccc3a6359c96.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347581.00000, (0, 0) as of 1717347581.00000, found as of 1717347581.00000> to ShardRange<'k' to 't' as of 1717347582.00000, (0, 0) as of 1717347582.00000, sharded as of 1717347582.00000>, path: .shards_a/shard_c_1717347582.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c96/9c35ebf2c2da9a49336dccc3a6359c96/9c35ebf2c2da9a49336dccc3a6359c96.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347582.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c96/9c35ebf2c2da9a49336dccc3a6359c96/9c35ebf2c2da9a49336dccc3a6359c96.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347584.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/84c/a7591461facd36eb69c99cee2c86784c/a7591461facd36eb69c99cee2c86784c.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347583.00000, (0, 0) as of 1717347583.00000, found as of 1717347583.00000> to ShardRange<'k' to 't' as of 1717347584.00000, (0, 0) as of 1717347584.00000, shrunk as of 1717347584.00000>, path: .shards_a/shard_c_1717347584.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/84c/a7591461facd36eb69c99cee2c86784c/a7591461facd36eb69c99cee2c86784c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347584.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/84c/a7591461facd36eb69c99cee2c86784c/a7591461facd36eb69c99cee2c86784c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347586.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c90/d3cac2eace99463e1ee8678c0c2edc90/d3cac2eace99463e1ee8678c0c2edc90.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347585.00000, (0, 0) as of 1717347585.00000, created as of 1717347585.00000> to ShardRange<'k' to 't' as of 1717347586.00000, (0, 0) as of 1717347586.00000, found as of 1717347586.00000>, path: .shards_a/shard_c_1717347586.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c90/d3cac2eace99463e1ee8678c0c2edc90/d3cac2eace99463e1ee8678c0c2edc90.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347588.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0ee/70f553867d7924fab8394588b10c10ee/70f553867d7924fab8394588b10c10ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347590.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/9fb/b05de8fabece49a9efb290faebccd9fb/b05de8fabece49a9efb290faebccd9fb.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347589.00000, (0, 0) as of 1717347589.00000, created as of 1717347589.00000> to ShardRange<'k' to 't' as of 1717347590.00000, (0, 0) as of 1717347590.00000, cleaved as of 1717347590.00000>, path: .shards_a/shard_c_1717347590.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/9fb/b05de8fabece49a9efb290faebccd9fb/b05de8fabece49a9efb290faebccd9fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347592.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/26d/9bc117fa0b6d6f708f944e0a49ace26d/9bc117fa0b6d6f708f944e0a49ace26d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347591.00000, (0, 0) as of 1717347591.00000, created as of 1717347591.00000> to ShardRange<'k' to 't' as of 1717347592.00000, (0, 0) as of 1717347592.00000, active as of 1717347592.00000>, path: .shards_a/shard_c_1717347592.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/26d/9bc117fa0b6d6f708f944e0a49ace26d/9bc117fa0b6d6f708f944e0a49ace26d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347594.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/729/a7b4c4357436e90778b65b0de8d2a729/a7b4c4357436e90778b65b0de8d2a729.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347593.00000, (0, 0) as of 1717347593.00000, created as of 1717347593.00000> to ShardRange<'k' to 't' as of 1717347594.00000, (0, 0) as of 1717347594.00000, shrinking as of 1717347594.00000>, path: .shards_a/shard_c_1717347594.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/729/a7b4c4357436e90778b65b0de8d2a729/a7b4c4357436e90778b65b0de8d2a729.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347594.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/729/a7b4c4357436e90778b65b0de8d2a729/a7b4c4357436e90778b65b0de8d2a729.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347596.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d3b/480689c569ecc2b36effcec24d25dd3b/480689c569ecc2b36effcec24d25dd3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347595.00000, (0, 0) as of 1717347595.00000, created as of 1717347595.00000> to ShardRange<'k' to 't' as of 1717347596.00000, (0, 0) as of 1717347596.00000, sharding as of 1717347596.00000>, path: .shards_a/shard_c_1717347596.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d3b/480689c569ecc2b36effcec24d25dd3b/480689c569ecc2b36effcec24d25dd3b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347596.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d3b/480689c569ecc2b36effcec24d25dd3b/480689c569ecc2b36effcec24d25dd3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347598.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/538/ed951779bae1d8879da21e18cbb6d538/ed951779bae1d8879da21e18cbb6d538.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347597.00000, (0, 0) as of 1717347597.00000, created as of 1717347597.00000> to ShardRange<'k' to 't' as of 1717347598.00000, (0, 0) as of 1717347598.00000, sharded as of 1717347598.00000>, path: .shards_a/shard_c_1717347598.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/538/ed951779bae1d8879da21e18cbb6d538/ed951779bae1d8879da21e18cbb6d538.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347598.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/538/ed951779bae1d8879da21e18cbb6d538/ed951779bae1d8879da21e18cbb6d538.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347600.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b02/cb8813844798d018fb6a75a5cd266b02/cb8813844798d018fb6a75a5cd266b02.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347599.00000, (0, 0) as of 1717347599.00000, created as of 1717347599.00000> to ShardRange<'k' to 't' as of 1717347600.00000, (0, 0) as of 1717347600.00000, shrunk as of 1717347600.00000>, path: .shards_a/shard_c_1717347600.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b02/cb8813844798d018fb6a75a5cd266b02/cb8813844798d018fb6a75a5cd266b02.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347600.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b02/cb8813844798d018fb6a75a5cd266b02/cb8813844798d018fb6a75a5cd266b02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347602.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/da4/40ba35370c26094f14b4a673493e2da4/40ba35370c26094f14b4a673493e2da4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347601.00000, (0, 0) as of 1717347601.00000, cleaved as of 1717347601.00000> to ShardRange<'k' to 't' as of 1717347602.00000, (0, 0) as of 1717347602.00000, found as of 1717347602.00000>, path: .shards_a/shard_c_1717347602.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/da4/40ba35370c26094f14b4a673493e2da4/40ba35370c26094f14b4a673493e2da4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347604.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/50b/6cf3babe52a190d3bc41512207fff50b/6cf3babe52a190d3bc41512207fff50b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347603.00000, (0, 0) as of 1717347603.00000, cleaved as of 1717347603.00000> to ShardRange<'k' to 't' as of 1717347604.00000, (0, 0) as of 1717347604.00000, created as of 1717347604.00000>, path: .shards_a/shard_c_1717347604.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/50b/6cf3babe52a190d3bc41512207fff50b/6cf3babe52a190d3bc41512207fff50b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347606.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/912/bac54f4ccd40aea7d16f45db0b37f912/bac54f4ccd40aea7d16f45db0b37f912.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347608.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/730/2192113353efc0b141e01898e0368730/2192113353efc0b141e01898e0368730.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347607.00000, (0, 0) as of 1717347607.00000, cleaved as of 1717347607.00000> to ShardRange<'k' to 't' as of 1717347608.00000, (0, 0) as of 1717347608.00000, active as of 1717347608.00000>, path: .shards_a/shard_c_1717347608.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/730/2192113353efc0b141e01898e0368730/2192113353efc0b141e01898e0368730.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347610.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/bfc/e6f9e2b8479867c543b80ee727448bfc/e6f9e2b8479867c543b80ee727448bfc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347609.00000, (0, 0) as of 1717347609.00000, cleaved as of 1717347609.00000> to ShardRange<'k' to 't' as of 1717347610.00000, (0, 0) as of 1717347610.00000, shrinking as of 1717347610.00000>, path: .shards_a/shard_c_1717347610.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/bfc/e6f9e2b8479867c543b80ee727448bfc/e6f9e2b8479867c543b80ee727448bfc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347610.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/bfc/e6f9e2b8479867c543b80ee727448bfc/e6f9e2b8479867c543b80ee727448bfc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347612.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/554/857484de50bb64537346fc4482491554/857484de50bb64537346fc4482491554.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347611.00000, (0, 0) as of 1717347611.00000, cleaved as of 1717347611.00000> to ShardRange<'k' to 't' as of 1717347612.00000, (0, 0) as of 1717347612.00000, sharding as of 1717347612.00000>, path: .shards_a/shard_c_1717347612.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/554/857484de50bb64537346fc4482491554/857484de50bb64537346fc4482491554.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347612.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/554/857484de50bb64537346fc4482491554/857484de50bb64537346fc4482491554.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347614.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/288/772b41b595459e0fad474cb66cc83288/772b41b595459e0fad474cb66cc83288.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347613.00000, (0, 0) as of 1717347613.00000, cleaved as of 1717347613.00000> to ShardRange<'k' to 't' as of 1717347614.00000, (0, 0) as of 1717347614.00000, sharded as of 1717347614.00000>, path: .shards_a/shard_c_1717347614.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/288/772b41b595459e0fad474cb66cc83288/772b41b595459e0fad474cb66cc83288.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347614.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/288/772b41b595459e0fad474cb66cc83288/772b41b595459e0fad474cb66cc83288.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347616.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d09/5b172d789fcae78d96e28231ebf61d09/5b172d789fcae78d96e28231ebf61d09.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347615.00000, (0, 0) as of 1717347615.00000, cleaved as of 1717347615.00000> to ShardRange<'k' to 't' as of 1717347616.00000, (0, 0) as of 1717347616.00000, shrunk as of 1717347616.00000>, path: .shards_a/shard_c_1717347616.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d09/5b172d789fcae78d96e28231ebf61d09/5b172d789fcae78d96e28231ebf61d09.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347616.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d09/5b172d789fcae78d96e28231ebf61d09/5b172d789fcae78d96e28231ebf61d09.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347618.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/975/d8d889042ecf29323a1f19221a2ca975/d8d889042ecf29323a1f19221a2ca975.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347617.00000, (0, 0) as of 1717347617.00000, active as of 1717347617.00000> to ShardRange<'k' to 't' as of 1717347618.00000, (0, 0) as of 1717347618.00000, found as of 1717347618.00000>, path: .shards_a/shard_c_1717347618.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/975/d8d889042ecf29323a1f19221a2ca975/d8d889042ecf29323a1f19221a2ca975.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347620.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347619.00000, (0, 0) as of 1717347619.00000, active as of 1717347619.00000> to ShardRange<'k' to 't' as of 1717347620.00000, (0, 0) as of 1717347620.00000, created as of 1717347620.00000>, path: .shards_a/shard_c_1717347620.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62/3269bbe6c2cbe1c8a2984ae6ac0d9c62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347622.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/053/a4e09fd8c8ae6bae6244b48a3bbf0053/a4e09fd8c8ae6bae6244b48a3bbf0053.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347621.00000, (0, 0) as of 1717347621.00000, active as of 1717347621.00000> to ShardRange<'k' to 't' as of 1717347622.00000, (0, 0) as of 1717347622.00000, cleaved as of 1717347622.00000>, path: .shards_a/shard_c_1717347622.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/053/a4e09fd8c8ae6bae6244b48a3bbf0053/a4e09fd8c8ae6bae6244b48a3bbf0053.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347624.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4fb/f0194b74a2cc438488b7bab88c7c84fb/f0194b74a2cc438488b7bab88c7c84fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347626.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/7cc/44083eb73e0b2a9ecfe13c79685d67cc/44083eb73e0b2a9ecfe13c79685d67cc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347625.00000, (0, 0) as of 1717347625.00000, active as of 1717347625.00000> to ShardRange<'k' to 't' as of 1717347626.00000, (0, 0) as of 1717347626.00000, shrinking as of 1717347626.00000>, path: .shards_a/shard_c_1717347626.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/7cc/44083eb73e0b2a9ecfe13c79685d67cc/44083eb73e0b2a9ecfe13c79685d67cc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347626.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/7cc/44083eb73e0b2a9ecfe13c79685d67cc/44083eb73e0b2a9ecfe13c79685d67cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347628.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0e9/6b924c534b5577f457d8cb4a8afa80e9/6b924c534b5577f457d8cb4a8afa80e9.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347627.00000, (0, 0) as of 1717347627.00000, active as of 1717347627.00000> to ShardRange<'k' to 't' as of 1717347628.00000, (0, 0) as of 1717347628.00000, sharding as of 1717347628.00000>, path: .shards_a/shard_c_1717347628.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0e9/6b924c534b5577f457d8cb4a8afa80e9/6b924c534b5577f457d8cb4a8afa80e9.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347628.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0e9/6b924c534b5577f457d8cb4a8afa80e9/6b924c534b5577f457d8cb4a8afa80e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347630.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/8f7/a3655c7da85a5b4b0846e60a8b13a8f7/a3655c7da85a5b4b0846e60a8b13a8f7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347629.00000, (0, 0) as of 1717347629.00000, active as of 1717347629.00000> to ShardRange<'k' to 't' as of 1717347630.00000, (0, 0) as of 1717347630.00000, sharded as of 1717347630.00000>, path: .shards_a/shard_c_1717347630.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/8f7/a3655c7da85a5b4b0846e60a8b13a8f7/a3655c7da85a5b4b0846e60a8b13a8f7.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347630.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/8f7/a3655c7da85a5b4b0846e60a8b13a8f7/a3655c7da85a5b4b0846e60a8b13a8f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347632.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/167/d15b78536e91bf1514e8847bc6809167/d15b78536e91bf1514e8847bc6809167.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347631.00000, (0, 0) as of 1717347631.00000, active as of 1717347631.00000> to ShardRange<'k' to 't' as of 1717347632.00000, (0, 0) as of 1717347632.00000, shrunk as of 1717347632.00000>, path: .shards_a/shard_c_1717347632.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/167/d15b78536e91bf1514e8847bc6809167/d15b78536e91bf1514e8847bc6809167.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347632.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/167/d15b78536e91bf1514e8847bc6809167/d15b78536e91bf1514e8847bc6809167.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347634.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4f7/59813a133ee4d28bf3292c9fd1a074f7/59813a133ee4d28bf3292c9fd1a074f7.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347633.00000, (0, 0) as of 1717347633.00000, shrinking as of 1717347633.00000> to ShardRange<'k' to 't' as of 1717347634.00000, (0, 0) as of 1717347634.00000, found as of 1717347634.00000>, path: .shards_a/shard_c_1717347634.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4f7/59813a133ee4d28bf3292c9fd1a074f7/59813a133ee4d28bf3292c9fd1a074f7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347636.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/a30/8d8f0f1e5cebd170889c9634ae569a30/8d8f0f1e5cebd170889c9634ae569a30.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347635.00000, (0, 0) as of 1717347635.00000, shrinking as of 1717347635.00000> to ShardRange<'k' to 't' as of 1717347636.00000, (0, 0) as of 1717347636.00000, created as of 1717347636.00000>, path: .shards_a/shard_c_1717347636.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/a30/8d8f0f1e5cebd170889c9634ae569a30/8d8f0f1e5cebd170889c9634ae569a30.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347638.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4dc/00c9534fdabeefdcabf68e2943f7f4dc/00c9534fdabeefdcabf68e2943f7f4dc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347637.00000, (0, 0) as of 1717347637.00000, shrinking as of 1717347637.00000> to ShardRange<'k' to 't' as of 1717347638.00000, (0, 0) as of 1717347638.00000, cleaved as of 1717347638.00000>, path: .shards_a/shard_c_1717347638.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4dc/00c9534fdabeefdcabf68e2943f7f4dc/00c9534fdabeefdcabf68e2943f7f4dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347640.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/60a/39531825230642a57978f47a5874660a/39531825230642a57978f47a5874660a.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347639.00000, (0, 0) as of 1717347639.00000, shrinking as of 1717347639.00000> to ShardRange<'k' to 't' as of 1717347640.00000, (0, 0) as of 1717347640.00000, active as of 1717347640.00000>, path: .shards_a/shard_c_1717347640.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/60a/39531825230642a57978f47a5874660a/39531825230642a57978f47a5874660a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347642.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/07c/a49ec302fcc8bec304fbbf13ee39b07c/a49ec302fcc8bec304fbbf13ee39b07c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347642.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/07c/a49ec302fcc8bec304fbbf13ee39b07c/a49ec302fcc8bec304fbbf13ee39b07c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347644.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c26/c2719e4be550e8f6bba30053f8b0dc26/c2719e4be550e8f6bba30053f8b0dc26.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347643.00000, (0, 0) as of 1717347643.00000, shrinking as of 1717347643.00000> to ShardRange<'k' to 't' as of 1717347644.00000, (0, 0) as of 1717347644.00000, sharding as of 1717347644.00000>, path: .shards_a/shard_c_1717347644.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c26/c2719e4be550e8f6bba30053f8b0dc26/c2719e4be550e8f6bba30053f8b0dc26.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347644.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c26/c2719e4be550e8f6bba30053f8b0dc26/c2719e4be550e8f6bba30053f8b0dc26.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347646.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/038/07dfc1d5715794bbc39721b607477038/07dfc1d5715794bbc39721b607477038.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347645.00000, (0, 0) as of 1717347645.00000, shrinking as of 1717347645.00000> to ShardRange<'k' to 't' as of 1717347646.00000, (0, 0) as of 1717347646.00000, sharded as of 1717347646.00000>, path: .shards_a/shard_c_1717347646.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/038/07dfc1d5715794bbc39721b607477038/07dfc1d5715794bbc39721b607477038.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347646.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/038/07dfc1d5715794bbc39721b607477038/07dfc1d5715794bbc39721b607477038.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347648.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c42/9124a4aeb227914548aa009d3f504c42/9124a4aeb227914548aa009d3f504c42.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347647.00000, (0, 0) as of 1717347647.00000, shrinking as of 1717347647.00000> to ShardRange<'k' to 't' as of 1717347648.00000, (0, 0) as of 1717347648.00000, shrunk as of 1717347648.00000>, path: .shards_a/shard_c_1717347648.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c42/9124a4aeb227914548aa009d3f504c42/9124a4aeb227914548aa009d3f504c42.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347648.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c42/9124a4aeb227914548aa009d3f504c42/9124a4aeb227914548aa009d3f504c42.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347650.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/173/fe560fbc02d686927dca800a9dbc3173/fe560fbc02d686927dca800a9dbc3173.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347649.00000, (0, 0) as of 1717347649.00000, sharding as of 1717347649.00000> to ShardRange<'k' to 't' as of 1717347650.00000, (0, 0) as of 1717347650.00000, found as of 1717347650.00000>, path: .shards_a/shard_c_1717347650.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/173/fe560fbc02d686927dca800a9dbc3173/fe560fbc02d686927dca800a9dbc3173.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347652.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/fa6/d57b70349d40b1fb1d531ddd1e514fa6/d57b70349d40b1fb1d531ddd1e514fa6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347651.00000, (0, 0) as of 1717347651.00000, sharding as of 1717347651.00000> to ShardRange<'k' to 't' as of 1717347652.00000, (0, 0) as of 1717347652.00000, created as of 1717347652.00000>, path: .shards_a/shard_c_1717347652.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/fa6/d57b70349d40b1fb1d531ddd1e514fa6/d57b70349d40b1fb1d531ddd1e514fa6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347654.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6b4/c9afbccbfe36ae619835c127424ed6b4/c9afbccbfe36ae619835c127424ed6b4.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347653.00000, (0, 0) as of 1717347653.00000, sharding as of 1717347653.00000> to ShardRange<'k' to 't' as of 1717347654.00000, (0, 0) as of 1717347654.00000, cleaved as of 1717347654.00000>, path: .shards_a/shard_c_1717347654.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6b4/c9afbccbfe36ae619835c127424ed6b4/c9afbccbfe36ae619835c127424ed6b4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347656.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2a8/074cd982b7ad263001d12006e2a772a8/074cd982b7ad263001d12006e2a772a8.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347655.00000, (0, 0) as of 1717347655.00000, sharding as of 1717347655.00000> to ShardRange<'k' to 't' as of 1717347656.00000, (0, 0) as of 1717347656.00000, active as of 1717347656.00000>, path: .shards_a/shard_c_1717347656.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2a8/074cd982b7ad263001d12006e2a772a8/074cd982b7ad263001d12006e2a772a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347658.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/142/72654e67a130da0b4c9994fa357d1142/72654e67a130da0b4c9994fa357d1142.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347657.00000, (0, 0) as of 1717347657.00000, sharding as of 1717347657.00000> to ShardRange<'k' to 't' as of 1717347658.00000, (0, 0) as of 1717347658.00000, shrinking as of 1717347658.00000>, path: .shards_a/shard_c_1717347658.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/142/72654e67a130da0b4c9994fa357d1142/72654e67a130da0b4c9994fa357d1142.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347658.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/142/72654e67a130da0b4c9994fa357d1142/72654e67a130da0b4c9994fa357d1142.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347660.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cd2/1889c5322918afbc174706402e9d0cd2/1889c5322918afbc174706402e9d0cd2.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347660.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cd2/1889c5322918afbc174706402e9d0cd2/1889c5322918afbc174706402e9d0cd2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347662.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f62/3c282437f4a35c0c8442b800f9bebf62/3c282437f4a35c0c8442b800f9bebf62.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347661.00000, (0, 0) as of 1717347661.00000, sharding as of 1717347661.00000> to ShardRange<'k' to 't' as of 1717347662.00000, (0, 0) as of 1717347662.00000, sharded as of 1717347662.00000>, path: .shards_a/shard_c_1717347662.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f62/3c282437f4a35c0c8442b800f9bebf62/3c282437f4a35c0c8442b800f9bebf62.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347662.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f62/3c282437f4a35c0c8442b800f9bebf62/3c282437f4a35c0c8442b800f9bebf62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347664.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/7c3/e730357db3a99bf8e85a5edf45c3f7c3/e730357db3a99bf8e85a5edf45c3f7c3.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347663.00000, (0, 0) as of 1717347663.00000, sharding as of 1717347663.00000> to ShardRange<'k' to 't' as of 1717347664.00000, (0, 0) as of 1717347664.00000, shrunk as of 1717347664.00000>, path: .shards_a/shard_c_1717347664.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/7c3/e730357db3a99bf8e85a5edf45c3f7c3/e730357db3a99bf8e85a5edf45c3f7c3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347664.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/7c3/e730357db3a99bf8e85a5edf45c3f7c3/e730357db3a99bf8e85a5edf45c3f7c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347666.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/14e/298fa02c11aeb357735c26e5b2cbc14e/298fa02c11aeb357735c26e5b2cbc14e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347665.00000, (0, 0) as of 1717347665.00000, sharded as of 1717347665.00000> to ShardRange<'k' to 't' as of 1717347666.00000, (0, 0) as of 1717347666.00000, found as of 1717347666.00000>, path: .shards_a/shard_c_1717347666.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/14e/298fa02c11aeb357735c26e5b2cbc14e/298fa02c11aeb357735c26e5b2cbc14e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347668.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/892/5cb60017e4b5671717e514d1c6744892/5cb60017e4b5671717e514d1c6744892.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347667.00000, (0, 0) as of 1717347667.00000, sharded as of 1717347667.00000> to ShardRange<'k' to 't' as of 1717347668.00000, (0, 0) as of 1717347668.00000, created as of 1717347668.00000>, path: .shards_a/shard_c_1717347668.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/892/5cb60017e4b5671717e514d1c6744892/5cb60017e4b5671717e514d1c6744892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347670.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/37b/8c09e68091cd469f053bfd06e509237b/8c09e68091cd469f053bfd06e509237b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347669.00000, (0, 0) as of 1717347669.00000, sharded as of 1717347669.00000> to ShardRange<'k' to 't' as of 1717347670.00000, (0, 0) as of 1717347670.00000, cleaved as of 1717347670.00000>, path: .shards_a/shard_c_1717347670.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/37b/8c09e68091cd469f053bfd06e509237b/8c09e68091cd469f053bfd06e509237b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347672.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/91f/dca688e5b450c45ad7e0b8cc5bdfd91f/dca688e5b450c45ad7e0b8cc5bdfd91f.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347671.00000, (0, 0) as of 1717347671.00000, sharded as of 1717347671.00000> to ShardRange<'k' to 't' as of 1717347672.00000, (0, 0) as of 1717347672.00000, active as of 1717347672.00000>, path: .shards_a/shard_c_1717347672.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/91f/dca688e5b450c45ad7e0b8cc5bdfd91f/dca688e5b450c45ad7e0b8cc5bdfd91f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347674.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6e6/a0ac6329c280c463624b6d8e558a06e6/a0ac6329c280c463624b6d8e558a06e6.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347673.00000, (0, 0) as of 1717347673.00000, sharded as of 1717347673.00000> to ShardRange<'k' to 't' as of 1717347674.00000, (0, 0) as of 1717347674.00000, shrinking as of 1717347674.00000>, path: .shards_a/shard_c_1717347674.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6e6/a0ac6329c280c463624b6d8e558a06e6/a0ac6329c280c463624b6d8e558a06e6.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347674.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6e6/a0ac6329c280c463624b6d8e558a06e6/a0ac6329c280c463624b6d8e558a06e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347676.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/3bc/2f5681d343374c08c138b5a9c47f63bc/2f5681d343374c08c138b5a9c47f63bc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347675.00000, (0, 0) as of 1717347675.00000, sharded as of 1717347675.00000> to ShardRange<'k' to 't' as of 1717347676.00000, (0, 0) as of 1717347676.00000, sharding as of 1717347676.00000>, path: .shards_a/shard_c_1717347676.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/3bc/2f5681d343374c08c138b5a9c47f63bc/2f5681d343374c08c138b5a9c47f63bc.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347676.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/3bc/2f5681d343374c08c138b5a9c47f63bc/2f5681d343374c08c138b5a9c47f63bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347678.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/a75/5fd9a7ba135c7dc5c81f8421105fda75/5fd9a7ba135c7dc5c81f8421105fda75.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347678.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/a75/5fd9a7ba135c7dc5c81f8421105fda75/5fd9a7ba135c7dc5c81f8421105fda75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347680.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/37e/d0937d32d57ff508e8a222915638237e/d0937d32d57ff508e8a222915638237e.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347679.00000, (0, 0) as of 1717347679.00000, sharded as of 1717347679.00000> to ShardRange<'k' to 't' as of 1717347680.00000, (0, 0) as of 1717347680.00000, shrunk as of 1717347680.00000>, path: .shards_a/shard_c_1717347680.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/37e/d0937d32d57ff508e8a222915638237e/d0937d32d57ff508e8a222915638237e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347680.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/37e/d0937d32d57ff508e8a222915638237e/d0937d32d57ff508e8a222915638237e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347682.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/77d/35cf5edad16247e789985f2535bec77d/35cf5edad16247e789985f2535bec77d.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347681.00000, (0, 0) as of 1717347681.00000, shrunk as of 1717347681.00000> to ShardRange<'k' to 't' as of 1717347682.00000, (0, 0) as of 1717347682.00000, found as of 1717347682.00000>, path: .shards_a/shard_c_1717347682.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/77d/35cf5edad16247e789985f2535bec77d/35cf5edad16247e789985f2535bec77d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347684.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c01/387ac88eab20559a19b8c37e0b275c01/387ac88eab20559a19b8c37e0b275c01.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347683.00000, (0, 0) as of 1717347683.00000, shrunk as of 1717347683.00000> to ShardRange<'k' to 't' as of 1717347684.00000, (0, 0) as of 1717347684.00000, created as of 1717347684.00000>, path: .shards_a/shard_c_1717347684.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/c01/387ac88eab20559a19b8c37e0b275c01/387ac88eab20559a19b8c37e0b275c01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347686.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/edc/0a35d8305e3aca3cab17aa5d6a8dbedc/0a35d8305e3aca3cab17aa5d6a8dbedc.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347685.00000, (0, 0) as of 1717347685.00000, shrunk as of 1717347685.00000> to ShardRange<'k' to 't' as of 1717347686.00000, (0, 0) as of 1717347686.00000, cleaved as of 1717347686.00000>, path: .shards_a/shard_c_1717347686.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/edc/0a35d8305e3aca3cab17aa5d6a8dbedc/0a35d8305e3aca3cab17aa5d6a8dbedc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347688.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e61/0d00dee615d776fdd6b2343df4edee61/0d00dee615d776fdd6b2343df4edee61.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347687.00000, (0, 0) as of 1717347687.00000, shrunk as of 1717347687.00000> to ShardRange<'k' to 't' as of 1717347688.00000, (0, 0) as of 1717347688.00000, active as of 1717347688.00000>, path: .shards_a/shard_c_1717347688.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e61/0d00dee615d776fdd6b2343df4edee61/0d00dee615d776fdd6b2343df4edee61.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347690.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e14/c0ce8e2407c2414443e34e4ffc983e14/c0ce8e2407c2414443e34e4ffc983e14.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347689.00000, (0, 0) as of 1717347689.00000, shrunk as of 1717347689.00000> to ShardRange<'k' to 't' as of 1717347690.00000, (0, 0) as of 1717347690.00000, shrinking as of 1717347690.00000>, path: .shards_a/shard_c_1717347690.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e14/c0ce8e2407c2414443e34e4ffc983e14/c0ce8e2407c2414443e34e4ffc983e14.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347690.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e14/c0ce8e2407c2414443e34e4ffc983e14/c0ce8e2407c2414443e34e4ffc983e14.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347692.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/1cd/4a57076b44057ac735701c7aa65ec1cd/4a57076b44057ac735701c7aa65ec1cd.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347691.00000, (0, 0) as of 1717347691.00000, shrunk as of 1717347691.00000> to ShardRange<'k' to 't' as of 1717347692.00000, (0, 0) as of 1717347692.00000, sharding as of 1717347692.00000>, path: .shards_a/shard_c_1717347692.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/1cd/4a57076b44057ac735701c7aa65ec1cd/4a57076b44057ac735701c7aa65ec1cd.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347692.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/1cd/4a57076b44057ac735701c7aa65ec1cd/4a57076b44057ac735701c7aa65ec1cd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347694.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b8b/63246a2cbcda46ab622668e3ae9a8b8b/63246a2cbcda46ab622668e3ae9a8b8b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347693.00000, (0, 0) as of 1717347693.00000, shrunk as of 1717347693.00000> to ShardRange<'k' to 't' as of 1717347694.00000, (0, 0) as of 1717347694.00000, sharded as of 1717347694.00000>, path: .shards_a/shard_c_1717347694.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b8b/63246a2cbcda46ab622668e3ae9a8b8b/63246a2cbcda46ab622668e3ae9a8b8b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347694.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/b8b/63246a2cbcda46ab622668e3ae9a8b8b/63246a2cbcda46ab622668e3ae9a8b8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347696.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cbe/79364d0fbc00195bc812679421d12cbe/79364d0fbc00195bc812679421d12cbe.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347696.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cbe/79364d0fbc00195bc812679421d12cbe/79364d0fbc00195bc812679421d12cbe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347697.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/783/eb360b501ebd66d0ac6ef38760e40783/eb360b501ebd66d0ac6ef38760e40783.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347699.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ee6/88a24c4ca91fc6ee40ceac03ae63fee6/88a24c4ca91fc6ee40ceac03ae63fee6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347701.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/541/89e441df910ae09679d6c259d1223541/89e441df910ae09679d6c259d1223541.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347703.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/1e6/447dfad4870f604b7ef9da37dd5341e6/447dfad4870f604b7ef9da37dd5341e6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347705.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/641/7fb7a4aeea0107d5a213f7f332eea641/7fb7a4aeea0107d5a213f7f332eea641.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347707.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/77e/e2ae7477d24c595a81a662f8049f777e/e2ae7477d24c595a81a662f8049f777e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347709.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2bf/ac798cc03a8462e14b05271c0e26f2bf/ac798cc03a8462e14b05271c0e26f2bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347711.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/fec/01bf42e316fb45675e57f0b8073ebfec/01bf42e316fb45675e57f0b8073ebfec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347713.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ca8/6df4a68a8a7734566efb70095e24bca8/6df4a68a8a7734566efb70095e24bca8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347715.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6d4/e2f329bf224e83d2845c07579d5ac6d4/e2f329bf224e83d2845c07579d5ac6d4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347717.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0ae/7663691181efb0a475df6832ecaf70ae/7663691181efb0a475df6832ecaf70ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347719.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/9c1/d3f8f1c74636075987cb6dee406b89c1/d3f8f1c74636075987cb6dee406b89c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347721.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f5c/51b325346429387282c9b8ae5398bf5c/51b325346429387282c9b8ae5398bf5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347723.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/59b/2c01f66ed60c4ff4a8c7e3cdb6ea559b/2c01f66ed60c4ff4a8c7e3cdb6ea559b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347725.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cc2/1d87c710aaa6069dbe6dcb10f9783cc2/1d87c710aaa6069dbe6dcb10f9783cc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347727.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/df7/b15201b52f7dedef31520f2b88b06df7/b15201b52f7dedef31520f2b88b06df7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347729.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/713/b44793ca64fed6ec80ab85bccc259713/b44793ca64fed6ec80ab85bccc259713.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347731.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/72d/84340448ece42bf8b763d9d22329c72d/84340448ece42bf8b763d9d22329c72d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347733.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/247/2f72086e9567be59fabae7b0f24fc247/2f72086e9567be59fabae7b0f24fc247.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347735.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/fed/9f27e843d6a46ca1498c3c8586ddffed/9f27e843d6a46ca1498c3c8586ddffed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347737.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/9cf/4fd1e46222678d500a338950dbe429cf/4fd1e46222678d500a338950dbe429cf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347739.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ab8/81e8f3dbb37962063734c2788e86cab8/81e8f3dbb37962063734c2788e86cab8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347741.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/87c/a8d69f38e6efed3092e68f6ec3a7787c/a8d69f38e6efed3092e68f6ec3a7787c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347743.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/76d/732d5acac6605750c4407bee124cf76d/732d5acac6605750c4407bee124cf76d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347745.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/70b/cbc55291c015632d748b6bfb26be570b/cbc55291c015632d748b6bfb26be570b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347747.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/3e4/168af284255765a320e4e9019cf1d3e4/168af284255765a320e4e9019cf1d3e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347749.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/612/907f0f2b4f9eafcc461de12596f06612/907f0f2b4f9eafcc461de12596f06612.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347751.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/73a/2f075fec0558d09cfe1464b79172273a/2f075fec0558d09cfe1464b79172273a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347753.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/38a/a512729c82f210b0b637683f9185038a/a512729c82f210b0b637683f9185038a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347755.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/8ae/8e125fe70da828d36913a687b50bb8ae/8e125fe70da828d36913a687b50bb8ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347757.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/516/4d88add553ff68178db81a35ff8c4516/4d88add553ff68178db81a35ff8c4516.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347759.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/291/c523317f40985535da16fb0a6f708291/c523317f40985535da16fb0a6f708291.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347761.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d1e/517c182950428b8026e9d67a96075d1e/517c182950428b8026e9d67a96075d1e.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347761.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/d1e/517c182950428b8026e9d67a96075d1e/517c182950428b8026e9d67a96075d1e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347763.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e9c/85555a3f841f726690cb6593d473de9c/85555a3f841f726690cb6593d473de9c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347763.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e9c/85555a3f841f726690cb6593d473de9c/85555a3f841f726690cb6593d473de9c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347765.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/539/b0b9b89c6c0f66ec21d3ba9a8f35d539/b0b9b89c6c0f66ec21d3ba9a8f35d539.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347765.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/539/b0b9b89c6c0f66ec21d3ba9a8f35d539/b0b9b89c6c0f66ec21d3ba9a8f35d539.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347767.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/a4b/b05a488a705698f2be5b6c65b933fa4b/b05a488a705698f2be5b6c65b933fa4b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347767.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/a4b/b05a488a705698f2be5b6c65b933fa4b/b05a488a705698f2be5b6c65b933fa4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347769.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/515/deab18cf0e11f99039f4b827dabe5515/deab18cf0e11f99039f4b827dabe5515.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347769.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/515/deab18cf0e11f99039f4b827dabe5515/deab18cf0e11f99039f4b827dabe5515.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347771.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/84f/2d2b9e30412dcdef3b53e4b90b76884f/2d2b9e30412dcdef3b53e4b90b76884f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347771.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/84f/2d2b9e30412dcdef3b53e4b90b76884f/2d2b9e30412dcdef3b53e4b90b76884f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347773.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f3c/b3c55d6c835ed1d8cf9229f4de748f3c/b3c55d6c835ed1d8cf9229f4de748f3c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347773.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f3c/b3c55d6c835ed1d8cf9229f4de748f3c/b3c55d6c835ed1d8cf9229f4de748f3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347775.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ed0/3da0b906614f1d69f381719f74d90ed0/3da0b906614f1d69f381719f74d90ed0.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347775.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/ed0/3da0b906614f1d69f381719f74d90ed0/3da0b906614f1d69f381719f74d90ed0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347777.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/59d/367f0316445c5e3693983d52e6efb59d/367f0316445c5e3693983d52e6efb59d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347777.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/59d/367f0316445c5e3693983d52e6efb59d/367f0316445c5e3693983d52e6efb59d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347779.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2db/3ad8759f8926220e8a6c4432900b32db/3ad8759f8926220e8a6c4432900b32db.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347779.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2db/3ad8759f8926220e8a6c4432900b32db/3ad8759f8926220e8a6c4432900b32db.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347781.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cc8/7a8d5f11809425d249947b168de76cc8/7a8d5f11809425d249947b168de76cc8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347781.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/cc8/7a8d5f11809425d249947b168de76cc8/7a8d5f11809425d249947b168de76cc8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347783.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/948/6b8eae1c3a767c32b320af8adadd0948/6b8eae1c3a767c32b320af8adadd0948.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347783.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/948/6b8eae1c3a767c32b320af8adadd0948/6b8eae1c3a767c32b320af8adadd0948.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347785.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/707/3168133b50d23ba8db24474e6ed3d707/3168133b50d23ba8db24474e6ed3d707.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347785.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/707/3168133b50d23ba8db24474e6ed3d707/3168133b50d23ba8db24474e6ed3d707.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347787.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/444/72ac5a4e1cbcc5493501c9f905bdc444/72ac5a4e1cbcc5493501c9f905bdc444.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347787.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/444/72ac5a4e1cbcc5493501c9f905bdc444/72ac5a4e1cbcc5493501c9f905bdc444.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347789.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f40/876f77696ad83bfab45dbd5517ca5f40/876f77696ad83bfab45dbd5517ca5f40.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347789.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f40/876f77696ad83bfab45dbd5517ca5f40/876f77696ad83bfab45dbd5517ca5f40.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347791.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/9cb/8ab7640f6b9912fc871e8e57866969cb/8ab7640f6b9912fc871e8e57866969cb.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347791.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/9cb/8ab7640f6b9912fc871e8e57866969cb/8ab7640f6b9912fc871e8e57866969cb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347793.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/051/bdae4541400ecdc86d45d8799e555051/bdae4541400ecdc86d45d8799e555051.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347793.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/051/bdae4541400ecdc86d45d8799e555051/bdae4541400ecdc86d45d8799e555051.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347795.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/efe/7043d202c2894ad9d1088fc5363b1efe/7043d202c2894ad9d1088fc5363b1efe.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347795.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/efe/7043d202c2894ad9d1088fc5363b1efe/7043d202c2894ad9d1088fc5363b1efe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347797.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/100/ffae73052fb179814a521ca2e4ece100/ffae73052fb179814a521ca2e4ece100.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347797.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/100/ffae73052fb179814a521ca2e4ece100/ffae73052fb179814a521ca2e4ece100.db test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges test.unit.container.test_sharder.TestSharder.test_audit_shard_container_merge_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_in_root_container ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container test.unit.container.test_sharder.TestSharder.test_audit_shard_deleted_range_missing_from_root_container ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347799.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e0b/72afd535a23f5501b187d58f2ce3ee0b/72afd535a23f5501b187d58f2ce3ee0b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347799.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e0b/72afd535a23f5501b187d58f2ce3ee0b/72afd535a23f5501b187d58f2ce3ee0b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347801.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e9f/a073666f141375aafe32fc85c709ee9f/a073666f141375aafe32fc85c709ee9f.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347801.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e9f/a073666f141375aafe32fc85c709ee9f/a073666f141375aafe32fc85c709ee9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347803.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/69d/04a857082e13d5650487c5d4fb5f269d/04a857082e13d5650487c5d4fb5f269d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347803.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/69d/04a857082e13d5650487c5d4fb5f269d/04a857082e13d5650487c5d4fb5f269d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347805.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f20/e1b864a098e7e97bacc49f0626c90f20/e1b864a098e7e97bacc49f0626c90f20.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347805.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f20/e1b864a098e7e97bacc49f0626c90f20/e1b864a098e7e97bacc49f0626c90f20.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347807.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/586/441a1b04fdf19a58d50f32ea943cf586/441a1b04fdf19a58d50f32ea943cf586.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347807.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/586/441a1b04fdf19a58d50f32ea943cf586/441a1b04fdf19a58d50f32ea943cf586.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347809.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/71d/42d785e251bd44da1a301939825a771d/42d785e251bd44da1a301939825a771d.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347809.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/71d/42d785e251bd44da1a301939825a771d/42d785e251bd44da1a301939825a771d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347811.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e8c/9dbab490338ee8a71220329686173e8c/9dbab490338ee8a71220329686173e8c.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347811.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/e8c/9dbab490338ee8a71220329686173e8c/9dbab490338ee8a71220329686173e8c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347813.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6af/919f704c03e6d170a1d0a9f7e03c46af/919f704c03e6d170a1d0a9f7e03c46af.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347813.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6af/919f704c03e6d170a1d0a9f7e03c46af/919f704c03e6d170a1d0a9f7e03c46af.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347815.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0e5/a679f87c850e0ab053274866a81220e5/a679f87c850e0ab053274866a81220e5.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347815.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/0e5/a679f87c850e0ab053274866a81220e5/a679f87c850e0ab053274866a81220e5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347817.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2e4/7de1356866ab8169799600a6b27be2e4/7de1356866ab8169799600a6b27be2e4.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347817.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/2e4/7de1356866ab8169799600a6b27be2e4/7de1356866ab8169799600a6b27be2e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347819.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6a8/4c18a466b2305837078cee9b005386a8/4c18a466b2305837078cee9b005386a8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347819.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/6a8/4c18a466b2305837078cee9b005386a8/4c18a466b2305837078cee9b005386a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347821.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4ce/d66da585757b414a19583ab17a88c4ce/d66da585757b414a19583ab17a88c4ce.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347821.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/4ce/d66da585757b414a19583ab17a88c4ce/d66da585757b414a19583ab17a88c4ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c_1717347823.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f8b/321fc0224774ddfed2828c9115b2ff8b/321fc0224774ddfed2828c9115b2ff8b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/shard_c_1717347823.00000, db: /tmp/tmpbb68ewpf/sda/containers/0/f8b/321fc0224774ddfed2828c9115b2ff8b/321fc0224774ddfed2828c9115b2ff8b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/shard_c, db: /tmp/tmpufwhvqk1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Updated own shard range from ShardRange<'k' to 't' as of 1717347574.00000, (0, 0) as of 1717347574.00000, active as of 1717347574.00000> to ShardRange<'k' to 't' as of 1717347575.00000, (0, 0) as of 1717347575.00000, sharded as of 1717347576.00000>, path: .shards_a/shard_c, db: /tmp/tmpufwhvqk1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmpufwhvqk1/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Marked shard container as deleted, path: .shards_a/shard_c, db: /tmp/tmp_5worvxx/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347575.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/9c0/3c9103365b9f07c97ada0f0bcdd9b9c0/3c9103365b9f07c97ada0f0bcdd9b9c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347578.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/ae0/fe7dc0b4fcabae928dc774d20f189ae0/fe7dc0b4fcabae928dc774d20f189ae0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347581.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/87c/095c2a458ebb8e52c9cd8b9fbbdbb87c/095c2a458ebb8e52c9cd8b9fbbdbb87c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347584.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/8ba/fd67c357747e438b715abc1516b398ba/fd67c357747e438b715abc1516b398ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347587.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/0d2/730115b1f8bafac52ebf0b3d880ed0d2/730115b1f8bafac52ebf0b3d880ed0d2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347590.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/658/9b50e5c8b4812ff4627ea04754907658/9b50e5c8b4812ff4627ea04754907658.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347593.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/720/c23b69e1d2afa0452f2d4e8fa937c720/c23b69e1d2afa0452f2d4e8fa937c720.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_not_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_range_overlap_not_merged_while_shrinking ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347596.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/952/0d04f62a905f37d3008784e8df3f8952/0d04f62a905f37d3008784e8df3f8952.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347599.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/78e/6d41d2ba87c5329e184ef9cfb14ce78e/6d41d2ba87c5329e184ef9cfb14ce78e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347602.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/41f/d083cf73b1a64f5c94056112ffea341f/d083cf73b1a64f5c94056112ffea341f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347605.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/34a/99fed0ed4ce2b49a1153aa26d07fb34a/99fed0ed4ce2b49a1153aa26d07fb34a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347608.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/cb5/94096e279e8022b3c12b658150de5cb5/94096e279e8022b3c12b658150de5cb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347611.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/c15/7a1b1890283c6d39e2b2a8372f554c15/7a1b1890283c6d39e2b2a8372f554c15.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347614.00000-0, db: /tmp/tmpkidh_zxp/sda/containers/0/846/75d938b921368310b53a2f14c8056846/75d938b921368310b53a2f14c8056846.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347575.00000-0, db: /tmp/tmpnftfs40h/sda/containers/0/9c0/3c9103365b9f07c97ada0f0bcdd9b9c0/3c9103365b9f07c97ada0f0bcdd9b9c0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347579.00000-0, db: /tmp/tmpnftfs40h/sda/containers/0/96c/acc4be3c2a5435a70ee5e100715ac96c/acc4be3c2a5435a70ee5e100715ac96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347577.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/f88/44578b1f5aee67a35309ce9691557f88/44578b1f5aee67a35309ce9691557f88.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347581.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/87c/095c2a458ebb8e52c9cd8b9fbbdbb87c/095c2a458ebb8e52c9cd8b9fbbdbb87c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347585.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/161/739fcb3c30c343a0dd6823bb2df35161/739fcb3c30c343a0dd6823bb2df35161.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347589.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a24/fab32b692102d3e957fc1168ff900a24/fab32b692102d3e957fc1168ff900a24.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347589.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a24/fab32b692102d3e957fc1168ff900a24/fab32b692102d3e957fc1168ff900a24.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347593.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/720/c23b69e1d2afa0452f2d4e8fa937c720/c23b69e1d2afa0452f2d4e8fa937c720.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347597.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/226/f247c22fbde8a35f6c4875b96b96d226/f247c22fbde8a35f6c4875b96b96d226.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347601.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/603/8c1d0fb6cd18823fe540266f482db603/8c1d0fb6cd18823fe540266f482db603.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347605.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/34a/99fed0ed4ce2b49a1153aa26d07fb34a/99fed0ed4ce2b49a1153aa26d07fb34a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347609.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/28b/b8d31f99f9d6684cfeed40ee3333028b/b8d31f99f9d6684cfeed40ee3333028b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347613.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/352/01b3bfbd138b2731a8632674be74c352/01b3bfbd138b2731a8632674be74c352.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347617.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/390/2ca63143b2b2619074d115ab9293a390/2ca63143b2b2619074d115ab9293a390.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347621.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ec5/d1e5ed1cb9b3ae4c20faa5558fbefec5/d1e5ed1cb9b3ae4c20faa5558fbefec5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347621.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ec5/d1e5ed1cb9b3ae4c20faa5558fbefec5/d1e5ed1cb9b3ae4c20faa5558fbefec5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347625.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/77f/aa1d6813e07cc32fc435d0608019277f/aa1d6813e07cc32fc435d0608019277f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347629.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/b13/345694a9442a1bd74f1a40afa9d9eb13/345694a9442a1bd74f1a40afa9d9eb13.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347633.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/43f/e7c0a3f7875fe9f46ce483c9a218243f/e7c0a3f7875fe9f46ce483c9a218243f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347637.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/198/ba445ae05d8528ca1c86b70710f6e198/ba445ae05d8528ca1c86b70710f6e198.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347649.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/e97/ef1513c6b007745b4778c09476f14e97/ef1513c6b007745b4778c09476f14e97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347653.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/93c/b578bb8c02a6767686b0a3a38dd6093c/b578bb8c02a6767686b0a3a38dd6093c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347653.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/93c/b578bb8c02a6767686b0a3a38dd6093c/b578bb8c02a6767686b0a3a38dd6093c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347657.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/e71/8d9ecda0d9168e958c4727b47f3d7e71/8d9ecda0d9168e958c4727b47f3d7e71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347661.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/fa9/2ee0011b7836ef50a6548f0f5439cfa9/2ee0011b7836ef50a6548f0f5439cfa9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347665.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/2dc/56231bb56535d7ae0dd86961366c52dc/56231bb56535d7ae0dd86961366c52dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347669.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/453/41b0108c741007ad07bbd693ed2cc453/41b0108c741007ad07bbd693ed2cc453.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347673.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/88f/c3a5850f5044123cd775611c63b9d88f/c3a5850f5044123cd775611c63b9d88f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347677.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/3f9/2ac69e37d2e96def5a2124e7c7de13f9/2ac69e37d2e96def5a2124e7c7de13f9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347685.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/70f/2e05998d4d14388abcf50dba1aaf370f/2e05998d4d14388abcf50dba1aaf370f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347685.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/70f/2e05998d4d14388abcf50dba1aaf370f/2e05998d4d14388abcf50dba1aaf370f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347689.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/1b7/92333d18cb1d33a67fb6eedc305511b7/92333d18cb1d33a67fb6eedc305511b7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347693.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a97/5c0ecabe8bb529f8452437dd5213ca97/5c0ecabe8bb529f8452437dd5213ca97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347697.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/567/ced821bae367c43ca18395e218ab6567/ced821bae367c43ca18395e218ab6567.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347701.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/340/338f10fab22b80566a33c54f7707b340/338f10fab22b80566a33c54f7707b340.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347709.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5e9/5efcd162ea1d20aee6db38d3d26545e9/5efcd162ea1d20aee6db38d3d26545e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347713.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/0b7/45cc01c8618b257d706bf3f755bb20b7/45cc01c8618b257d706bf3f755bb20b7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347717.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a45/b15814e0ee6abb76e58d1fada8263a45/b15814e0ee6abb76e58d1fada8263a45.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347717.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a45/b15814e0ee6abb76e58d1fada8263a45/b15814e0ee6abb76e58d1fada8263a45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347721.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/30b/a840879dd01db1abe26a5be62c2b730b/a840879dd01db1abe26a5be62c2b730b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347725.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/9f3/4d5c42343ad62e7bfb8d38e83e6349f3/4d5c42343ad62e7bfb8d38e83e6349f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347729.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/dab/d2186b63fdc704a51a4604fbfaa9bdab/d2186b63fdc704a51a4604fbfaa9bdab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347733.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/525/28b6ba0c3e5c7cdf6fd952292d2c8525/28b6ba0c3e5c7cdf6fd952292d2c8525.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347737.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ba6/15b26149b51d73b7ccbfe4607f405ba6/15b26149b51d73b7ccbfe4607f405ba6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347745.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/dda/27a9ca1c7ae225c426acfa89d211fdda/27a9ca1c7ae225c426acfa89d211fdda.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347749.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/955/ab056e3351fc4d981138fbfc76044955/ab056e3351fc4d981138fbfc76044955.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347749.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/955/ab056e3351fc4d981138fbfc76044955/ab056e3351fc4d981138fbfc76044955.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347753.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/e18/3c452bf7dc2198444c3b6de75365ae18/3c452bf7dc2198444c3b6de75365ae18.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347757.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/0ea/08cb921c21020117fed1eea6610550ea/08cb921c21020117fed1eea6610550ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347761.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/0c3/4b2285ee40dca96fe129bd70f95380c3/4b2285ee40dca96fe129bd70f95380c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347769.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/65e/80a57d17469abfbc80764e851746265e/80a57d17469abfbc80764e851746265e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347773.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/889/049bbfbe6ef786bb43f66b123b90f889/049bbfbe6ef786bb43f66b123b90f889.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347777.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/aa4/0ac2721c681eddec28c20770fdc98aa4/0ac2721c681eddec28c20770fdc98aa4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347781.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ca0/6759e6311b4637ebee8520056518fca0/6759e6311b4637ebee8520056518fca0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347781.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ca0/6759e6311b4637ebee8520056518fca0/6759e6311b4637ebee8520056518fca0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347785.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5aa/a948b8c8c554e6f77656406f38f145aa/a948b8c8c554e6f77656406f38f145aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347789.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/8ae/d0229f4a42f0273dff20f749815978ae/d0229f4a42f0273dff20f749815978ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347793.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/eb2/f8507ac5db3577d577391f3dc44f0eb2/f8507ac5db3577d577391f3dc44f0eb2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347797.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/0a4/5a804dd3697dd22d69cfebae0f32e0a4/5a804dd3697dd22d69cfebae0f32e0a4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347801.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/f97/83e0c884ad3cccd2eeb6c7628ad72f97/83e0c884ad3cccd2eeb6c7628ad72f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347805.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/957/bd01253df32bee6088b3c10b2c231957/bd01253df32bee6088b3c10b2c231957.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347809.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/c28/9b4391a2d139a98471a3e1773dc2ec28/9b4391a2d139a98471a3e1773dc2ec28.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347813.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a05/0a79a4a64428d91e5053ac2723e9fa05/0a79a4a64428d91e5053ac2723e9fa05.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347813.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a05/0a79a4a64428d91e5053ac2723e9fa05/0a79a4a64428d91e5053ac2723e9fa05.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347817.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/d6b/7f013d5dcb0478fd40074186bcc7ed6b/7f013d5dcb0478fd40074186bcc7ed6b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347821.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5e1/5c5cd5d35895fceb0a365e99a32285e1/5c5cd5d35895fceb0a365e99a32285e1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347825.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/8de/73f855b7cb3477d481a0c22e6505a8de/73f855b7cb3477d481a0c22e6505a8de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347829.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/6eb/c68b041bc57c6f58da08f70e120936eb/c68b041bc57c6f58da08f70e120936eb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347833.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/b3e/23cea69d772c2a9903f486490ec0fb3e/23cea69d772c2a9903f486490ec0fb3e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347837.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/31d/2bd1f9ff214a6fb8b0ef316e5971331d/2bd1f9ff214a6fb8b0ef316e5971331d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347841.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5c3/3b58f7e1fdada2f37f5bc1d682bc15c3/3b58f7e1fdada2f37f5bc1d682bc15c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347845.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/4c7/bee591a6cdc9e5f3a62c798dfe91e4c7/bee591a6cdc9e5f3a62c798dfe91e4c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347849.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/c7a/840057809bbc18997beb236ec2e4ec7a/840057809bbc18997beb236ec2e4ec7a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347853.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/7ea/a238f7ad04397dcca305f0cf3b55a7ea/a238f7ad04397dcca305f0cf3b55a7ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347857.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/116/b81426e614b30071f1c067e28b2d6116/b81426e614b30071f1c067e28b2d6116.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347861.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/64b/9fb673b94d6407e7932278e8e1acc64b/9fb673b94d6407e7932278e8e1acc64b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347865.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/083/0c3c57f47368c3acbf8f2c59c55bd083/0c3c57f47368c3acbf8f2c59c55bd083.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347869.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/b1c/a4f22c0e7ae5221d3071ea2fbb7deb1c/a4f22c0e7ae5221d3071ea2fbb7deb1c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347873.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/da6/cd5f0e7827e76e6129b1cc529b3c2da6/cd5f0e7827e76e6129b1cc529b3c2da6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347877.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/71a/0dc717888962b37261af24ebf05c271a/0dc717888962b37261af24ebf05c271a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347881.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/61b/2f3aec0313542588c84e9f2dfc09e61b/2f3aec0313542588c84e9f2dfc09e61b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347885.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ccb/4eda514f33b17a9f427637c5b7392ccb/4eda514f33b17a9f427637c5b7392ccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347889.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a4b/7c764bf589a5579292e7d19658442a4b/7c764bf589a5579292e7d19658442a4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347893.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/9b2/96102c59344e720a262fee0da86669b2/96102c59344e720a262fee0da86669b2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347897.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/79f/c999334612db849c3166f5bbf549a79f/c999334612db849c3166f5bbf549a79f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347901.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/131/dcf7cfdeb95325c6bd5dba5eaeb06131/dcf7cfdeb95325c6bd5dba5eaeb06131.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347905.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a27/0f882f1c2c943e0a369d3e5eb5cb3a27/0f882f1c2c943e0a369d3e5eb5cb3a27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347909.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/853/332c8b81fc58c26ed7eebf18a75bc853/332c8b81fc58c26ed7eebf18a75bc853.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347913.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a54/86524c18893bf5f2dd7773e2bddcda54/86524c18893bf5f2dd7773e2bddcda54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347917.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/d5b/bd5f9b8e51caf4be614fbce3a0d9bd5b/bd5f9b8e51caf4be614fbce3a0d9bd5b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347921.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/adc/8ab0e947c4b4e8ad7b9cd32da9874adc/8ab0e947c4b4e8ad7b9cd32da9874adc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347925.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/3c8/dadd317843955d4a8078835095dc53c8/dadd317843955d4a8078835095dc53c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347929.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/1d0/795f249a7d3aac79b88b59c13d9f41d0/795f249a7d3aac79b88b59c13d9f41d0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347933.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/265/3e9eff0b4b0061d71b01835d5d10b265/3e9eff0b4b0061d71b01835d5d10b265.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347937.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/fbc/5586db5b3d153e79d75556e43c8dcfbc/5586db5b3d153e79d75556e43c8dcfbc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347941.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/4fb/edac15b85b7c7a507807b52cf0f844fb/edac15b85b7c7a507807b52cf0f844fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347945.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/044/a33b6292a60d0f796e2cb49059b9e044/a33b6292a60d0f796e2cb49059b9e044.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347949.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/1f8/69b7a2188f049e87174805be325611f8/69b7a2188f049e87174805be325611f8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347953.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5b3/88d6fd70cd031a39adf96f86e17b85b3/88d6fd70cd031a39adf96f86e17b85b3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347957.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/c84/e36081f2a3cd7e221d0627c909e50c84/e36081f2a3cd7e221d0627c909e50c84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347961.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/267/d6a5f2131e549f18bb5b242406490267/d6a5f2131e549f18bb5b242406490267.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347965.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/cb7/fd0e749512ecb60cee0051ccf8d3fcb7/fd0e749512ecb60cee0051ccf8d3fcb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347969.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a83/f430773fca40ecc3321e376e0c81da83/f430773fca40ecc3321e376e0c81da83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347973.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5ec/a0e4afafa98a4e57240d87519e2195ec/a0e4afafa98a4e57240d87519e2195ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347977.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/719/329c130f5e939b884c89e0eb83628719/329c130f5e939b884c89e0eb83628719.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347981.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/81b/4e945cb38c3ebbc1ee6fd98ed6b6281b/4e945cb38c3ebbc1ee6fd98ed6b6281b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347985.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/f10/1b00a5c88b65373d5e637161327b2f10/1b00a5c88b65373d5e637161327b2f10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347989.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/918/b56fc246fb08ea2e0fb97509a8808918/b56fc246fb08ea2e0fb97509a8808918.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347993.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/620/6f1f83423dec2e5cd5e0b6e672680620/6f1f83423dec2e5cd5e0b6e672680620.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347997.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/99b/f862d3ab695db47b2b9cf2a6a9f4e99b/f862d3ab695db47b2b9cf2a6a9f4e99b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348001.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/f33/5116e0ccec346a3dc6d0aaddd04b6f33/5116e0ccec346a3dc6d0aaddd04b6f33.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348005.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/912/595904597e1790fbdfd9e32790dc6912/595904597e1790fbdfd9e32790dc6912.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348009.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/391/8d7a30a9822c8705c83fce6b962b2391/8d7a30a9822c8705c83fce6b962b2391.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348013.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/795/8d0a2a4d26bd883068b5d2b1eff6e795/8d0a2a4d26bd883068b5d2b1eff6e795.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348017.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/2e2/bad8028e20f09a9a00456d08914c72e2/bad8028e20f09a9a00456d08914c72e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348021.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5b9/dfa7c8da860bfa45974ae43fd7c8d5b9/dfa7c8da860bfa45974ae43fd7c8d5b9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348025.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/5ee/d2b979a99eec3622d4c66df3bc6ed5ee/d2b979a99eec3622d4c66df3bc6ed5ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348029.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/fef/87d66db4473c14e724ebc51bb7835fef/87d66db4473c14e724ebc51bb7835fef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348033.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/917/7077d863537b96d099e13162c9671917/7077d863537b96d099e13162c9671917.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348037.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/459/818bfd55e55680c721872b57f7d04459/818bfd55e55680c721872b57f7d04459.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348041.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/add/ad8b005ff9a10fe82fc94bf386225add/ad8b005ff9a10fe82fc94bf386225add.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348045.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/49a/2f0b29f01bda657a965865bafd44249a/2f0b29f01bda657a965865bafd44249a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348049.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/a9c/5f5e5f0315b9831059ec04d531c24a9c/5f5e5f0315b9831059ec04d531c24a9c.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_cleaving_not_merged_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_fetch_fails_while_shrinking ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348053.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/ccd/b3d80091bf69ea0a4796d9769df1accd/b3d80091bf69ea0a4796d9769df1accd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348057.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/d89/9f23653fc3c37315c088d8d25e3c2d89/9f23653fc3c37315c088d8d25e3c2d89.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348061.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/089/4eb071be17b3731b7bcd20315fb19089/4eb071be17b3731b7bcd20315fb19089.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348065.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/710/7fc15467f16b6c761e8b49d881f2f710/7fc15467f16b6c761e8b49d881f2f710.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348069.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/c0e/364470450047654239b25d476d77fc0e/364470450047654239b25d476d77fc0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348073.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/9fb/8fd67d6943c30a8bccb8f79ddf6609fb/8fd67d6943c30a8bccb8f79ddf6609fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348077.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/30c/91dfb6041cbef861d4999d767322b30c/91dfb6041cbef861d4999d767322b30c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348081.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/8cc/f0afae1cd204d19f6f07e2161f9d48cc/f0afae1cd204d19f6f07e2161f9d48cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348085.00000-0, db: /tmp/tmpnp_k119k/sda/containers/0/958/9c71fc26ef1336fa8d1b46f2660a5958/9c71fc26ef1336fa8d1b46f2660a5958.db sharder-test WARNING: Failed to get shard ranges from a/c: bad, path: .shards_a/shard_c, db: /tmp/tmprzgh1v1v/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347580.00000.db sharder-test WARNING: Audit warnings for shard: unable to get shard ranges from root, path: .shards_a/shard_c, db: /tmp/tmprzgh1v1v/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347580.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347583.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/f22/ee6ab8bc025e8ba0946a428867430f22/ee6ab8bc025e8ba0946a428867430f22_1717347583.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347583.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/f22/ee6ab8bc025e8ba0946a428867430f22/ee6ab8bc025e8ba0946a428867430f22_1717347583.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347586.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ab0/5c4022110f825609da641f495808dab0/5c4022110f825609da641f495808dab0_1717347586.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347586.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ab0/5c4022110f825609da641f495808dab0/5c4022110f825609da641f495808dab0_1717347586.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347589.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/a24/fab32b692102d3e957fc1168ff900a24/fab32b692102d3e957fc1168ff900a24_1717347589.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347589.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/a24/fab32b692102d3e957fc1168ff900a24/fab32b692102d3e957fc1168ff900a24_1717347589.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347592.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/9b0/99a535e40c925c8c09f57ce8f52479b0/99a535e40c925c8c09f57ce8f52479b0_1717347592.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347592.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/9b0/99a535e40c925c8c09f57ce8f52479b0/99a535e40c925c8c09f57ce8f52479b0_1717347592.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347595.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/927/cfb45dd245e2aafe51f405678f957927/cfb45dd245e2aafe51f405678f957927_1717347595.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347595.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/927/cfb45dd245e2aafe51f405678f957927/cfb45dd245e2aafe51f405678f957927_1717347595.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347598.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/463/eed9d24fda58d3b865a1b7b889005463/eed9d24fda58d3b865a1b7b889005463_1717347598.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347598.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/463/eed9d24fda58d3b865a1b7b889005463/eed9d24fda58d3b865a1b7b889005463_1717347598.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347601.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/603/8c1d0fb6cd18823fe540266f482db603/8c1d0fb6cd18823fe540266f482db603_1717347601.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347601.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/603/8c1d0fb6cd18823fe540266f482db603/8c1d0fb6cd18823fe540266f482db603_1717347601.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347604.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/3ba/c797fe14a96763a0daa5cfe761f623ba/c797fe14a96763a0daa5cfe761f623ba_1717347604.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347604.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/3ba/c797fe14a96763a0daa5cfe761f623ba/c797fe14a96763a0daa5cfe761f623ba_1717347604.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347607.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ccc/b8d05aee727bb6f6a1aeacb6fb3ccccc/b8d05aee727bb6f6a1aeacb6fb3ccccc_1717347607.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347607.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ccc/b8d05aee727bb6f6a1aeacb6fb3ccccc/b8d05aee727bb6f6a1aeacb6fb3ccccc_1717347607.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347610.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/363/a7f6204dd0f22ac8597f755aeed90363/a7f6204dd0f22ac8597f755aeed90363_1717347610.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347610.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/363/a7f6204dd0f22ac8597f755aeed90363/a7f6204dd0f22ac8597f755aeed90363_1717347610.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347613.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/352/01b3bfbd138b2731a8632674be74c352/01b3bfbd138b2731a8632674be74c352_1717347613.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347613.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/352/01b3bfbd138b2731a8632674be74c352/01b3bfbd138b2731a8632674be74c352_1717347613.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822_1717347616.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822_1717347616.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347619.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8_1717347619.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347619.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8/76c4b2ad2cbdb3d3726dbf73f2754ff8_1717347619.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347622.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/17e/bf0929fa3af2563e8204e8715a42d17e/bf0929fa3af2563e8204e8715a42d17e_1717347622.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347622.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/17e/bf0929fa3af2563e8204e8715a42d17e/bf0929fa3af2563e8204e8715a42d17e_1717347622.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347625.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/77f/aa1d6813e07cc32fc435d0608019277f/aa1d6813e07cc32fc435d0608019277f_1717347625.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347625.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/77f/aa1d6813e07cc32fc435d0608019277f/aa1d6813e07cc32fc435d0608019277f_1717347625.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347628.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/6ec/d1fcdcdcd0525c610d9af7acdaa856ec/d1fcdcdcd0525c610d9af7acdaa856ec_1717347628.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347628.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/6ec/d1fcdcdcd0525c610d9af7acdaa856ec/d1fcdcdcd0525c610d9af7acdaa856ec_1717347628.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347631.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/823/272092cf742dfbeef04f71672e31d823/272092cf742dfbeef04f71672e31d823_1717347631.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347631.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/823/272092cf742dfbeef04f71672e31d823/272092cf742dfbeef04f71672e31d823_1717347631.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347634.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/8eb/09167dd7c49ded2cf2c282eaf034d8eb/09167dd7c49ded2cf2c282eaf034d8eb_1717347634.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347634.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/8eb/09167dd7c49ded2cf2c282eaf034d8eb/09167dd7c49ded2cf2c282eaf034d8eb_1717347634.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347637.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/198/ba445ae05d8528ca1c86b70710f6e198/ba445ae05d8528ca1c86b70710f6e198_1717347637.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347637.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/198/ba445ae05d8528ca1c86b70710f6e198/ba445ae05d8528ca1c86b70710f6e198_1717347637.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c_1717347640.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c_1717347640.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347643.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/edc/4d7ad7abd294928b3ea16b25c810cedc/4d7ad7abd294928b3ea16b25c810cedc_1717347643.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347643.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/edc/4d7ad7abd294928b3ea16b25c810cedc/4d7ad7abd294928b3ea16b25c810cedc_1717347643.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf_1717347646.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf_1717347646.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347649.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/e97/ef1513c6b007745b4778c09476f14e97/ef1513c6b007745b4778c09476f14e97_1717347649.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347649.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/e97/ef1513c6b007745b4778c09476f14e97/ef1513c6b007745b4778c09476f14e97_1717347649.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347652.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/a67/ba2680550578d85f47449360c6495a67/ba2680550578d85f47449360c6495a67_1717347652.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347652.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/a67/ba2680550578d85f47449360c6495a67/ba2680550578d85f47449360c6495a67_1717347652.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347655.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/b50/6af70ffeca5372ca4cd9d87b84614b50/6af70ffeca5372ca4cd9d87b84614b50_1717347655.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347655.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/b50/6af70ffeca5372ca4cd9d87b84614b50/6af70ffeca5372ca4cd9d87b84614b50_1717347655.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347658.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/34d/29871b709e3938202bb7974cebfb534d/29871b709e3938202bb7974cebfb534d_1717347658.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347658.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/34d/29871b709e3938202bb7974cebfb534d/29871b709e3938202bb7974cebfb534d_1717347658.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347661.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/fa9/2ee0011b7836ef50a6548f0f5439cfa9/2ee0011b7836ef50a6548f0f5439cfa9_1717347661.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347661.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/fa9/2ee0011b7836ef50a6548f0f5439cfa9/2ee0011b7836ef50a6548f0f5439cfa9_1717347661.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347664.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/393/4bbf27b69b8057524d6e994b2e2d3393/4bbf27b69b8057524d6e994b2e2d3393_1717347664.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347664.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/393/4bbf27b69b8057524d6e994b2e2d3393/4bbf27b69b8057524d6e994b2e2d3393_1717347664.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347667.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/160/d5d4b98b5cade45bbaa678858ce60160/d5d4b98b5cade45bbaa678858ce60160_1717347667.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347667.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/160/d5d4b98b5cade45bbaa678858ce60160/d5d4b98b5cade45bbaa678858ce60160_1717347667.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347670.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/35b/fcd26c3abb10b783d89d36fdf44e135b/fcd26c3abb10b783d89d36fdf44e135b_1717347670.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347670.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/35b/fcd26c3abb10b783d89d36fdf44e135b/fcd26c3abb10b783d89d36fdf44e135b_1717347670.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347673.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/88f/c3a5850f5044123cd775611c63b9d88f/c3a5850f5044123cd775611c63b9d88f_1717347673.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347673.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/88f/c3a5850f5044123cd775611c63b9d88f/c3a5850f5044123cd775611c63b9d88f_1717347673.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77_1717347676.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77_1717347676.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347679.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/77f/410c026e68360671f267262b98eb577f/410c026e68360671f267262b98eb577f_1717347679.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347679.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/77f/410c026e68360671f267262b98eb577f/410c026e68360671f267262b98eb577f_1717347679.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347682.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/06d/a3ca461ceaacfb06214d27a1fe29106d/a3ca461ceaacfb06214d27a1fe29106d_1717347682.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347682.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/06d/a3ca461ceaacfb06214d27a1fe29106d/a3ca461ceaacfb06214d27a1fe29106d_1717347682.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347685.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/70f/2e05998d4d14388abcf50dba1aaf370f/2e05998d4d14388abcf50dba1aaf370f_1717347685.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347685.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/70f/2e05998d4d14388abcf50dba1aaf370f/2e05998d4d14388abcf50dba1aaf370f_1717347685.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347688.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/47e/3d36a1f8bc39a691e323f1c76675047e/3d36a1f8bc39a691e323f1c76675047e_1717347688.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347688.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/47e/3d36a1f8bc39a691e323f1c76675047e/3d36a1f8bc39a691e323f1c76675047e_1717347688.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347691.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/223/9562307612d5aeab22c4e29141429223/9562307612d5aeab22c4e29141429223_1717347691.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347691.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/223/9562307612d5aeab22c4e29141429223/9562307612d5aeab22c4e29141429223_1717347691.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347694.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/1b9/ef9b6c055f332026c60c7481e5cef1b9/ef9b6c055f332026c60c7481e5cef1b9_1717347694.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347694.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/1b9/ef9b6c055f332026c60c7481e5cef1b9/ef9b6c055f332026c60c7481e5cef1b9_1717347694.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347697.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/567/ced821bae367c43ca18395e218ab6567/ced821bae367c43ca18395e218ab6567_1717347697.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347697.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/567/ced821bae367c43ca18395e218ab6567/ced821bae367c43ca18395e218ab6567_1717347697.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2_1717347700.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2_1717347700.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347703.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/f69/34e69c6bbffc71c1a2b38d6be943df69/34e69c6bbffc71c1a2b38d6be943df69_1717347703.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347703.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/f69/34e69c6bbffc71c1a2b38d6be943df69/34e69c6bbffc71c1a2b38d6be943df69_1717347703.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347706.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/250/390b9a2e91110c85bc4826e989ad2250/390b9a2e91110c85bc4826e989ad2250_1717347706.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347706.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/250/390b9a2e91110c85bc4826e989ad2250/390b9a2e91110c85bc4826e989ad2250_1717347706.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347709.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/5e9/5efcd162ea1d20aee6db38d3d26545e9/5efcd162ea1d20aee6db38d3d26545e9_1717347709.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347709.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/5e9/5efcd162ea1d20aee6db38d3d26545e9/5efcd162ea1d20aee6db38d3d26545e9_1717347709.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347712.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/a17/b086ebc18f0c64e71cc2cd0e4c282a17/b086ebc18f0c64e71cc2cd0e4c282a17_1717347712.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347712.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/a17/b086ebc18f0c64e71cc2cd0e4c282a17/b086ebc18f0c64e71cc2cd0e4c282a17_1717347712.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347715.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/aec/d611b19ffc806aa25c87a098e4465aec/d611b19ffc806aa25c87a098e4465aec_1717347715.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347715.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/aec/d611b19ffc806aa25c87a098e4465aec/d611b19ffc806aa25c87a098e4465aec_1717347715.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347718.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/841/bda60b2d29541a1a238679542593e841/bda60b2d29541a1a238679542593e841_1717347718.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347718.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/841/bda60b2d29541a1a238679542593e841/bda60b2d29541a1a238679542593e841_1717347718.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347721.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/30b/a840879dd01db1abe26a5be62c2b730b/a840879dd01db1abe26a5be62c2b730b_1717347721.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347721.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/30b/a840879dd01db1abe26a5be62c2b730b/a840879dd01db1abe26a5be62c2b730b_1717347721.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347724.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/5fb/09149bd74e1a4c2bb86e1343145d25fb/09149bd74e1a4c2bb86e1343145d25fb_1717347724.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347724.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/5fb/09149bd74e1a4c2bb86e1343145d25fb/09149bd74e1a4c2bb86e1343145d25fb_1717347724.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347727.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/e53/cc24599acc5a47d31b6df3f260012e53/cc24599acc5a47d31b6df3f260012e53_1717347727.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347727.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/e53/cc24599acc5a47d31b6df3f260012e53/cc24599acc5a47d31b6df3f260012e53_1717347727.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347730.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/c84/a1c5cdafd5f74fb95eb78d4e96445c84/a1c5cdafd5f74fb95eb78d4e96445c84_1717347730.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347730.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/c84/a1c5cdafd5f74fb95eb78d4e96445c84/a1c5cdafd5f74fb95eb78d4e96445c84_1717347730.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347733.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/525/28b6ba0c3e5c7cdf6fd952292d2c8525/28b6ba0c3e5c7cdf6fd952292d2c8525_1717347733.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347733.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/525/28b6ba0c3e5c7cdf6fd952292d2c8525/28b6ba0c3e5c7cdf6fd952292d2c8525_1717347733.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e_1717347736.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e_1717347736.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347739.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/d9a/dcbc0ccc8d597a609dde9e69d4d98d9a/dcbc0ccc8d597a609dde9e69d4d98d9a_1717347739.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347739.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/d9a/dcbc0ccc8d597a609dde9e69d4d98d9a/dcbc0ccc8d597a609dde9e69d4d98d9a_1717347739.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347742.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/15a/bff72dea412bc33a289af8308a11515a/bff72dea412bc33a289af8308a11515a_1717347742.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347742.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/15a/bff72dea412bc33a289af8308a11515a/bff72dea412bc33a289af8308a11515a_1717347742.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347745.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/dda/27a9ca1c7ae225c426acfa89d211fdda/27a9ca1c7ae225c426acfa89d211fdda_1717347745.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347745.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/dda/27a9ca1c7ae225c426acfa89d211fdda/27a9ca1c7ae225c426acfa89d211fdda_1717347745.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347748.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/d26/18305906879ad68520323d0c64288d26/18305906879ad68520323d0c64288d26_1717347748.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347748.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/d26/18305906879ad68520323d0c64288d26/18305906879ad68520323d0c64288d26_1717347748.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347751.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/b8b/ef2502503030e69f7858112638ac2b8b/ef2502503030e69f7858112638ac2b8b_1717347751.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347751.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/b8b/ef2502503030e69f7858112638ac2b8b/ef2502503030e69f7858112638ac2b8b_1717347751.00000.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_sharding ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347754.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/2ce/4865290b94ae30f9c26697c74691a2ce/4865290b94ae30f9c26697c74691a2ce_1717347754.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347754.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/2ce/4865290b94ae30f9c26697c74691a2ce/4865290b94ae30f9c26697c74691a2ce_1717347754.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347757.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/0ea/08cb921c21020117fed1eea6610550ea/08cb921c21020117fed1eea6610550ea_1717347757.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347757.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/0ea/08cb921c21020117fed1eea6610550ea/08cb921c21020117fed1eea6610550ea_1717347757.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9_1717347760.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9_1717347760.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347763.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/92e/daf3f1bca7d6aeb84a143ae7e087c92e/daf3f1bca7d6aeb84a143ae7e087c92e_1717347763.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347763.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/92e/daf3f1bca7d6aeb84a143ae7e087c92e/daf3f1bca7d6aeb84a143ae7e087c92e_1717347763.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347766.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/c45/e6438e2ceb444065cbbbb631c9be9c45/e6438e2ceb444065cbbbb631c9be9c45_1717347766.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347766.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/c45/e6438e2ceb444065cbbbb631c9be9c45/e6438e2ceb444065cbbbb631c9be9c45_1717347766.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347769.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/65e/80a57d17469abfbc80764e851746265e/80a57d17469abfbc80764e851746265e_1717347769.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347769.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/65e/80a57d17469abfbc80764e851746265e/80a57d17469abfbc80764e851746265e_1717347769.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347772.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/320/21e0eccc354bf736c324ba3edcd56320/21e0eccc354bf736c324ba3edcd56320_1717347772.00000.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347772.00000-0, db: /tmp/tmpsbkccazb/sda/containers/0/320/21e0eccc354bf736c324ba3edcd56320/21e0eccc354bf736c324ba3edcd56320_1717347772.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347600.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/02c/c963245de37fe34ad4083a0687d2702c/c963245de37fe34ad4083a0687d2702c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347600.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/02c/c963245de37fe34ad4083a0687d2702c/c963245de37fe34ad4083a0687d2702c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347604.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/3ba/c797fe14a96763a0daa5cfe761f623ba/c797fe14a96763a0daa5cfe761f623ba.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347604.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/3ba/c797fe14a96763a0daa5cfe761f623ba/c797fe14a96763a0daa5cfe761f623ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347608.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/cb5/94096e279e8022b3c12b658150de5cb5/94096e279e8022b3c12b658150de5cb5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347608.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/cb5/94096e279e8022b3c12b658150de5cb5/94096e279e8022b3c12b658150de5cb5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347612.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/00b/db4fa3e7f8c43ad63e7bd93837a1200b/db4fa3e7f8c43ad63e7bd93837a1200b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347612.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/00b/db4fa3e7f8c43ad63e7bd93837a1200b/db4fa3e7f8c43ad63e7bd93837a1200b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347620.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/058/5cdb99d2d569d9da3bc419ff1dfc6058/5cdb99d2d569d9da3bc419ff1dfc6058.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347620.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/058/5cdb99d2d569d9da3bc419ff1dfc6058/5cdb99d2d569d9da3bc419ff1dfc6058.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347624.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/497/91f469d461ed1b40629e3b651f7ff497/91f469d461ed1b40629e3b651f7ff497.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347624.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/497/91f469d461ed1b40629e3b651f7ff497/91f469d461ed1b40629e3b651f7ff497.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347628.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/6ec/d1fcdcdcd0525c610d9af7acdaa856ec/d1fcdcdcd0525c610d9af7acdaa856ec.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347628.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/6ec/d1fcdcdcd0525c610d9af7acdaa856ec/d1fcdcdcd0525c610d9af7acdaa856ec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347632.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/088/2fbebd125f1e26ddb4a8d15fdc820088/2fbebd125f1e26ddb4a8d15fdc820088.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347632.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/088/2fbebd125f1e26ddb4a8d15fdc820088/2fbebd125f1e26ddb4a8d15fdc820088.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347652.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a67/ba2680550578d85f47449360c6495a67/ba2680550578d85f47449360c6495a67.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347652.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a67/ba2680550578d85f47449360c6495a67/ba2680550578d85f47449360c6495a67.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347656.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/b0e/0d131d949dc99f78dddc400126474b0e/0d131d949dc99f78dddc400126474b0e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347656.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/b0e/0d131d949dc99f78dddc400126474b0e/0d131d949dc99f78dddc400126474b0e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347664.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/393/4bbf27b69b8057524d6e994b2e2d3393/4bbf27b69b8057524d6e994b2e2d3393.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347664.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/393/4bbf27b69b8057524d6e994b2e2d3393/4bbf27b69b8057524d6e994b2e2d3393.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347668.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/092/b97e0ac82d3b45e45d84ab82ea3eb092/b97e0ac82d3b45e45d84ab82ea3eb092.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347668.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/092/b97e0ac82d3b45e45d84ab82ea3eb092/b97e0ac82d3b45e45d84ab82ea3eb092.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347680.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/8bb/aae77f028aa9a9780ee8fc5297c6c8bb/aae77f028aa9a9780ee8fc5297c6c8bb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347680.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/8bb/aae77f028aa9a9780ee8fc5297c6c8bb/aae77f028aa9a9780ee8fc5297c6c8bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347688.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/47e/3d36a1f8bc39a691e323f1c76675047e/3d36a1f8bc39a691e323f1c76675047e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347688.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/47e/3d36a1f8bc39a691e323f1c76675047e/3d36a1f8bc39a691e323f1c76675047e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347692.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2cc/cc14e786fd504b53d4d0a8be32fc12cc/cc14e786fd504b53d4d0a8be32fc12cc.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347692.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2cc/cc14e786fd504b53d4d0a8be32fc12cc/cc14e786fd504b53d4d0a8be32fc12cc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347704.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/cab/d4b698071714b8df8dafd2afbd822cab/d4b698071714b8df8dafd2afbd822cab.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347704.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/cab/d4b698071714b8df8dafd2afbd822cab/d4b698071714b8df8dafd2afbd822cab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347712.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a17/b086ebc18f0c64e71cc2cd0e4c282a17/b086ebc18f0c64e71cc2cd0e4c282a17.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347712.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a17/b086ebc18f0c64e71cc2cd0e4c282a17/b086ebc18f0c64e71cc2cd0e4c282a17.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347716.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ef8/2bd31ca5e5fb2da1a764945d5aaeeef8/2bd31ca5e5fb2da1a764945d5aaeeef8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347716.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ef8/2bd31ca5e5fb2da1a764945d5aaeeef8/2bd31ca5e5fb2da1a764945d5aaeeef8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347724.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/5fb/09149bd74e1a4c2bb86e1343145d25fb/09149bd74e1a4c2bb86e1343145d25fb.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347724.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/5fb/09149bd74e1a4c2bb86e1343145d25fb/09149bd74e1a4c2bb86e1343145d25fb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347728.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/7d6/1899a69b54a05dd498c0310f9e00e7d6/1899a69b54a05dd498c0310f9e00e7d6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347728.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/7d6/1899a69b54a05dd498c0310f9e00e7d6/1899a69b54a05dd498c0310f9e00e7d6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347740.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/45c/9c8ec32ccf58393b11dc3cd4e0e7e45c/9c8ec32ccf58393b11dc3cd4e0e7e45c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347740.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/45c/9c8ec32ccf58393b11dc3cd4e0e7e45c/9c8ec32ccf58393b11dc3cd4e0e7e45c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347748.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/d26/18305906879ad68520323d0c64288d26/18305906879ad68520323d0c64288d26.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347748.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/d26/18305906879ad68520323d0c64288d26/18305906879ad68520323d0c64288d26.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347752.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/fa7/ef8e62c7146b163ea34986579107dfa7/ef8e62c7146b163ea34986579107dfa7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347752.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/fa7/ef8e62c7146b163ea34986579107dfa7/ef8e62c7146b163ea34986579107dfa7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347764.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ee0/03f57669aa76c0435d6085b558941ee0/03f57669aa76c0435d6085b558941ee0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347764.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ee0/03f57669aa76c0435d6085b558941ee0/03f57669aa76c0435d6085b558941ee0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347772.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/320/21e0eccc354bf736c324ba3edcd56320/21e0eccc354bf736c324ba3edcd56320.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347772.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/320/21e0eccc354bf736c324ba3edcd56320/21e0eccc354bf736c324ba3edcd56320.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347776.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a49/76c0b29cc74d17206aace3a14e682a49/76c0b29cc74d17206aace3a14e682a49.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347776.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a49/76c0b29cc74d17206aace3a14e682a49/76c0b29cc74d17206aace3a14e682a49.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347784.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/273/7645cfa2cb0275a5639231cc10720273/7645cfa2cb0275a5639231cc10720273.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347784.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/273/7645cfa2cb0275a5639231cc10720273/7645cfa2cb0275a5639231cc10720273.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347788.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/087/b7316f7a20743476f017f05b78a8c087/b7316f7a20743476f017f05b78a8c087.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347788.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/087/b7316f7a20743476f017f05b78a8c087/b7316f7a20743476f017f05b78a8c087.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347792.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e41/d1936b417b2226818d22874cab17de41/d1936b417b2226818d22874cab17de41.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347792.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e41/d1936b417b2226818d22874cab17de41/d1936b417b2226818d22874cab17de41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347796.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2e9/b24ef43711af2c751d4669c3d515d2e9/b24ef43711af2c751d4669c3d515d2e9.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347796.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2e9/b24ef43711af2c751d4669c3d515d2e9/b24ef43711af2c751d4669c3d515d2e9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347800.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/d60/29c4e4c38b2a9b66d6dc06a3a31c5d60/29c4e4c38b2a9b66d6dc06a3a31c5d60.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347800.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/d60/29c4e4c38b2a9b66d6dc06a3a31c5d60/29c4e4c38b2a9b66d6dc06a3a31c5d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347804.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/699/9320cfa5738742e281472b44ad44d699/9320cfa5738742e281472b44ad44d699.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347804.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/699/9320cfa5738742e281472b44ad44d699/9320cfa5738742e281472b44ad44d699.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347808.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/66f/e3ce65f8f58e896cb52cce184f61566f/e3ce65f8f58e896cb52cce184f61566f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347808.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/66f/e3ce65f8f58e896cb52cce184f61566f/e3ce65f8f58e896cb52cce184f61566f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347812.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a9f/745ed25c6ff5d22e968e252f502b2a9f/745ed25c6ff5d22e968e252f502b2a9f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347812.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/a9f/745ed25c6ff5d22e968e252f502b2a9f/745ed25c6ff5d22e968e252f502b2a9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347816.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/48c/d67b765547bbc5013b2795518c7a348c/d67b765547bbc5013b2795518c7a348c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347816.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/48c/d67b765547bbc5013b2795518c7a348c/d67b765547bbc5013b2795518c7a348c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347820.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/5fd/006fcabd0221977584acf3840c2005fd/006fcabd0221977584acf3840c2005fd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347820.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/5fd/006fcabd0221977584acf3840c2005fd/006fcabd0221977584acf3840c2005fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347824.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/29f/bcead454d54b4a183d519cb02152529f/bcead454d54b4a183d519cb02152529f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347824.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/29f/bcead454d54b4a183d519cb02152529f/bcead454d54b4a183d519cb02152529f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347828.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2a5/14ee8038b1be82bea712800683ac72a5/14ee8038b1be82bea712800683ac72a5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347828.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/2a5/14ee8038b1be82bea712800683ac72a5/14ee8038b1be82bea712800683ac72a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347832.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e85/409c279027f476ab688ec24a5e847e85/409c279027f476ab688ec24a5e847e85.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_merge_while_unsharded ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347832.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e85/409c279027f476ab688ec24a5e847e85/409c279027f476ab688ec24a5e847e85.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347836.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e5c/44d385caacc6153ade9eca4f23cfde5c/44d385caacc6153ade9eca4f23cfde5c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347836.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/e5c/44d385caacc6153ade9eca4f23cfde5c/44d385caacc6153ade9eca4f23cfde5c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347840.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ba7/1be2ce69ea2bd1dc593afb7bb33fdba7/1be2ce69ea2bd1dc593afb7bb33fdba7.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347840.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ba7/1be2ce69ea2bd1dc593afb7bb33fdba7/1be2ce69ea2bd1dc593afb7bb33fdba7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347844.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ce4/041213edcf15454f4f918910b38bfce4/041213edcf15454f4f918910b38bfce4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347844.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/ce4/041213edcf15454f4f918910b38bfce4/041213edcf15454f4f918910b38bfce4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347848.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/4ff/3469ebe48b8b4a0b9da43b25d9ef64ff/3469ebe48b8b4a0b9da43b25d9ef64ff.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347848.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/4ff/3469ebe48b8b4a0b9da43b25d9ef64ff/3469ebe48b8b4a0b9da43b25d9ef64ff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347852.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/9c8/f6b92390955db34ccc10e7236b6d79c8/f6b92390955db34ccc10e7236b6d79c8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347852.00000-0, db: /tmp/tmpedq1lnyw/sda/containers/0/9c8/f6b92390955db34ccc10e7236b6d79c8/f6b92390955db34ccc10e7236b6d79c8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347603.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/400/6fb2dffc3e38f009cd93bfa020eed400/6fb2dffc3e38f009cd93bfa020eed400.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347603.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/400/6fb2dffc3e38f009cd93bfa020eed400/6fb2dffc3e38f009cd93bfa020eed400.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347606.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/d62/0bc6cfc11cda2d7d9ee4de6a7e320d62/0bc6cfc11cda2d7d9ee4de6a7e320d62.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347606.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/d62/0bc6cfc11cda2d7d9ee4de6a7e320d62/0bc6cfc11cda2d7d9ee4de6a7e320d62.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347609.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/28b/b8d31f99f9d6684cfeed40ee3333028b/b8d31f99f9d6684cfeed40ee3333028b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347609.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/28b/b8d31f99f9d6684cfeed40ee3333028b/b8d31f99f9d6684cfeed40ee3333028b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347612.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/00b/db4fa3e7f8c43ad63e7bd93837a1200b/db4fa3e7f8c43ad63e7bd93837a1200b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347612.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/00b/db4fa3e7f8c43ad63e7bd93837a1200b/db4fa3e7f8c43ad63e7bd93837a1200b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347615.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/eee/615f85f1d7a071cefbd4654669b7beee/615f85f1d7a071cefbd4654669b7beee.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347615.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/eee/615f85f1d7a071cefbd4654669b7beee/615f85f1d7a071cefbd4654669b7beee.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347618.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/646/a22158a1a918b78348b8276a22abf646/a22158a1a918b78348b8276a22abf646.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347618.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/646/a22158a1a918b78348b8276a22abf646/a22158a1a918b78348b8276a22abf646.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347621.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/ec5/d1e5ed1cb9b3ae4c20faa5558fbefec5/d1e5ed1cb9b3ae4c20faa5558fbefec5.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347621.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/ec5/d1e5ed1cb9b3ae4c20faa5558fbefec5/d1e5ed1cb9b3ae4c20faa5558fbefec5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347624.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/497/91f469d461ed1b40629e3b651f7ff497/91f469d461ed1b40629e3b651f7ff497.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347624.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/497/91f469d461ed1b40629e3b651f7ff497/91f469d461ed1b40629e3b651f7ff497.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347627.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/7e6/8f813e3ffa71e500721ac7a7309197e6/8f813e3ffa71e500721ac7a7309197e6.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347627.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/7e6/8f813e3ffa71e500721ac7a7309197e6/8f813e3ffa71e500721ac7a7309197e6.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347630.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/006/adc08935a25d855056391774cf81a006/adc08935a25d855056391774cf81a006.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347630.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/006/adc08935a25d855056391774cf81a006/adc08935a25d855056391774cf81a006.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347633.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/43f/e7c0a3f7875fe9f46ce483c9a218243f/e7c0a3f7875fe9f46ce483c9a218243f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347633.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/43f/e7c0a3f7875fe9f46ce483c9a218243f/e7c0a3f7875fe9f46ce483c9a218243f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347639.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/ea1/1f99b787458c5c33f5803b31cdb8bea1/1f99b787458c5c33f5803b31cdb8bea1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347639.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/ea1/1f99b787458c5c33f5803b31cdb8bea1/1f99b787458c5c33f5803b31cdb8bea1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347654.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/dbd/097740ddacdce3d4e2421474224c0dbd/097740ddacdce3d4e2421474224c0dbd.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347654.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/dbd/097740ddacdce3d4e2421474224c0dbd/097740ddacdce3d4e2421474224c0dbd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347657.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e71/8d9ecda0d9168e958c4727b47f3d7e71/8d9ecda0d9168e958c4727b47f3d7e71.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347657.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e71/8d9ecda0d9168e958c4727b47f3d7e71/8d9ecda0d9168e958c4727b47f3d7e71.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347663.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/b16/374952945e317b566e0211f2168adb16/374952945e317b566e0211f2168adb16.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347663.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/b16/374952945e317b566e0211f2168adb16/374952945e317b566e0211f2168adb16.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347666.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/b01/ded215a90f8af0bd03a459a3039a7b01/ded215a90f8af0bd03a459a3039a7b01.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347666.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/b01/ded215a90f8af0bd03a459a3039a7b01/ded215a90f8af0bd03a459a3039a7b01.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347669.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/453/41b0108c741007ad07bbd693ed2cc453/41b0108c741007ad07bbd693ed2cc453.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347669.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/453/41b0108c741007ad07bbd693ed2cc453/41b0108c741007ad07bbd693ed2cc453.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347678.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/67c/1289950a2088fb5fd8792eff1064d67c/1289950a2088fb5fd8792eff1064d67c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347678.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/67c/1289950a2088fb5fd8792eff1064d67c/1289950a2088fb5fd8792eff1064d67c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347687.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/7a5/54dc622594c35edea8f581205cbbf7a5/54dc622594c35edea8f581205cbbf7a5.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347687.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/7a5/54dc622594c35edea8f581205cbbf7a5/54dc622594c35edea8f581205cbbf7a5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347690.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/354/f379bcf67d17b83e8612a2b17eb0c354/f379bcf67d17b83e8612a2b17eb0c354.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347690.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/354/f379bcf67d17b83e8612a2b17eb0c354/f379bcf67d17b83e8612a2b17eb0c354.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347693.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a97/5c0ecabe8bb529f8452437dd5213ca97/5c0ecabe8bb529f8452437dd5213ca97.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347693.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a97/5c0ecabe8bb529f8452437dd5213ca97/5c0ecabe8bb529f8452437dd5213ca97.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347699.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/192/fdbab859519c6bc89c2614bd151e6192/fdbab859519c6bc89c2614bd151e6192.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347699.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/192/fdbab859519c6bc89c2614bd151e6192/fdbab859519c6bc89c2614bd151e6192.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347702.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/988/7c549a494f0bfebe80a7ed84753f6988/7c549a494f0bfebe80a7ed84753f6988.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347702.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/988/7c549a494f0bfebe80a7ed84753f6988/7c549a494f0bfebe80a7ed84753f6988.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347711.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/6df/57dd647dd6d1de8ae7123608ff9736df/57dd647dd6d1de8ae7123608ff9736df.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347711.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/6df/57dd647dd6d1de8ae7123608ff9736df/57dd647dd6d1de8ae7123608ff9736df.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347714.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/44e/25c6c380f4c0a13b36131a3e1202844e/25c6c380f4c0a13b36131a3e1202844e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347714.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/44e/25c6c380f4c0a13b36131a3e1202844e/25c6c380f4c0a13b36131a3e1202844e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347717.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a45/b15814e0ee6abb76e58d1fada8263a45/b15814e0ee6abb76e58d1fada8263a45.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347717.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a45/b15814e0ee6abb76e58d1fada8263a45/b15814e0ee6abb76e58d1fada8263a45.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347723.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/02e/c931380aa2221677ef56675beb80602e/c931380aa2221677ef56675beb80602e.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347723.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/02e/c931380aa2221677ef56675beb80602e/c931380aa2221677ef56675beb80602e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347726.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/919/2d3fccec5449ced10bab5e792b82d919/2d3fccec5449ced10bab5e792b82d919.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347726.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/919/2d3fccec5449ced10bab5e792b82d919/2d3fccec5449ced10bab5e792b82d919.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347729.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/dab/d2186b63fdc704a51a4604fbfaa9bdab/d2186b63fdc704a51a4604fbfaa9bdab.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_missing_own_merged_while_shrinking ... ok sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347729.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/dab/d2186b63fdc704a51a4604fbfaa9bdab/d2186b63fdc704a51a4604fbfaa9bdab.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347735.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/b6f/66be2eb4b3cf41d35caa1c96a69b1b6f/66be2eb4b3cf41d35caa1c96a69b1b6f.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347735.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/b6f/66be2eb4b3cf41d35caa1c96a69b1b6f/66be2eb4b3cf41d35caa1c96a69b1b6f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347738.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a91/bc595ce8fa6c7086403ec8c34ff5ba91/bc595ce8fa6c7086403ec8c34ff5ba91.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347738.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/a91/bc595ce8fa6c7086403ec8c34ff5ba91/bc595ce8fa6c7086403ec8c34ff5ba91.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347747.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/51c/4f33ed770ca99d94ef786d8ba55af51c/4f33ed770ca99d94ef786d8ba55af51c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347747.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/51c/4f33ed770ca99d94ef786d8ba55af51c/4f33ed770ca99d94ef786d8ba55af51c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347750.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/c40/3533c4527a93b20163c3e3ad474d2c40/3533c4527a93b20163c3e3ad474d2c40.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347750.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/c40/3533c4527a93b20163c3e3ad474d2c40/3533c4527a93b20163c3e3ad474d2c40.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347753.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e18/3c452bf7dc2198444c3b6de75365ae18/3c452bf7dc2198444c3b6de75365ae18.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347753.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/e18/3c452bf7dc2198444c3b6de75365ae18/3c452bf7dc2198444c3b6de75365ae18.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347759.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/10c/87c64fc016d99651aa51f01c4447a10c/87c64fc016d99651aa51f01c4447a10c.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347759.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/10c/87c64fc016d99651aa51f01c4447a10c/87c64fc016d99651aa51f01c4447a10c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347762.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/f50/60bb6f35de82226a6ab635b227d46f50/60bb6f35de82226a6ab635b227d46f50.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347762.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/f50/60bb6f35de82226a6ab635b227d46f50/60bb6f35de82226a6ab635b227d46f50.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db sharder-test WARNING: Audit warnings for shard: root has no matching shard range, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmpe8l4khid/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347605.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/34a/99fed0ed4ce2b49a1153aa26d07fb34a/99fed0ed4ce2b49a1153aa26d07fb34a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347610.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/363/a7f6204dd0f22ac8597f755aeed90363/a7f6204dd0f22ac8597f755aeed90363.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347615.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/eee/615f85f1d7a071cefbd4654669b7beee/615f85f1d7a071cefbd4654669b7beee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347620.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/058/5cdb99d2d569d9da3bc419ff1dfc6058/5cdb99d2d569d9da3bc419ff1dfc6058.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347625.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/77f/aa1d6813e07cc32fc435d0608019277f/aa1d6813e07cc32fc435d0608019277f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347630.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/006/adc08935a25d855056391774cf81a006/adc08935a25d855056391774cf81a006.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347635.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/bc2/846d5e69d8b9bf77558085c3f5cc0bc2/846d5e69d8b9bf77558085c3f5cc0bc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347650.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/939/a65f6f7661617c4312bcbbcf34c63939/a65f6f7661617c4312bcbbcf34c63939.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347655.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b50/6af70ffeca5372ca4cd9d87b84614b50/6af70ffeca5372ca4cd9d87b84614b50.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347665.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/2dc/56231bb56535d7ae0dd86961366c52dc/56231bb56535d7ae0dd86961366c52dc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347670.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/35b/fcd26c3abb10b783d89d36fdf44e135b/fcd26c3abb10b783d89d36fdf44e135b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347680.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/8bb/aae77f028aa9a9780ee8fc5297c6c8bb/aae77f028aa9a9780ee8fc5297c6c8bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347685.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/70f/2e05998d4d14388abcf50dba1aaf370f/2e05998d4d14388abcf50dba1aaf370f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347690.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/354/f379bcf67d17b83e8612a2b17eb0c354/f379bcf67d17b83e8612a2b17eb0c354.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347695.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/7fc/960d0403382d4f530771e01fb325c7fc/960d0403382d4f530771e01fb325c7fc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347710.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b73/a440a74f1454af0ddb6aa057909f0b73/a440a74f1454af0ddb6aa057909f0b73.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347715.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/aec/d611b19ffc806aa25c87a098e4465aec/d611b19ffc806aa25c87a098e4465aec.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347725.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9f3/4d5c42343ad62e7bfb8d38e83e6349f3/4d5c42343ad62e7bfb8d38e83e6349f3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347730.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c84/a1c5cdafd5f74fb95eb78d4e96445c84/a1c5cdafd5f74fb95eb78d4e96445c84.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347735.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b6f/66be2eb4b3cf41d35caa1c96a69b1b6f/66be2eb4b3cf41d35caa1c96a69b1b6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347740.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/45c/9c8ec32ccf58393b11dc3cd4e0e7e45c/9c8ec32ccf58393b11dc3cd4e0e7e45c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347745.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/dda/27a9ca1c7ae225c426acfa89d211fdda/27a9ca1c7ae225c426acfa89d211fdda.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347750.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c40/3533c4527a93b20163c3e3ad474d2c40/3533c4527a93b20163c3e3ad474d2c40.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347755.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f97/a845b58185a0bf9de204f95f5d076f97/a845b58185a0bf9de204f95f5d076f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347770.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/96c/54f948a1165aaf139573e9079fa8b96c/54f948a1165aaf139573e9079fa8b96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347775.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/57e/c71a3503c848b63d82c2502f0583457e/c71a3503c848b63d82c2502f0583457e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347785.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/5aa/a948b8c8c554e6f77656406f38f145aa/a948b8c8c554e6f77656406f38f145aa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347790.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/965/c55e12e6ea051eec2ce7e93485fa8965/c55e12e6ea051eec2ce7e93485fa8965.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347795.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d9b/050560945ef5bbab351b2b51d557ad9b/050560945ef5bbab351b2b51d557ad9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347800.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d60/29c4e4c38b2a9b66d6dc06a3a31c5d60/29c4e4c38b2a9b66d6dc06a3a31c5d60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347805.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/957/bd01253df32bee6088b3c10b2c231957/bd01253df32bee6088b3c10b2c231957.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347810.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e2f/c575f6bb053862175e0b67f38ba58e2f/c575f6bb053862175e0b67f38ba58e2f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347815.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/1d5/f0e6aeb96f0c36c9151e7628306601d5/f0e6aeb96f0c36c9151e7628306601d5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347820.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/5fd/006fcabd0221977584acf3840c2005fd/006fcabd0221977584acf3840c2005fd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347825.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/8de/73f855b7cb3477d481a0c22e6505a8de/73f855b7cb3477d481a0c22e6505a8de.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347830.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c6a/e0e3afd71788c838aad06292c0db6c6a/e0e3afd71788c838aad06292c0db6c6a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347835.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/992/011065492122d68ab4286cefef06f992/011065492122d68ab4286cefef06f992.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347840.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ba7/1be2ce69ea2bd1dc593afb7bb33fdba7/1be2ce69ea2bd1dc593afb7bb33fdba7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347845.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/4c7/bee591a6cdc9e5f3a62c798dfe91e4c7/bee591a6cdc9e5f3a62c798dfe91e4c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347850.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b83/5aca788f2fbfa8c9c9f2e4d7bf817b83/5aca788f2fbfa8c9c9f2e4d7bf817b83.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347855.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d1a/01313479a199746172fd553e96c37d1a/01313479a199746172fd553e96c37d1a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347860.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/1f1/525911717c691d9742a0d63954d751f1/525911717c691d9742a0d63954d751f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347865.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/083/0c3c57f47368c3acbf8f2c59c55bd083/0c3c57f47368c3acbf8f2c59c55bd083.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347870.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/691/a39a45f99fb6ad1281550b1c36ed7691/a39a45f99fb6ad1281550b1c36ed7691.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347875.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d97/381cbda8f21b520fd6fa8af128774d97/381cbda8f21b520fd6fa8af128774d97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347880.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b3e/670079b1c2086bc65d0a8c9c0e61fb3e/670079b1c2086bc65d0a8c9c0e61fb3e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347885.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ccb/4eda514f33b17a9f427637c5b7392ccb/4eda514f33b17a9f427637c5b7392ccb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347890.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/eae/d24ceadcb229e1c8ccaa073a62934eae/d24ceadcb229e1c8ccaa073a62934eae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347895.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/8a7/474dd4fa2d6f8c8a5c8595d1be2a18a7/474dd4fa2d6f8c8a5c8595d1be2a18a7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347900.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/233/afd45040e81a45be47f71516659dc233/afd45040e81a45be47f71516659dc233.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347905.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/a27/0f882f1c2c943e0a369d3e5eb5cb3a27/0f882f1c2c943e0a369d3e5eb5cb3a27.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347910.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/192/7ba0c40a17511607b306399c5e5f0192/7ba0c40a17511607b306399c5e5f0192.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347915.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d1d/a7a4505712d9f26b6a929a3e753f4d1d/a7a4505712d9f26b6a929a3e753f4d1d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347920.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/3fe/580e9c5d3741cad4261f2df0237143fe/580e9c5d3741cad4261f2df0237143fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347925.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/3c8/dadd317843955d4a8078835095dc53c8/dadd317843955d4a8078835095dc53c8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347930.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/250/77556b341f9024a065dc6d7c4237f250/77556b341f9024a065dc6d7c4237f250.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347935.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e31/37dea06684b0588036a2d1fb21310e31/37dea06684b0588036a2d1fb21310e31.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347940.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/eef/3854af72dd9d24247430f1e7621dbeef/3854af72dd9d24247430f1e7621dbeef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347945.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/044/a33b6292a60d0f796e2cb49059b9e044/a33b6292a60d0f796e2cb49059b9e044.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347950.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/29a/477acbcddd059e7fad03b1cc3cf8429a/477acbcddd059e7fad03b1cc3cf8429a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347955.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/149/0606b83a54a6eff2c6bd118aba5fe149/0606b83a54a6eff2c6bd118aba5fe149.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347960.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/96c/1b9ff9b0c59e9547a4750f0d431bb96c/1b9ff9b0c59e9547a4750f0d431bb96c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347965.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/cb7/fd0e749512ecb60cee0051ccf8d3fcb7/fd0e749512ecb60cee0051ccf8d3fcb7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347970.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/256/a5519f5f0d0cb84115cc3b6c8a191256/a5519f5f0d0cb84115cc3b6c8a191256.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347975.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d02/0d6ea296208ee085c870b76a8f3cbd02/0d6ea296208ee085c870b76a8f3cbd02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347980.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f97/2a80ffebc7c69a504f975770c6693f97/2a80ffebc7c69a504f975770c6693f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347985.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f10/1b00a5c88b65373d5e637161327b2f10/1b00a5c88b65373d5e637161327b2f10.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347990.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/31b/2376b644dbfe9ba7fd100ae89204c31b/2376b644dbfe9ba7fd100ae89204c31b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347995.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/db1/3d73127ef8d2da0bbc046caeecae3db1/3d73127ef8d2da0bbc046caeecae3db1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348000.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ba5/b598bcb80f3103b28c49f79766e04ba5/b598bcb80f3103b28c49f79766e04ba5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348005.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/912/595904597e1790fbdfd9e32790dc6912/595904597e1790fbdfd9e32790dc6912.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348010.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/34b/0d8f3b88d3cfe5cbd2aeaad2b5aa834b/0d8f3b88d3cfe5cbd2aeaad2b5aa834b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348015.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/dd1/8aca460283f0bb4f76698c0e35ba3dd1/8aca460283f0bb4f76698c0e35ba3dd1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348020.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9ce/a425f11f2290689b2f5878d7cfe5e9ce/a425f11f2290689b2f5878d7cfe5e9ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348025.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/5ee/d2b979a99eec3622d4c66df3bc6ed5ee/d2b979a99eec3622d4c66df3bc6ed5ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348030.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/a2a/b48ba937fa34b94b3829e71beb61da2a/b48ba937fa34b94b3829e71beb61da2a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348035.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fa1/02e3cf74f37ce0e83b0aa44e8acc5fa1/02e3cf74f37ce0e83b0aa44e8acc5fa1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348040.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f97/2a2166a062677ec01259d644dab03f97/2a2166a062677ec01259d644dab03f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348045.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/49a/2f0b29f01bda657a965865bafd44249a/2f0b29f01bda657a965865bafd44249a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348050.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ea2/94fbd703423313426921aff051b5fea2/94fbd703423313426921aff051b5fea2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348055.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ea7/fc171f72480fd73453e103c463a8eea7/fc171f72480fd73453e103c463a8eea7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348060.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b7d/bc7cfaf8392671e45668c28e4202fb7d/bc7cfaf8392671e45668c28e4202fb7d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348065.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/710/7fc15467f16b6c761e8b49d881f2f710/7fc15467f16b6c761e8b49d881f2f710.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348070.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9f1/1722d9f4636e014c07aabaa20b82a9f1/1722d9f4636e014c07aabaa20b82a9f1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348075.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/11a/48d0037f06bce7151eea6da51d27b11a/48d0037f06bce7151eea6da51d27b11a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348080.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/1ca/32658bb760b2094363eccf14bb1ef1ca/32658bb760b2094363eccf14bb1ef1ca.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348085.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/958/9c71fc26ef1336fa8d1b46f2660a5958/9c71fc26ef1336fa8d1b46f2660a5958.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348090.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/1ad/cf5efb62101cfaad6c008a01af77c1ad/cf5efb62101cfaad6c008a01af77c1ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348095.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/11e/6e9363bde772f4e5af2c91f10691b11e/6e9363bde772f4e5af2c91f10691b11e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348100.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e50/16c14f0b3d21999f1e1378d49943ce50/16c14f0b3d21999f1e1378d49943ce50.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348105.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/2a9/52c958e2b95fda3f78ea5771c335b2a9/52c958e2b95fda3f78ea5771c335b2a9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348110.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/bcd/7d466eac11c27e27b2321ad4eb291bcd/7d466eac11c27e27b2321ad4eb291bcd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348115.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/3bb/61579d0ddb3167076c2a5ce91a6df3bb/61579d0ddb3167076c2a5ce91a6df3bb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348120.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/984/ce3bf1de18e53a815a6efebe6da41984/ce3bf1de18e53a815a6efebe6da41984.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348125.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/dc2/de9b7a20006c5694d6ca75f65a52adc2/de9b7a20006c5694d6ca75f65a52adc2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348130.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/cba/1fede09e63384a435cfd9ef3ea3c9cba/1fede09e63384a435cfd9ef3ea3c9cba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348135.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fea/84f9202a504c9a0b81f505bb00fb0fea/84f9202a504c9a0b81f505bb00fb0fea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348140.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9bd/af9286ec9c837fdebdc64b369f38d9bd/af9286ec9c837fdebdc64b369f38d9bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348145.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/6ba/d138ea09bfed69872e1b973dbcf566ba/d138ea09bfed69872e1b973dbcf566ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348150.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9e2/42c08c17d37df6696e31d2aba92f19e2/42c08c17d37df6696e31d2aba92f19e2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348155.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f28/526a19766b268d93e739292f1fa2ff28/526a19766b268d93e739292f1fa2ff28.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348160.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/763/8da02c52ce3916899e377db52f0b7763/8da02c52ce3916899e377db52f0b7763.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348165.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/91f/1307f57b4fc06a773c92e26bfeedf91f/1307f57b4fc06a773c92e26bfeedf91f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348170.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/77e/adad2e594313509933ecb58c142bd77e/adad2e594313509933ecb58c142bd77e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348175.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/734/6bd8627252ffca7ca4080c3996ca5734/6bd8627252ffca7ca4080c3996ca5734.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348180.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/4ce/40b92ace61f6a035a00ad75c8505a4ce/40b92ace61f6a035a00ad75c8505a4ce.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348185.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/eed/6d612780c9230166851a6ed7ef51feed/6d612780c9230166851a6ed7ef51feed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348190.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/4fe/0a1382ba0e0bc02835dbed3b68a494fe/0a1382ba0e0bc02835dbed3b68a494fe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348195.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/381/3792c9b2a749e4e955adbe5e20cc6381/3792c9b2a749e4e955adbe5e20cc6381.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348200.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/223/20f945c49eeb9156a91118f8f84b1223/20f945c49eeb9156a91118f8f84b1223.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348205.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/690/e71a1d3a6f49b2368310ef49c1803690/e71a1d3a6f49b2368310ef49c1803690.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348210.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c65/1f115b7058374289c9ae905f7b5f7c65/1f115b7058374289c9ae905f7b5f7c65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348215.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/120/20a27009a9346a8915c6202603f62120/20a27009a9346a8915c6202603f62120.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348220.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/675/b26440bb50e7d0e3245478962fefe675/b26440bb50e7d0e3245478962fefe675.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348225.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/384/0934c95d5a78fdc42930ee6e3d677384/0934c95d5a78fdc42930ee6e3d677384.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348230.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ccf/9dfbc58b7aafeb8800b972daaaa12ccf/9dfbc58b7aafeb8800b972daaaa12ccf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348235.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/37a/08f19c05663d8c08adea4ce5fa63e37a/08f19c05663d8c08adea4ce5fa63e37a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348240.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/3a8/db9e2f57ff2134686bcdea79528083a8/db9e2f57ff2134686bcdea79528083a8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348245.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/8ed/e425beb7100568d77f593318dc1948ed/e425beb7100568d77f593318dc1948ed.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348250.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/05c/c27264c23dfdfc32e126812b5434305c/c27264c23dfdfc32e126812b5434305c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348255.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/002/434a0e0967310f955ad2727c79f3d002/434a0e0967310f955ad2727c79f3d002.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348260.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/966/be887be5d893ab70b836c94d770cb966/be887be5d893ab70b836c94d770cb966.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348265.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/efe/34cbb3484023489bbdb7c6e1bbb01efe/34cbb3484023489bbdb7c6e1bbb01efe.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348270.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/5fa/7a658eee0c95553dfb98ba5d88c0d5fa/7a658eee0c95553dfb98ba5d88c0d5fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348275.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/482/dc3fc77d2644dea12dd9a51779489482/dc3fc77d2644dea12dd9a51779489482.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348280.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e00/271e1216e64e9f867453f84c350c1e00/271e1216e64e9f867453f84c350c1e00.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348285.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/402/fe9e4d9268a518f70f30191ec69a4402/fe9e4d9268a518f70f30191ec69a4402.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348290.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fff/a2f7950eb02f2f5efd37098fe2da0fff/a2f7950eb02f2f5efd37098fe2da0fff.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348295.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fbb/5ecea0974c7dbc77f7113841db3bcfbb/5ecea0974c7dbc77f7113841db3bcfbb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348300.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d3c/b766b321ae4bc28ebc5d917c38164d3c/b766b321ae4bc28ebc5d917c38164d3c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348305.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e51/9df200507457cb65d18353b537ba6e51/9df200507457cb65d18353b537ba6e51.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348310.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f4f/42e3c1dbd83f277538ef815a54febf4f/42e3c1dbd83f277538ef815a54febf4f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348315.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/209/9a643e7f79a0f1df5930b289a22c3209/9a643e7f79a0f1df5930b289a22c3209.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348320.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/997/95b6ce24d3c65f9a1cd529ee6c503997/95b6ce24d3c65f9a1cd529ee6c503997.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348325.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/486/228c259ca2244b437095ce27f8147486/228c259ca2244b437095ce27f8147486.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348330.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/6ba/3da3ac137dc47c729415bf31e6cca6ba/3da3ac137dc47c729415bf31e6cca6ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348335.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d59/d7d6c6e8ff3754b4ab79a6d4decbdd59/d7d6c6e8ff3754b4ab79a6d4decbdd59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348340.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/2a7/e5e3e64c532bce2ace3a91d1713482a7/e5e3e64c532bce2ace3a91d1713482a7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348345.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b05/a77a8db3a798c72f319b47d89f9adb05/a77a8db3a798c72f319b47d89f9adb05.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348350.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c31/68c0c991d4380831728d65366a820c31/68c0c991d4380831728d65366a820c31.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348355.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/19f/9456567126af49e6910011715cbcf19f/9456567126af49e6910011715cbcf19f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348360.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/bd3/3ca31264c6219b18b0e1ed365913dbd3/3ca31264c6219b18b0e1ed365913dbd3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348365.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/03b/fbb81aef876b36d1a1d3c10c882c903b/fbb81aef876b36d1a1d3c10c882c903b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348370.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/16b/56137a1cb3bbcc190a0800a1afdba16b/56137a1cb3bbcc190a0800a1afdba16b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348375.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/3bc/95262600647a76561dd6c1a8eb3163bc/95262600647a76561dd6c1a8eb3163bc.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348380.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b75/610c64ca24fb70a45dac042f27ce5b75/610c64ca24fb70a45dac042f27ce5b75.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348385.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/6ee/48e19fcc5565af486e287105c4eb46ee/48e19fcc5565af486e287105c4eb46ee.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348390.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/7a6/e2fe4734903a5e9a75f9712d3c5d97a6/e2fe4734903a5e9a75f9712d3c5d97a6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348395.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c19/4876ee85117ada8520fadf6e7a196c19/4876ee85117ada8520fadf6e7a196c19.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348400.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/91f/5416c8a52d221b1fd896399024da991f/5416c8a52d221b1fd896399024da991f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348405.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ee2/dc935c37a5f162ff6ea559e989b31ee2/dc935c37a5f162ff6ea559e989b31ee2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348410.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/52c/c6f129c5a152df97360693457a24952c/c6f129c5a152df97360693457a24952c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348415.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f94/43e1e952c853b20ebdf0c319cbb2af94/43e1e952c853b20ebdf0c319cbb2af94.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348420.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/0ba/248fe19803695fd9d1de01a085bea0ba/248fe19803695fd9d1de01a085bea0ba.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348425.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/604/be776072afcc8f2b46bed52167d52604/be776072afcc8f2b46bed52167d52604.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348430.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/364/5ab4394ac26ef6903b236be1ad563364/5ab4394ac26ef6903b236be1ad563364.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348435.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/18d/af6334a2d7504ed955c3d5379573b18d/af6334a2d7504ed955c3d5379573b18d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348440.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e62/ab6405b9791b6c7d9bd3f613af9b3e62/ab6405b9791b6c7d9bd3f613af9b3e62.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348445.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fe2/ccfbca61b13f2cade63e000c8567dfe2/ccfbca61b13f2cade63e000c8567dfe2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348450.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/336/5a3ee7b9f73a0b696e58f731e5cb7336/5a3ee7b9f73a0b696e58f731e5cb7336.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348455.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/7bd/d16950d0bcbc09a0e7047cff092d37bd/d16950d0bcbc09a0e7047cff092d37bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348460.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/a02/38627e463622b2ab85682029e5408a02/38627e463622b2ab85682029e5408a02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348465.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c57/0b022911f76049e94b3b6e2a916a7c57/0b022911f76049e94b3b6e2a916a7c57.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348470.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/241/51b52d05863c42542eddbbf9e4da1241/51b52d05863c42542eddbbf9e4da1241.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348475.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/853/90d96cb85f4ac123cc95f71575f4e853/90d96cb85f4ac123cc95f71575f4e853.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348480.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f60/554b638b8590cbc9b07a6fef081b8f60/554b638b8590cbc9b07a6fef081b8f60.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348485.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/2c2/fc884f55dcccf60b50e84656095972c2/fc884f55dcccf60b50e84656095972c2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348490.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fb2/b1e52d98f196b13e69bfea07afbc7fb2/b1e52d98f196b13e69bfea07afbc7fb2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348495.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/536/79d9bb12faedb7010d613d0555f6c536/79d9bb12faedb7010d613d0555f6c536.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348500.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fe3/fdd891acfa799a04886bcbdd022a4fe3/fdd891acfa799a04886bcbdd022a4fe3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348505.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e4b/16448043b77b6de15c09fdb0fd0a7e4b/16448043b77b6de15c09fdb0fd0a7e4b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348510.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f03/fbc3c2071e41733dadda0c1f7cffff03/fbc3c2071e41733dadda0c1f7cffff03.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348515.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/4ea/fcc6d4872c7c48b5d7350ad6b5d114ea/fcc6d4872c7c48b5d7350ad6b5d114ea.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348520.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/42c/ec37b602416e36604fd77b410216542c/ec37b602416e36604fd77b410216542c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348525.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/94e/4fff2656d95c093683b65e3db482894e/4fff2656d95c093683b65e3db482894e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348530.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b0a/f37fdf17d2606364f7dc5deb05094b0a/f37fdf17d2606364f7dc5deb05094b0a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348535.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/903/e050ea15fc8416cd4ae74ab6831e4903/e050ea15fc8416cd4ae74ab6831e4903.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348540.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/19e/5bf73b1e190f535adcf051e3a422419e/5bf73b1e190f535adcf051e3a422419e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348545.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/4b8/19c07aa0510b1835822c3c86e14994b8/19c07aa0510b1835822c3c86e14994b8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348550.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/6f6/7ab0fe2433331ad862ebc0a9123416f6/7ab0fe2433331ad862ebc0a9123416f6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348555.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d01/10863ab32a7c96b86d7c05fb15e81d01/10863ab32a7c96b86d7c05fb15e81d01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348560.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b16/f7d9bb22f1f2ffd0bf3a30158d064b16/f7d9bb22f1f2ffd0bf3a30158d064b16.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348565.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b45/a346476cc3c22dd655ca0115951a3b45/a346476cc3c22dd655ca0115951a3b45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348570.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/154/c46139c5ab113abe6618d6f349bdb154/c46139c5ab113abe6618d6f349bdb154.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348575.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/eb1/6643be26330dff914d200e3c2c381eb1/6643be26330dff914d200e3c2c381eb1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348580.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/5d9/ca048f445b3fa820476ccbb8e2ae15d9/ca048f445b3fa820476ccbb8e2ae15d9.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348585.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/441/93022679b6a84ead2bb1f1d57e437441/93022679b6a84ead2bb1f1d57e437441.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348590.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/681/78cec865b6e7cc2409a8070a5a16d681/78cec865b6e7cc2409a8070a5a16d681.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348595.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/34d/bbe5240d11e9ea43b14ae2ce51ce034d/bbe5240d11e9ea43b14ae2ce51ce034d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348600.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c04/562b7721b50a8f2fb91633b6855bec04/562b7721b50a8f2fb91633b6855bec04.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348605.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f4d/c2e12ed78dfc9379613108d21af18f4d/c2e12ed78dfc9379613108d21af18f4d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348610.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/af4/d213edd2fe5bb9ad715154b1a6e6caf4/d213edd2fe5bb9ad715154b1a6e6caf4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348615.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/74b/d0809f071782ba8f52463789f8ac874b/d0809f071782ba8f52463789f8ac874b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348620.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/324/50cf98cb92f1f3ddab2a5927afbd8324/50cf98cb92f1f3ddab2a5927afbd8324.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348625.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d9f/2001df02a9203b2e0dc3003ea5b22d9f/2001df02a9203b2e0dc3003ea5b22d9f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348630.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9fa/656402c044a4950009b5f15a521119fa/656402c044a4950009b5f15a521119fa.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348635.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/fc6/70ed22e7ba784091bf9f44b57f4f6fc6/70ed22e7ba784091bf9f44b57f4f6fc6.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348640.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/3d3/92616908a5063620cff78c68c36313d3/92616908a5063620cff78c68c36313d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348645.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d39/032e5fc18015ae034e3525a7958a4d39/032e5fc18015ae034e3525a7958a4d39.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348650.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b80/7e35283a9e978b1e874b7f9ec38a0b80/7e35283a9e978b1e874b7f9ec38a0b80.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348655.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/074/1dcc897f4c98b73bb87ff2871e6fb074/1dcc897f4c98b73bb87ff2871e6fb074.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348660.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b99/fccb2ed5fe6bc269310cae9d1e37bb99/fccb2ed5fe6bc269310cae9d1e37bb99.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348665.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/622/3ae169413303727e56a377bb581a7622/3ae169413303727e56a377bb581a7622.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348670.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/57a/95e9515ebbbe808902a56b2349c7f57a/95e9515ebbbe808902a56b2349c7f57a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348675.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/119/23be2c8393330d0797eb21aeaaf09119/23be2c8393330d0797eb21aeaaf09119.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348680.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d1b/ee1368276b79cd6ff697b372ec013d1b/ee1368276b79cd6ff697b372ec013d1b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348685.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/acd/e750d7f7a0098c4577bc00b7e3fdaacd/e750d7f7a0098c4577bc00b7e3fdaacd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348690.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/ddb/25e5fe9c34a81b6883376cdbe6ed6ddb/25e5fe9c34a81b6883376cdbe6ed6ddb.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348695.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/158/5296f4eb06cf79caf8accf442bb1f158/5296f4eb06cf79caf8accf442bb1f158.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348700.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/488/f74e83fac25a36ec34d5fd0e2fe91488/f74e83fac25a36ec34d5fd0e2fe91488.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348705.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f55/dfbd9847ef03a94cfe7df8ae5134af55/dfbd9847ef03a94cfe7df8ae5134af55.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348710.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d59/59e3df290e4775005b19f3c4af837d59/59e3df290e4775005b19f3c4af837d59.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348715.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c81/42f5ac27426d2a559a3b26fbaa553c81/42f5ac27426d2a559a3b26fbaa553c81.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348720.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e0c/fba3ff17603f4bd4dbda7585be18ae0c/fba3ff17603f4bd4dbda7585be18ae0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348725.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/220/0795f619d64804780637cb72d4a5b220/0795f619d64804780637cb72d4a5b220.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348730.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b5f/1f5852a8e4bb91dfdd4108cae7b33b5f/1f5852a8e4bb91dfdd4108cae7b33b5f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348735.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/7a5/166c1ddb0ab9c1173f7753521abdf7a5/166c1ddb0ab9c1173f7753521abdf7a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348740.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/7d3/7532c8d7b25914e8518aa19f6c6d77d3/7532c8d7b25914e8518aa19f6c6d77d3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348745.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/cd0/a06d5d00d689624133971d5bd0ca3cd0/a06d5d00d689624133971d5bd0ca3cd0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348750.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/932/571ab1d6d1986446a401595ba57fb932/571ab1d6d1986446a401595ba57fb932.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348755.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/bfd/4eea9c5516594734065b19ade255fbfd/4eea9c5516594734065b19ade255fbfd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348760.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/6ad/caa9dac73fe06a0279cf320e3426e6ad/caa9dac73fe06a0279cf320e3426e6ad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348765.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/627/a2f4d191a9443f9cea74f725a7584627/a2f4d191a9443f9cea74f725a7584627.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348770.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/26c/74847d44fd47693f0e5eb7417769f26c/74847d44fd47693f0e5eb7417769f26c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348775.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d29/4568af56df0fe57cbf21debcee37dd29/4568af56df0fe57cbf21debcee37dd29.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348780.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/745/099a33ac9118903d82aa58a508920745/099a33ac9118903d82aa58a508920745.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348785.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/86d/9e4e01ca49210ebf6780d40d095eb86d/9e4e01ca49210ebf6780d40d095eb86d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348790.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/958/6833a908e0a1f79f9781006e97a8d958/6833a908e0a1f79f9781006e97a8d958.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348795.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/a65/b0248a7653269a0befbc0d843b7e5a65/b0248a7653269a0befbc0d843b7e5a65.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348800.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/a0c/7acef9801031d30a340fa514e8a32a0c/7acef9801031d30a340fa514e8a32a0c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348805.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/d4e/277a9a7b46c13e2cb26599030ac2dd4e/277a9a7b46c13e2cb26599030ac2dd4e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348810.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/9c7/78990da72e23faec3a9095a2791399c7/78990da72e23faec3a9095a2791399c7.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348815.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b37/4b9e4bc15a0fa25921d2d45615241b37/4b9e4bc15a0fa25921d2d45615241b37.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_not_cleaving ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348820.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/a54/8685c69fabba176be7b050a7b5a89a54/8685c69fabba176be7b050a7b5a89a54.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348825.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/f77/f0519496f7719dfae28928b189bcff77/f0519496f7719dfae28928b189bcff77.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348830.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b02/79ba8abecdd686ff63ac361bcabc2b02/79ba8abecdd686ff63ac361bcabc2b02.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348835.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/e82/d9c2908eb6aa86f1e2d96943445d9e82/d9c2908eb6aa86f1e2d96943445d9e82.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348840.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/738/09da2499f96994fd20f7dcc08cf5d738/09da2499f96994fd20f7dcc08cf5d738.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348845.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/918/96eb8ee2b723453ceb06d592f9483918/96eb8ee2b723453ceb06d592f9483918.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348850.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/57a/5c483aee85083c5cab4fb02ab74fd57a/5c483aee85083c5cab4fb02ab74fd57a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348855.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/594/d2e09666fdc1bf81d8af09284156c594/d2e09666fdc1bf81d8af09284156c594.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348860.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/78e/bb0ff3c344bae622dbc8a2d87693b78e/bb0ff3c344bae622dbc8a2d87693b78e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348865.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/649/040f56ce540624fa15cbcb7b0dcef649/040f56ce540624fa15cbcb7b0dcef649.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348870.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/c3a/2c474cf727833ef5ed4ca15ff4d8cc3a/2c474cf727833ef5ed4ca15ff4d8cc3a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348875.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/84d/d12e6879b1ccf65b0b0766547c1fe84d/d12e6879b1ccf65b0b0766547c1fe84d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348880.00000-0, db: /tmp/tmpmn0fvu9q/sda/containers/0/b3c/6f3a91f67e517d08f841914dec341b3c/6f3a91f67e517d08f841914dec341b3c.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347616.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/822/76d0d2dd115adbe182bab8cba54a5822/76d0d2dd115adbe182bab8cba54a5822_1717347616.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/058/5cdb99d2d569d9da3bc419ff1dfc6058/5cdb99d2d569d9da3bc419ff1dfc6058.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347620.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/058/5cdb99d2d569d9da3bc419ff1dfc6058/5cdb99d2d569d9da3bc419ff1dfc6058_1717347620.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/497/91f469d461ed1b40629e3b651f7ff497/91f469d461ed1b40629e3b651f7ff497.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347624.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/497/91f469d461ed1b40629e3b651f7ff497/91f469d461ed1b40629e3b651f7ff497_1717347624.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/6ec/d1fcdcdcd0525c610d9af7acdaa856ec/d1fcdcdcd0525c610d9af7acdaa856ec.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347628.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/6ec/d1fcdcdcd0525c610d9af7acdaa856ec/d1fcdcdcd0525c610d9af7acdaa856ec_1717347628.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/088/2fbebd125f1e26ddb4a8d15fdc820088/2fbebd125f1e26ddb4a8d15fdc820088.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347632.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/088/2fbebd125f1e26ddb4a8d15fdc820088/2fbebd125f1e26ddb4a8d15fdc820088_1717347632.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b_1717347636.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c_1717347640.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b_1717347644.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e_1717347648.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a67/ba2680550578d85f47449360c6495a67/ba2680550578d85f47449360c6495a67.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347652.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a67/ba2680550578d85f47449360c6495a67/ba2680550578d85f47449360c6495a67_1717347652.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/b0e/0d131d949dc99f78dddc400126474b0e/0d131d949dc99f78dddc400126474b0e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347656.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/b0e/0d131d949dc99f78dddc400126474b0e/0d131d949dc99f78dddc400126474b0e_1717347656.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994_1717347660.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/393/4bbf27b69b8057524d6e994b2e2d3393/4bbf27b69b8057524d6e994b2e2d3393.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347664.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/393/4bbf27b69b8057524d6e994b2e2d3393/4bbf27b69b8057524d6e994b2e2d3393_1717347664.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/092/b97e0ac82d3b45e45d84ab82ea3eb092/b97e0ac82d3b45e45d84ab82ea3eb092.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347668.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/092/b97e0ac82d3b45e45d84ab82ea3eb092/b97e0ac82d3b45e45d84ab82ea3eb092_1717347668.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f_1717347672.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347676.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/b77/d25316e205bcc33d39107459b2dacb77/d25316e205bcc33d39107459b2dacb77_1717347676.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/8bb/aae77f028aa9a9780ee8fc5297c6c8bb/aae77f028aa9a9780ee8fc5297c6c8bb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347680.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/8bb/aae77f028aa9a9780ee8fc5297c6c8bb/aae77f028aa9a9780ee8fc5297c6c8bb_1717347680.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d_1717347684.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/47e/3d36a1f8bc39a691e323f1c76675047e/3d36a1f8bc39a691e323f1c76675047e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347688.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/47e/3d36a1f8bc39a691e323f1c76675047e/3d36a1f8bc39a691e323f1c76675047e_1717347688.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/2cc/cc14e786fd504b53d4d0a8be32fc12cc/cc14e786fd504b53d4d0a8be32fc12cc.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347692.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/2cc/cc14e786fd504b53d4d0a8be32fc12cc/cc14e786fd504b53d4d0a8be32fc12cc_1717347692.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b_1717347696.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347700.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/ab2/6f471c35f90e03bfc9ce3925a93adab2/6f471c35f90e03bfc9ce3925a93adab2_1717347700.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/cab/d4b698071714b8df8dafd2afbd822cab/d4b698071714b8df8dafd2afbd822cab.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347704.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/cab/d4b698071714b8df8dafd2afbd822cab/d4b698071714b8df8dafd2afbd822cab_1717347704.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a_1717347708.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a17/b086ebc18f0c64e71cc2cd0e4c282a17/b086ebc18f0c64e71cc2cd0e4c282a17.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347712.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a17/b086ebc18f0c64e71cc2cd0e4c282a17/b086ebc18f0c64e71cc2cd0e4c282a17_1717347712.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/ef8/2bd31ca5e5fb2da1a764945d5aaeeef8/2bd31ca5e5fb2da1a764945d5aaeeef8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347716.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/ef8/2bd31ca5e5fb2da1a764945d5aaeeef8/2bd31ca5e5fb2da1a764945d5aaeeef8_1717347716.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892_1717347720.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/5fb/09149bd74e1a4c2bb86e1343145d25fb/09149bd74e1a4c2bb86e1343145d25fb.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347724.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/5fb/09149bd74e1a4c2bb86e1343145d25fb/09149bd74e1a4c2bb86e1343145d25fb_1717347724.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/7d6/1899a69b54a05dd498c0310f9e00e7d6/1899a69b54a05dd498c0310f9e00e7d6.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347728.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/7d6/1899a69b54a05dd498c0310f9e00e7d6/1899a69b54a05dd498c0310f9e00e7d6_1717347728.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd_1717347732.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347736.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/67e/485ea85f3da4e9cfaa3ca8bca1cb867e/485ea85f3da4e9cfaa3ca8bca1cb867e_1717347736.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/45c/9c8ec32ccf58393b11dc3cd4e0e7e45c/9c8ec32ccf58393b11dc3cd4e0e7e45c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347740.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/45c/9c8ec32ccf58393b11dc3cd4e0e7e45c/9c8ec32ccf58393b11dc3cd4e0e7e45c_1717347740.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637_1717347744.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/d26/18305906879ad68520323d0c64288d26/18305906879ad68520323d0c64288d26.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347748.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/d26/18305906879ad68520323d0c64288d26/18305906879ad68520323d0c64288d26_1717347748.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/fa7/ef8e62c7146b163ea34986579107dfa7/ef8e62c7146b163ea34986579107dfa7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347752.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/fa7/ef8e62c7146b163ea34986579107dfa7/ef8e62c7146b163ea34986579107dfa7_1717347752.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef_1717347756.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347760.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/7c9/640d21a4ac6a2e54c8c665bb9bbb87c9/640d21a4ac6a2e54c8c665bb9bbb87c9_1717347760.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/ee0/03f57669aa76c0435d6085b558941ee0/03f57669aa76c0435d6085b558941ee0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347764.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/ee0/03f57669aa76c0435d6085b558941ee0/03f57669aa76c0435d6085b558941ee0_1717347764.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4_1717347768.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/320/21e0eccc354bf736c324ba3edcd56320/21e0eccc354bf736c324ba3edcd56320.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347772.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/320/21e0eccc354bf736c324ba3edcd56320/21e0eccc354bf736c324ba3edcd56320_1717347772.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a49/76c0b29cc74d17206aace3a14e682a49/76c0b29cc74d17206aace3a14e682a49.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347776.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a49/76c0b29cc74d17206aace3a14e682a49/76c0b29cc74d17206aace3a14e682a49_1717347776.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253_1717347780.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/273/7645cfa2cb0275a5639231cc10720273/7645cfa2cb0275a5639231cc10720273.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347784.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/273/7645cfa2cb0275a5639231cc10720273/7645cfa2cb0275a5639231cc10720273_1717347784.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/087/b7316f7a20743476f017f05b78a8c087/b7316f7a20743476f017f05b78a8c087.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347788.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/087/b7316f7a20743476f017f05b78a8c087/b7316f7a20743476f017f05b78a8c087_1717347788.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/e41/d1936b417b2226818d22874cab17de41/d1936b417b2226818d22874cab17de41.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347792.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/e41/d1936b417b2226818d22874cab17de41/d1936b417b2226818d22874cab17de41_1717347792.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/2e9/b24ef43711af2c751d4669c3d515d2e9/b24ef43711af2c751d4669c3d515d2e9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347796.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/2e9/b24ef43711af2c751d4669c3d515d2e9/b24ef43711af2c751d4669c3d515d2e9_1717347796.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/d60/29c4e4c38b2a9b66d6dc06a3a31c5d60/29c4e4c38b2a9b66d6dc06a3a31c5d60.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347800.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/d60/29c4e4c38b2a9b66d6dc06a3a31c5d60/29c4e4c38b2a9b66d6dc06a3a31c5d60_1717347800.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/699/9320cfa5738742e281472b44ad44d699/9320cfa5738742e281472b44ad44d699.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347804.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/699/9320cfa5738742e281472b44ad44d699/9320cfa5738742e281472b44ad44d699_1717347804.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/66f/e3ce65f8f58e896cb52cce184f61566f/e3ce65f8f58e896cb52cce184f61566f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347808.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/66f/e3ce65f8f58e896cb52cce184f61566f/e3ce65f8f58e896cb52cce184f61566f_1717347808.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a9f/745ed25c6ff5d22e968e252f502b2a9f/745ed25c6ff5d22e968e252f502b2a9f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347812.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a9f/745ed25c6ff5d22e968e252f502b2a9f/745ed25c6ff5d22e968e252f502b2a9f_1717347812.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/48c/d67b765547bbc5013b2795518c7a348c/d67b765547bbc5013b2795518c7a348c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347816.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/48c/d67b765547bbc5013b2795518c7a348c/d67b765547bbc5013b2795518c7a348c_1717347816.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/5fd/006fcabd0221977584acf3840c2005fd/006fcabd0221977584acf3840c2005fd.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347820.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/5fd/006fcabd0221977584acf3840c2005fd/006fcabd0221977584acf3840c2005fd_1717347820.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/29f/bcead454d54b4a183d519cb02152529f/bcead454d54b4a183d519cb02152529f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347824.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/29f/bcead454d54b4a183d519cb02152529f/bcead454d54b4a183d519cb02152529f_1717347824.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/2a5/14ee8038b1be82bea712800683ac72a5/14ee8038b1be82bea712800683ac72a5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347828.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/2a5/14ee8038b1be82bea712800683ac72a5/14ee8038b1be82bea712800683ac72a5_1717347828.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/e85/409c279027f476ab688ec24a5e847e85/409c279027f476ab688ec24a5e847e85.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347832.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/e85/409c279027f476ab688ec24a5e847e85/409c279027f476ab688ec24a5e847e85_1717347832.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/e5c/44d385caacc6153ade9eca4f23cfde5c/44d385caacc6153ade9eca4f23cfde5c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347836.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/e5c/44d385caacc6153ade9eca4f23cfde5c/44d385caacc6153ade9eca4f23cfde5c_1717347836.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/ba7/1be2ce69ea2bd1dc593afb7bb33fdba7/1be2ce69ea2bd1dc593afb7bb33fdba7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347840.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/ba7/1be2ce69ea2bd1dc593afb7bb33fdba7/1be2ce69ea2bd1dc593afb7bb33fdba7_1717347840.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/ce4/041213edcf15454f4f918910b38bfce4/041213edcf15454f4f918910b38bfce4.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347844.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/ce4/041213edcf15454f4f918910b38bfce4/041213edcf15454f4f918910b38bfce4_1717347844.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/4ff/3469ebe48b8b4a0b9da43b25d9ef64ff/3469ebe48b8b4a0b9da43b25d9ef64ff.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347848.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/4ff/3469ebe48b8b4a0b9da43b25d9ef64ff/3469ebe48b8b4a0b9da43b25d9ef64ff_1717347848.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/9c8/f6b92390955db34ccc10e7236b6d79c8/f6b92390955db34ccc10e7236b6d79c8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347852.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/9c8/f6b92390955db34ccc10e7236b6d79c8/f6b92390955db34ccc10e7236b6d79c8_1717347852.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/285/d3a10181415135d1939d0a3d5bb4d285/d3a10181415135d1939d0a3d5bb4d285.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347856.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/285/d3a10181415135d1939d0a3d5bb4d285/d3a10181415135d1939d0a3d5bb4d285_1717347856.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/1f1/525911717c691d9742a0d63954d751f1/525911717c691d9742a0d63954d751f1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347860.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/1f1/525911717c691d9742a0d63954d751f1/525911717c691d9742a0d63954d751f1_1717347860.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/98f/13dea24fc7cbf289728965b21604098f/13dea24fc7cbf289728965b21604098f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347864.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/98f/13dea24fc7cbf289728965b21604098f/13dea24fc7cbf289728965b21604098f_1717347864.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/128/13c46c5ff78984828a265c969ec15128/13c46c5ff78984828a265c969ec15128.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347868.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/128/13c46c5ff78984828a265c969ec15128/13c46c5ff78984828a265c969ec15128_1717347868.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/604/55357dbd6608b7c48d2b43fbf99c6604/55357dbd6608b7c48d2b43fbf99c6604.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347872.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/604/55357dbd6608b7c48d2b43fbf99c6604/55357dbd6608b7c48d2b43fbf99c6604_1717347872.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/565/b413d5e29b4d50e7764db0333e0a8565/b413d5e29b4d50e7764db0333e0a8565.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347876.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/565/b413d5e29b4d50e7764db0333e0a8565/b413d5e29b4d50e7764db0333e0a8565_1717347876.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/b3e/670079b1c2086bc65d0a8c9c0e61fb3e/670079b1c2086bc65d0a8c9c0e61fb3e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347880.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/b3e/670079b1c2086bc65d0a8c9c0e61fb3e/670079b1c2086bc65d0a8c9c0e61fb3e_1717347880.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/5f0/d510078a15be2f1d5a984cfcc656d5f0/d510078a15be2f1d5a984cfcc656d5f0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347884.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/5f0/d510078a15be2f1d5a984cfcc656d5f0/d510078a15be2f1d5a984cfcc656d5f0_1717347884.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a36/c936db8f4e97f3e74f3a3a0aa79a1a36/c936db8f4e97f3e74f3a3a0aa79a1a36.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347888.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a36/c936db8f4e97f3e74f3a3a0aa79a1a36/c936db8f4e97f3e74f3a3a0aa79a1a36_1717347888.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/1be/c4fe4a3892da26fadbba8860d8e721be/c4fe4a3892da26fadbba8860d8e721be.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347892.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/1be/c4fe4a3892da26fadbba8860d8e721be/c4fe4a3892da26fadbba8860d8e721be_1717347892.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/05b/e2ed524e8060da23f95e47135292005b/e2ed524e8060da23f95e47135292005b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347896.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/05b/e2ed524e8060da23f95e47135292005b/e2ed524e8060da23f95e47135292005b_1717347896.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/233/afd45040e81a45be47f71516659dc233/afd45040e81a45be47f71516659dc233.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347900.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/233/afd45040e81a45be47f71516659dc233/afd45040e81a45be47f71516659dc233_1717347900.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/664/85608fc93d22dd95a6f4a5d8a5ab8664/85608fc93d22dd95a6f4a5d8a5ab8664.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347904.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/664/85608fc93d22dd95a6f4a5d8a5ab8664/85608fc93d22dd95a6f4a5d8a5ab8664_1717347904.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/4c5/ebf147f750f1ac49ec4ea00deeddd4c5/ebf147f750f1ac49ec4ea00deeddd4c5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347908.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/4c5/ebf147f750f1ac49ec4ea00deeddd4c5/ebf147f750f1ac49ec4ea00deeddd4c5_1717347908.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/0ad/28c146296bcaab438a721157da1500ad/28c146296bcaab438a721157da1500ad.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347912.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/0ad/28c146296bcaab438a721157da1500ad/28c146296bcaab438a721157da1500ad_1717347912.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/c09/dd1ed34f0917ea13f0b85442b07ecc09/dd1ed34f0917ea13f0b85442b07ecc09.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347916.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/c09/dd1ed34f0917ea13f0b85442b07ecc09/dd1ed34f0917ea13f0b85442b07ecc09_1717347916.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/3fe/580e9c5d3741cad4261f2df0237143fe/580e9c5d3741cad4261f2df0237143fe.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347920.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/3fe/580e9c5d3741cad4261f2df0237143fe/580e9c5d3741cad4261f2df0237143fe_1717347920.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/cd0/622fdea2cb3fd9138910988e2d3f0cd0/622fdea2cb3fd9138910988e2d3f0cd0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347924.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/cd0/622fdea2cb3fd9138910988e2d3f0cd0/622fdea2cb3fd9138910988e2d3f0cd0_1717347924.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/958/ba1e8ebd5e792aac461d8f62bb86d958/ba1e8ebd5e792aac461d8f62bb86d958.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347928.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/958/ba1e8ebd5e792aac461d8f62bb86d958/ba1e8ebd5e792aac461d8f62bb86d958_1717347928.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/4cf/c7e0b8bc40c7d148bc9509c7c5be24cf/c7e0b8bc40c7d148bc9509c7c5be24cf.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347932.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/4cf/c7e0b8bc40c7d148bc9509c7c5be24cf/c7e0b8bc40c7d148bc9509c7c5be24cf_1717347932.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/aef/350415aca3e9717e132d2b4b9c284aef/350415aca3e9717e132d2b4b9c284aef.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347936.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/aef/350415aca3e9717e132d2b4b9c284aef/350415aca3e9717e132d2b4b9c284aef_1717347936.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/eef/3854af72dd9d24247430f1e7621dbeef/3854af72dd9d24247430f1e7621dbeef.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347940.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/eef/3854af72dd9d24247430f1e7621dbeef/3854af72dd9d24247430f1e7621dbeef_1717347940.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/666/c575dd99b852a9dfa1f1e61d239a0666/c575dd99b852a9dfa1f1e61d239a0666.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347944.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/666/c575dd99b852a9dfa1f1e61d239a0666/c575dd99b852a9dfa1f1e61d239a0666_1717347944.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/5d1/c657bee1d0cd66ce6fb949dd866ad5d1/c657bee1d0cd66ce6fb949dd866ad5d1.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347948.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/5d1/c657bee1d0cd66ce6fb949dd866ad5d1/c657bee1d0cd66ce6fb949dd866ad5d1_1717347948.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/12c/872a80915038c869a21b70f0f295e12c/872a80915038c869a21b70f0f295e12c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347952.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/12c/872a80915038c869a21b70f0f295e12c/872a80915038c869a21b70f0f295e12c_1717347952.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/d2d/88c1ca3ab9b1764849c170469e2ced2d/88c1ca3ab9b1764849c170469e2ced2d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347956.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/d2d/88c1ca3ab9b1764849c170469e2ced2d/88c1ca3ab9b1764849c170469e2ced2d_1717347956.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/96c/1b9ff9b0c59e9547a4750f0d431bb96c/1b9ff9b0c59e9547a4750f0d431bb96c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347960.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/96c/1b9ff9b0c59e9547a4750f0d431bb96c/1b9ff9b0c59e9547a4750f0d431bb96c_1717347960.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/615/b82b89a2f572a80616ae5b544bb7a615/b82b89a2f572a80616ae5b544bb7a615.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347964.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/615/b82b89a2f572a80616ae5b544bb7a615/b82b89a2f572a80616ae5b544bb7a615_1717347964.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/d8d/02fe3ad79df2ad66a5d19cc3dccf6d8d/02fe3ad79df2ad66a5d19cc3dccf6d8d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347968.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/d8d/02fe3ad79df2ad66a5d19cc3dccf6d8d/02fe3ad79df2ad66a5d19cc3dccf6d8d_1717347968.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/afa/b387f46972e50b6a11fb253629661afa/b387f46972e50b6a11fb253629661afa.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347972.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/afa/b387f46972e50b6a11fb253629661afa/b387f46972e50b6a11fb253629661afa_1717347972.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/00f/3d2d8bbcb194eab9832eef73529dc00f/3d2d8bbcb194eab9832eef73529dc00f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347976.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/00f/3d2d8bbcb194eab9832eef73529dc00f/3d2d8bbcb194eab9832eef73529dc00f_1717347976.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/f97/2a80ffebc7c69a504f975770c6693f97/2a80ffebc7c69a504f975770c6693f97.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347980.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/f97/2a80ffebc7c69a504f975770c6693f97/2a80ffebc7c69a504f975770c6693f97_1717347980.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/cc9/8837fb85b1f1e4d70080d9a6b07f5cc9/8837fb85b1f1e4d70080d9a6b07f5cc9.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347984.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/cc9/8837fb85b1f1e4d70080d9a6b07f5cc9/8837fb85b1f1e4d70080d9a6b07f5cc9_1717347984.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/c57/ad45fc4c93f6fa8ed45caa5c54310c57/ad45fc4c93f6fa8ed45caa5c54310c57.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347988.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/c57/ad45fc4c93f6fa8ed45caa5c54310c57/ad45fc4c93f6fa8ed45caa5c54310c57_1717347988.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/c7e/6e3ae188ccb2d3e7f2f494bf7c68ec7e/6e3ae188ccb2d3e7f2f494bf7c68ec7e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347992.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/c7e/6e3ae188ccb2d3e7f2f494bf7c68ec7e/6e3ae188ccb2d3e7f2f494bf7c68ec7e_1717347992.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/6f8/a8a95b8853f53efd045879a9fe8456f8/a8a95b8853f53efd045879a9fe8456f8.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347996.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/6f8/a8a95b8853f53efd045879a9fe8456f8/a8a95b8853f53efd045879a9fe8456f8_1717347996.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/ba5/b598bcb80f3103b28c49f79766e04ba5/b598bcb80f3103b28c49f79766e04ba5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348000.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/ba5/b598bcb80f3103b28c49f79766e04ba5/b598bcb80f3103b28c49f79766e04ba5_1717348000.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/f26/8102adcac61c8a610a53bb7609235f26/8102adcac61c8a610a53bb7609235f26.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348004.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/f26/8102adcac61c8a610a53bb7609235f26/8102adcac61c8a610a53bb7609235f26_1717348004.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/960/877ae45676b0e47be4f0865aac525960/877ae45676b0e47be4f0865aac525960.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348008.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/960/877ae45676b0e47be4f0865aac525960/877ae45676b0e47be4f0865aac525960_1717348008.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/7e7/a6f865d083a450f65908934c865bc7e7/a6f865d083a450f65908934c865bc7e7.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348012.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/7e7/a6f865d083a450f65908934c865bc7e7/a6f865d083a450f65908934c865bc7e7_1717348012.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/00b/d9ee39111d673d7cae8a0355141e000b/d9ee39111d673d7cae8a0355141e000b.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348016.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/00b/d9ee39111d673d7cae8a0355141e000b/d9ee39111d673d7cae8a0355141e000b_1717348016.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/9ce/a425f11f2290689b2f5878d7cfe5e9ce/a425f11f2290689b2f5878d7cfe5e9ce.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348020.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/9ce/a425f11f2290689b2f5878d7cfe5e9ce/a425f11f2290689b2f5878d7cfe5e9ce_1717348020.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/d01/005742d613c28016ac0878c42fd93d01/005742d613c28016ac0878c42fd93d01.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348024.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/d01/005742d613c28016ac0878c42fd93d01/005742d613c28016ac0878c42fd93d01_1717348024.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/f47/68ed789ecdbddb60858ac54af8b77f47/68ed789ecdbddb60858ac54af8b77f47.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348028.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/f47/68ed789ecdbddb60858ac54af8b77f47/68ed789ecdbddb60858ac54af8b77f47_1717348028.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/67c/432775c553e2e6628bb467901fb5267c/432775c553e2e6628bb467901fb5267c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348032.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/67c/432775c553e2e6628bb467901fb5267c/432775c553e2e6628bb467901fb5267c_1717348032.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/1a5/514f3ee9b3f8b8fcd3adea99f02401a5/514f3ee9b3f8b8fcd3adea99f02401a5.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348036.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/1a5/514f3ee9b3f8b8fcd3adea99f02401a5/514f3ee9b3f8b8fcd3adea99f02401a5_1717348036.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/f97/2a2166a062677ec01259d644dab03f97/2a2166a062677ec01259d644dab03f97.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348040.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/f97/2a2166a062677ec01259d644dab03f97/2a2166a062677ec01259d644dab03f97_1717348040.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/858/38d68fa5c3818d02f8ab3e2a52487858/38d68fa5c3818d02f8ab3e2a52487858.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348044.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/858/38d68fa5c3818d02f8ab3e2a52487858/38d68fa5c3818d02f8ab3e2a52487858_1717348044.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/277/eae3d4cb55d095e734e5a6de1be5f277/eae3d4cb55d095e734e5a6de1be5f277.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348048.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/277/eae3d4cb55d095e734e5a6de1be5f277/eae3d4cb55d095e734e5a6de1be5f277_1717348048.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/aee/0af863175f2b5fb453158e4ef9309aee/0af863175f2b5fb453158e4ef9309aee.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348052.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/aee/0af863175f2b5fb453158e4ef9309aee/0af863175f2b5fb453158e4ef9309aee_1717348052.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/9fa/cc4bcab6963551ceb295ca97841339fa/cc4bcab6963551ceb295ca97841339fa.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348056.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/9fa/cc4bcab6963551ceb295ca97841339fa/cc4bcab6963551ceb295ca97841339fa_1717348056.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/b7d/bc7cfaf8392671e45668c28e4202fb7d/bc7cfaf8392671e45668c28e4202fb7d.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348060.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/b7d/bc7cfaf8392671e45668c28e4202fb7d/bc7cfaf8392671e45668c28e4202fb7d_1717348060.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/01d/39930ef104f23b7340a1e8923457e01d/39930ef104f23b7340a1e8923457e01d.db' test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_not_merged_once_sharded ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_1 ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348064.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/01d/39930ef104f23b7340a1e8923457e01d/39930ef104f23b7340a1e8923457e01d_1717348064.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/9b0/56fb8bffafdb309a8f6e4acda54449b0/56fb8bffafdb309a8f6e4acda54449b0.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348068.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/9b0/56fb8bffafdb309a8f6e4acda54449b0/56fb8bffafdb309a8f6e4acda54449b0_1717348068.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/166/e5d4dd24d99d5543e7b76d4a28fdc166/e5d4dd24d99d5543e7b76d4a28fdc166.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348072.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/166/e5d4dd24d99d5543e7b76d4a28fdc166/e5d4dd24d99d5543e7b76d4a28fdc166_1717348072.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/aa3/f42c5e768772f5ca5a9592890f391aa3/f42c5e768772f5ca5a9592890f391aa3.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348076.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/aa3/f42c5e768772f5ca5a9592890f391aa3/f42c5e768772f5ca5a9592890f391aa3_1717348076.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/1ca/32658bb760b2094363eccf14bb1ef1ca/32658bb760b2094363eccf14bb1ef1ca.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348080.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/1ca/32658bb760b2094363eccf14bb1ef1ca/32658bb760b2094363eccf14bb1ef1ca_1717348080.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/921/6765d35ff3c369dcfda26e90db628921/6765d35ff3c369dcfda26e90db628921.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348084.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/921/6765d35ff3c369dcfda26e90db628921/6765d35ff3c369dcfda26e90db628921_1717348084.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/b50/d387e3f94bbb6e6a37a83065404dab50/d387e3f94bbb6e6a37a83065404dab50.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348088.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/b50/d387e3f94bbb6e6a37a83065404dab50/d387e3f94bbb6e6a37a83065404dab50_1717348088.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/972/af972eb3d386fed6640145c3bab8d972/af972eb3d386fed6640145c3bab8d972.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348092.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/972/af972eb3d386fed6640145c3bab8d972/af972eb3d386fed6640145c3bab8d972_1717348092.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/f67/cc3de10bf4129ecbc2470be5ac438f67/cc3de10bf4129ecbc2470be5ac438f67.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348096.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/f67/cc3de10bf4129ecbc2470be5ac438f67/cc3de10bf4129ecbc2470be5ac438f67_1717348096.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/e50/16c14f0b3d21999f1e1378d49943ce50/16c14f0b3d21999f1e1378d49943ce50.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348100.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/e50/16c14f0b3d21999f1e1378d49943ce50/16c14f0b3d21999f1e1378d49943ce50_1717348100.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/322/012e9d1415cf211ace64de0e0a686322/012e9d1415cf211ace64de0e0a686322.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348104.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/322/012e9d1415cf211ace64de0e0a686322/012e9d1415cf211ace64de0e0a686322_1717348104.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/73f/95a510ede92cae4493478c8fd64d973f/95a510ede92cae4493478c8fd64d973f.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348108.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/73f/95a510ede92cae4493478c8fd64d973f/95a510ede92cae4493478c8fd64d973f_1717348108.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/b64/0f858c47d7b8e60cd647dca0df5a9b64/0f858c47d7b8e60cd647dca0df5a9b64.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348112.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/b64/0f858c47d7b8e60cd647dca0df5a9b64/0f858c47d7b8e60cd647dca0df5a9b64_1717348112.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/c2e/8aaf648894753f3b784680267123fc2e/8aaf648894753f3b784680267123fc2e.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348116.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/c2e/8aaf648894753f3b784680267123fc2e/8aaf648894753f3b784680267123fc2e_1717348116.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/984/ce3bf1de18e53a815a6efebe6da41984/ce3bf1de18e53a815a6efebe6da41984.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348120.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/984/ce3bf1de18e53a815a6efebe6da41984/ce3bf1de18e53a815a6efebe6da41984_1717348120.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi4yy0m5x/sda/containers/0/a5c/cb803d1ab0a9f5604bd8e21af9194a5c/cb803d1ab0a9f5604bd8e21af9194a5c.db' sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717348124.00000-0, db: /tmp/tmpi4yy0m5x/sda/containers/0/a5c/cb803d1ab0a9f5604bd8e21af9194a5c/cb803d1ab0a9f5604bd8e21af9194a5c_1717348124.00000.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347636.00000-0, db: /tmp/tmpm3hn2wy9/sda/containers/0/a9b/49be6657fa6bd50376f9a5e7ff672a9b/49be6657fa6bd50376f9a5e7ff672a9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpm3hn2wy9/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpm3hn2wy9/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmpm3hn2wy9/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpxev7nihk/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmpxev7nihk/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpxev7nihk/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_overlap_not_merged_while_cleaving_2 ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_replace_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_deleted_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_supplement_existing_while_cleaving ... ok test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_gap_not_merged_while_cleaving ... ok sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347647.00000-0, db: /tmp/tmpxev7nihk/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347646.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/5bf/79bdd50edee9a7fffcf155be357035bf/79bdd50edee9a7fffcf155be357035bf.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347650.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/939/a65f6f7661617c4312bcbbcf34c63939/a65f6f7661617c4312bcbbcf34c63939.db sharder-test DEBUG: Updating 1 children shard ranges from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347650.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/939/a65f6f7661617c4312bcbbcf34c63939/a65f6f7661617c4312bcbbcf34c63939.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347650.00000-0, db: /tmp/tmpoffdzs0k/sda/containers/0/939/a65f6f7661617c4312bcbbcf34c63939/a65f6f7661617c4312bcbbcf34c63939.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347647.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8.db sharder-test DEBUG: Updating 2 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347647.00000-0, db: /tmp/tmp4soxp5yu/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347641.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/de8/5e8853d488154add897576e56d9f0de8/5e8853d488154add897576e56d9f0de8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347647.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347647.00000-0, db: /tmp/tmpjyt9n5wz/sda/containers/0/cd8/bd666163debc4ae5b19982111d5cfcd8/bd666163debc4ae5b19982111d5cfcd8.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347638.00000-0, db: /tmp/tmpwx6hi9eo/sda/containers/0/cf3/bcdf4668cbcb36c4d985535508273cf3/bcdf4668cbcb36c4d985535508273cf3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347640.00000-0, db: /tmp/tmpwx6hi9eo/sda/containers/0/d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c/bfb7b4e7464a6df3fd9e605b8ac14d6c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmpwx6hi9eo/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347644.00000-0, db: /tmp/tmpwx6hi9eo/sda/containers/0/62b/fbf87d8d3042208a1111dcbca65a762b/fbf87d8d3042208a1111dcbca65a762b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347639.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/ea1/1f99b787458c5c33f5803b31cdb8bea1/1f99b787458c5c33f5803b31cdb8bea1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347639.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/ea1/1f99b787458c5c33f5803b31cdb8bea1/1f99b787458c5c33f5803b31cdb8bea1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347642.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a/4a5803c4e4f1de15ebafb6a6b5dc5e0a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347645.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/183/6bf079a05dae9790d9e9796781c0c183/6bf079a05dae9790d9e9796781c0c183.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347648.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/29e/58a700fd2022db7224df34eb5271529e/58a700fd2022db7224df34eb5271529e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347651.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/3c3/184c391698913c1bf1099c278d14c3c3/184c391698913c1bf1099c278d14c3c3.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347654.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/dbd/097740ddacdce3d4e2421474224c0dbd/097740ddacdce3d4e2421474224c0dbd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347654.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/dbd/097740ddacdce3d4e2421474224c0dbd/097740ddacdce3d4e2421474224c0dbd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347657.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e71/8d9ecda0d9168e958c4727b47f3d7e71/8d9ecda0d9168e958c4727b47f3d7e71.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347657.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e71/8d9ecda0d9168e958c4727b47f3d7e71/8d9ecda0d9168e958c4727b47f3d7e71.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347660.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/994/0d70eb98adf5ee310a83e81c7e847994/0d70eb98adf5ee310a83e81c7e847994.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347663.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/b16/374952945e317b566e0211f2168adb16/374952945e317b566e0211f2168adb16.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347663.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/b16/374952945e317b566e0211f2168adb16/374952945e317b566e0211f2168adb16.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347666.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/b01/ded215a90f8af0bd03a459a3039a7b01/ded215a90f8af0bd03a459a3039a7b01.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347666.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/b01/ded215a90f8af0bd03a459a3039a7b01/ded215a90f8af0bd03a459a3039a7b01.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347669.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/453/41b0108c741007ad07bbd693ed2cc453/41b0108c741007ad07bbd693ed2cc453.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347669.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/453/41b0108c741007ad07bbd693ed2cc453/41b0108c741007ad07bbd693ed2cc453.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347672.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/17f/339e1f98a012885ac78b73f755c7b17f/339e1f98a012885ac78b73f755c7b17f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347675.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/4f0/d5294c7b7f66012c8eac60156d4414f0/d5294c7b7f66012c8eac60156d4414f0.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347678.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/67c/1289950a2088fb5fd8792eff1064d67c/1289950a2088fb5fd8792eff1064d67c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347678.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/67c/1289950a2088fb5fd8792eff1064d67c/1289950a2088fb5fd8792eff1064d67c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347681.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/5c1/cf8695527a87cd6772599df69bd4d5c1/cf8695527a87cd6772599df69bd4d5c1.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347684.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e2d/68763d16902f9ba135ef8d039f431e2d/68763d16902f9ba135ef8d039f431e2d.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347687.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/7a5/54dc622594c35edea8f581205cbbf7a5/54dc622594c35edea8f581205cbbf7a5.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347687.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/7a5/54dc622594c35edea8f581205cbbf7a5/54dc622594c35edea8f581205cbbf7a5.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347690.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/354/f379bcf67d17b83e8612a2b17eb0c354/f379bcf67d17b83e8612a2b17eb0c354.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347690.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/354/f379bcf67d17b83e8612a2b17eb0c354/f379bcf67d17b83e8612a2b17eb0c354.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347693.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/a97/5c0ecabe8bb529f8452437dd5213ca97/5c0ecabe8bb529f8452437dd5213ca97.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347693.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/a97/5c0ecabe8bb529f8452437dd5213ca97/5c0ecabe8bb529f8452437dd5213ca97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347696.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/78b/990d9579dee54e60599184a4e3b7e78b/990d9579dee54e60599184a4e3b7e78b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347699.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/192/fdbab859519c6bc89c2614bd151e6192/fdbab859519c6bc89c2614bd151e6192.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347699.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/192/fdbab859519c6bc89c2614bd151e6192/fdbab859519c6bc89c2614bd151e6192.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347702.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/988/7c549a494f0bfebe80a7ed84753f6988/7c549a494f0bfebe80a7ed84753f6988.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347702.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/988/7c549a494f0bfebe80a7ed84753f6988/7c549a494f0bfebe80a7ed84753f6988.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347705.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/395/7cf7c7cbe2cb355b54853e361868d395/7cf7c7cbe2cb355b54853e361868d395.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347708.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/19a/169a121762785b6171fd2dd87cb3019a/169a121762785b6171fd2dd87cb3019a.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347711.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/6df/57dd647dd6d1de8ae7123608ff9736df/57dd647dd6d1de8ae7123608ff9736df.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347711.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/6df/57dd647dd6d1de8ae7123608ff9736df/57dd647dd6d1de8ae7123608ff9736df.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347714.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/44e/25c6c380f4c0a13b36131a3e1202844e/25c6c380f4c0a13b36131a3e1202844e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347714.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/44e/25c6c380f4c0a13b36131a3e1202844e/25c6c380f4c0a13b36131a3e1202844e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347717.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/a45/b15814e0ee6abb76e58d1fada8263a45/b15814e0ee6abb76e58d1fada8263a45.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347717.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/a45/b15814e0ee6abb76e58d1fada8263a45/b15814e0ee6abb76e58d1fada8263a45.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347720.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/892/2ade507575941e408bc2e02da201b892/2ade507575941e408bc2e02da201b892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347723.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/02e/c931380aa2221677ef56675beb80602e/c931380aa2221677ef56675beb80602e.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347723.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/02e/c931380aa2221677ef56675beb80602e/c931380aa2221677ef56675beb80602e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347726.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/919/2d3fccec5449ced10bab5e792b82d919/2d3fccec5449ced10bab5e792b82d919.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347726.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/919/2d3fccec5449ced10bab5e792b82d919/2d3fccec5449ced10bab5e792b82d919.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347729.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/dab/d2186b63fdc704a51a4604fbfaa9bdab/d2186b63fdc704a51a4604fbfaa9bdab.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347729.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/dab/d2186b63fdc704a51a4604fbfaa9bdab/d2186b63fdc704a51a4604fbfaa9bdab.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347732.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/6bd/85a811513b054920d4c47a1e993756bd/85a811513b054920d4c47a1e993756bd.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347735.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/b6f/66be2eb4b3cf41d35caa1c96a69b1b6f/66be2eb4b3cf41d35caa1c96a69b1b6f.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347735.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/b6f/66be2eb4b3cf41d35caa1c96a69b1b6f/66be2eb4b3cf41d35caa1c96a69b1b6f.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347738.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/a91/bc595ce8fa6c7086403ec8c34ff5ba91/bc595ce8fa6c7086403ec8c34ff5ba91.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347738.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/a91/bc595ce8fa6c7086403ec8c34ff5ba91/bc595ce8fa6c7086403ec8c34ff5ba91.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347741.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/050/e728370f5c95fbf6f7eb2b609df42050/e728370f5c95fbf6f7eb2b609df42050.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347744.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/637/48b58c57a3ded628e75d2a90c25d6637/48b58c57a3ded628e75d2a90c25d6637.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347747.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/51c/4f33ed770ca99d94ef786d8ba55af51c/4f33ed770ca99d94ef786d8ba55af51c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347747.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/51c/4f33ed770ca99d94ef786d8ba55af51c/4f33ed770ca99d94ef786d8ba55af51c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347750.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/c40/3533c4527a93b20163c3e3ad474d2c40/3533c4527a93b20163c3e3ad474d2c40.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347750.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/c40/3533c4527a93b20163c3e3ad474d2c40/3533c4527a93b20163c3e3ad474d2c40.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347753.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e18/3c452bf7dc2198444c3b6de75365ae18/3c452bf7dc2198444c3b6de75365ae18.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347753.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e18/3c452bf7dc2198444c3b6de75365ae18/3c452bf7dc2198444c3b6de75365ae18.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347756.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/7ef/730772246f881e2f714ae8176bccc7ef/730772246f881e2f714ae8176bccc7ef.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347759.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/10c/87c64fc016d99651aa51f01c4447a10c/87c64fc016d99651aa51f01c4447a10c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347759.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/10c/87c64fc016d99651aa51f01c4447a10c/87c64fc016d99651aa51f01c4447a10c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347762.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/f50/60bb6f35de82226a6ab635b227d46f50/60bb6f35de82226a6ab635b227d46f50.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347762.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/f50/60bb6f35de82226a6ab635b227d46f50/60bb6f35de82226a6ab635b227d46f50.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347765.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/bad/777eb0ba01bcabd869da513d2661bbad/777eb0ba01bcabd869da513d2661bbad.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347768.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/2e4/dfa745bd9c7a0b0f81e63819de2ab2e4/dfa745bd9c7a0b0f81e63819de2ab2e4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347771.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/41c/318dbae52b2facfd3d7308f395f2b41c/318dbae52b2facfd3d7308f395f2b41c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347771.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/41c/318dbae52b2facfd3d7308f395f2b41c/318dbae52b2facfd3d7308f395f2b41c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347774.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/892/3e6d6bc001af0c90b4955706dfeba892/3e6d6bc001af0c90b4955706dfeba892.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347774.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/892/3e6d6bc001af0c90b4955706dfeba892/3e6d6bc001af0c90b4955706dfeba892.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347777.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/aa4/0ac2721c681eddec28c20770fdc98aa4/0ac2721c681eddec28c20770fdc98aa4.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347777.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/aa4/0ac2721c681eddec28c20770fdc98aa4/0ac2721c681eddec28c20770fdc98aa4.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347780.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/253/a3c7c3f2dc00da7527f6375fe0f39253/a3c7c3f2dc00da7527f6375fe0f39253.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347783.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/0f2/702527c1285e7b149163e20d31aa90f2/702527c1285e7b149163e20d31aa90f2.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347783.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/0f2/702527c1285e7b149163e20d31aa90f2/702527c1285e7b149163e20d31aa90f2.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347786.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/48e/3cd1b208bf2bcff2ebe5a4ef91ae948e/3cd1b208bf2bcff2ebe5a4ef91ae948e.db test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking test.unit.container.test_sharder.TestSharder.test_audit_shard_root_ranges_with_own_merged_while_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_check_node test.unit.container.test_sharder.TestSharder.test_check_node ... ok sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347786.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/48e/3cd1b208bf2bcff2ebe5a4ef91ae948e/3cd1b208bf2bcff2ebe5a4ef91ae948e.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347789.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/8ae/d0229f4a42f0273dff20f749815978ae/d0229f4a42f0273dff20f749815978ae.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347789.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/8ae/d0229f4a42f0273dff20f749815978ae/d0229f4a42f0273dff20f749815978ae.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347792.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e41/d1936b417b2226818d22874cab17de41/d1936b417b2226818d22874cab17de41.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347792.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/e41/d1936b417b2226818d22874cab17de41/d1936b417b2226818d22874cab17de41.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347795.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/d9b/050560945ef5bbab351b2b51d557ad9b/050560945ef5bbab351b2b51d557ad9b.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347795.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/d9b/050560945ef5bbab351b2b51d557ad9b/050560945ef5bbab351b2b51d557ad9b.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347798.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/94c/93c7721a720410f9d9068b6d124a894c/93c7721a720410f9d9068b6d124a894c.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347798.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/94c/93c7721a720410f9d9068b6d124a894c/93c7721a720410f9d9068b6d124a894c.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347801.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/f97/83e0c884ad3cccd2eeb6c7628ad72f97/83e0c884ad3cccd2eeb6c7628ad72f97.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347801.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/f97/83e0c884ad3cccd2eeb6c7628ad72f97/83e0c884ad3cccd2eeb6c7628ad72f97.db sharder-test DEBUG: Updating own shard range from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347804.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/699/9320cfa5738742e281472b44ad44d699/9320cfa5738742e281472b44ad44d699.db sharder-test DEBUG: Updating 1 other shard range(s) from root, path: .shards_a/c-4a8a08f09d37b73795649038408b5f33-1717347804.00000-0, db: /tmp/tmp7ggbm_iy/sda/containers/0/699/9320cfa5738742e281472b44ad44d699/9320cfa5738742e281472b44ad44d699.db sharder-test WARNING: Skipping d100 as it is not mounted sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002944469451904297s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017523765563964844s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_0 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpff6e5axh/sda/containers/0/b85/9d7a328814f4e99b3c0e6a0597883b85/9d7a328814f4e99b3c0e6a0597883b85.db, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication test.unit.container.test_sharder.TestSharder.test_cleave_insufficient_replication ... ok sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Cleaved ShardRange in 0.013s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00032520294189453125s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001678466796875s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpff6e5axh/sda/containers/0/069/5a8214a697fa9e61b996df73488a7069/5a8214a697fa9e61b996df73488a7069.db, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: misplaced object source bounds [('', 'obj004')], path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018453598022460938s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00011181831359863281s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018668174743652344s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002841949462890625s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015163421630859375s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717347261.21455, (5, 5) as of 1717347637.28752, cleaved as of 1717347261.21455> in 0.023s, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpff6e5axh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347636.91321.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies test.unit.container.test_sharder.TestSharder.test_cleave_multiple_storage_policies ... ok sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002601146697998047s, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00016760826110839844s, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test INFO: Cleaved ShardRange in 0.032s, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002532005310058594s, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0001704692840576172s, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717347261.21455, (1, 6144) as of 1717347637.57306, cleaved as of 1717347261.21455> in 0.037s, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpl_hof5w7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.36036.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002913475036621094s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000156402587890625s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaved ShardRange in 0.035s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: got 6 rows (deleted=False) in 0.00028634071350097656s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001685619354248047s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_repeated test.unit.container.test_sharder.TestSharder.test_cleave_repeated ... ok sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717347261.21455, (6, 6) as of 1717347637.86109, cleaved as of 1717347261.21455> in 0.041s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '237fd49b-a481-47fc-abd0-5a31576770a4-sda', 'cursor': '', 'max_row': 12, 'cleave_to_row': 10, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Completed cleaving, DB remaining in sharding state, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaving from row 10 into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002510547637939453s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014591217041015625s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaved ShardRange in 0.032s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaving from row 10 into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717347261.21455, (6, 6) as of 1717347637.86109, cleaved as of 1717347261.21455>, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024509429931640625s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001418590545654297s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717347261.21455, (6, 6) as of 1717347637.86109, cleaved as of 1717347261.21455> in 0.018s, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp0ocyt_u8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347637.63701.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002579689025878906s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020122528076171875s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaved ShardRange in 0.033s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, found as of 1717347261.21455>, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003676414489746094s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001583099365234375s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmpakz5kwdv/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - shard db already in sync, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717347261.21455, (3, 6) as of 1717347638.43621, cleaved as of 1717347261.21455> in 0.01s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003464221954345703s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019431114196777344s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717347261.21455, (1, 100) as of 1717347638.50393, cleaved as of 1717347261.21455> in 0.025s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002627372741699219s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_root test.unit.container.test_sharder.TestSharder.test_cleave_root ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_pre_existing_shard_db_handoff ... ok sharder-test DEBUG: got 1 rows (deleted=True) in 0.00015664100646972656s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717347261.21455, (1, 1000) as of 1717347638.60577, cleaved as of 1717347261.21455> in 0.03s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717347261.21455, (2, 15) as of 1717347638.65615, active as of 1717347261.21455>, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003383159637451172s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021004676818847656s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717347261.21455, (2, 15) as of 1717347638.65615, active as of 1717347261.21455> in 0.024s, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmpakz5kwdv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.13027.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003464221954345703s, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001983642578125s, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test INFO: Cleaved ShardRange in 0.037s, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmprohj7cwf/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.77843.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022482872009277344s, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013208389282226562s, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges test.unit.container.test_sharder.TestSharder.test_cleave_root_empty_db_with_ranges ... ok sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00029969215393066406s, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001385211944580078s, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'d' to 'x' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - zero objects found, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'x' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002231597900390625s, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001327991485595703s, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Cleaving ShardRange<'x' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - zero objects found, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp_jz7tvrz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347638.98217.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (0 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaving from row None into .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024580955505371094s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000171661376953125s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.584426879882812e-05s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013947486877441406s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_here_0 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaved ShardRange in 0.03s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Stopped cleave at unready ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, found as of 1717347261.21455>, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002493858337402344s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 9.72747802734375e-05s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00011396408081054688s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 8.96453857421875e-05s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000148773193359375s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_there_1 in state cleaved: 1 successes, 2 required, shard db: /tmp/tmplpksld3b/sda/containers/0/7a6/40d00a528944ce0468e507db329647a6/40d00a528944ce0468e507db329647a6.db, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_1 for ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaving ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - shard db already in sync, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Replicating new shard container .shards_a/c_there_1 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717347261.21455, (3, 6) as of 1717347639.57370, cleaved as of 1717347261.21455> in 0.009s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_2 for ShardRange<'there' to 'where' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025463104248046875s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.942054748535156e-05s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013875961303710938s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_where_2 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717347261.21455, (1, 100) as of 1717347639.64581, cleaved as of 1717347261.21455> in 0.029s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaving from row None into .shards_a/c_yonder_3 for ShardRange<'where' to 'yonder' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002620220184326172s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001049041748046875s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00023174285888671875s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 9.679794311523438e-05s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_yonder_3 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaved ShardRange<'where' to 'yonder' as of 1717347261.21455, (1, 1000) as of 1717347639.74883, cleaved as of 1717347261.21455> in 0.034s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Continuing to cleave (4 done, 0 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Continuing to cleave (4 done, 1 todo), path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'yonder' to MaxBound as of 1717347261.21455, (2, 15) as of 1717347639.79433, active as of 1717347261.21455>, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002465248107910156s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 9.775161743164062e-05s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014400482177734375s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one test.unit.container.test_sharder.TestSharder.test_cleave_root_listing_limit_one ... ok test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change test.unit.container.test_sharder.TestSharder.test_cleave_root_ranges_change ... ok sharder-test INFO: Cleaved ShardRange<'yonder' to MaxBound as of 1717347261.21455, (2, 15) as of 1717347639.79433, active as of 1717347261.21455> in 0.022s, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Cleaving already complete for container, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Passing over already sharded container, path: a/c, db: /tmp/tmplpksld3b/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.25961.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Starting to cleave (3 todo), path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002853870391845703s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001888275146484375s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_d_0 for ShardRange, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Cleaved ShardRange in 0.028s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Cleaving from row None into .shards_a/c_x_1 for ShardRange<'d' to 'x' as of 1717347261.21455, (500000, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002658367156982422s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00019788742065429688s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_x_1 for ShardRange, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Cleaved ShardRange<'d' to 'x' as of 1717347261.21455, (2, 103) as of 1717347640.10905, cleaved as of 1717347261.21455> in 0.037s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: Continuing to cleave (2 done, 1 todo), path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Cleaving from row None into .shards_a/c__2 for ShardRange<'d' to MaxBound as of 1717347640.15046, (500000, 0) as of 1717347640.15046, created as of 1717347640.15046>, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002639293670654297s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016307830810546875s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__2 for ShardRange, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test INFO: Cleaved ShardRange<'d' to MaxBound as of 1717347640.15046, (3, 1103) as of 1717347640.23037, cleaved as of 1717347640.15046> in 0.036s, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmpx_4fr6re/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347639.90986.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023412704467773438s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 9.202957153320312e-05s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016736984252929688s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/c_here_0 in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpapuhd1q9/sda/containers/0/e19/9e88c3bdfd03a830d1fc09d35d671e19/9e88c3bdfd03a830d1fc09d35d671e19.db, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014472007751464844s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00014734268188476562s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 7.843971252441406e-05s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=True) in 9.274482727050781e-05s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test INFO: Cleaving from row None into .shards_a/c_there_0 for ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002689361572265625s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017404556274414062s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_there_0 for ShardRange<'here' to 'where' as of 1717347640.27561, (0, 0) as of 1717347640.27561, sharding as of 1717347640.27561>, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test INFO: Cleaved ShardRange<'here' to 'there' as of 1717347261.21455, (3, 6) as of 1717347640.53727, cleaved as of 1717347261.21455> in 0.039s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023651123046875s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 1 rows (deleted=False) in 9.1552734375e-05s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001900196075439453s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014829635620117188s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001442432403564453s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Continuing to cleave (1 done, 1 todo), path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test INFO: Cleaving from row None into .shards_a/c_where_1 for ShardRange<'there' to 'where' as of 1717347261.21455, (2, 15) as of 1717347640.57101, cleaved as of 1717347261.21455>, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026226043701171875s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015616416931152344s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/c_where_1 for ShardRange<'here' to 'where' as of 1717347640.27561, (0, 0) as of 1717347640.27561, sharding as of 1717347640.27561>, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db test.unit.container.test_sharder.TestSharder.test_cleave_shard test.unit.container.test_sharder.TestSharder.test_cleave_shard ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range test.unit.container.test_sharder.TestSharder.test_cleave_shard_range_no_own_shard_range ... ok sharder-test INFO: Cleaved ShardRange<'there' to 'where' as of 1717347261.21455, (2, 15) as of 1717347640.57101, cleaved as of 1717347261.21455> in 0.038s, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpapuhd1q9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347640.27563.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmpkdgkxwq4/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000217437744140625s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012350082397460938s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013065338134765625s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012731552124023438s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_1_ for ShardRange<'h' to MaxBound as of 1717347640.87737, (1000, 11111) as of 1717347640.87737, created as of 1717347640.87737>, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002682209014892578s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016641616821289062s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_1_ for ShardRange<'h' to 'w' as of 1717347663.00000, (0, 0) as of 1717347652.00000, shrunk as of 1717347652.00000>, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717347640.87737, (1000, 11111) as of 1717347640.87737, created as of 1717347640.87737> in 0.034s, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_1, db: /tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82_1717347653.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/f82/510f8d055bc4c497feaa044595034f82/510f8d055bc4c497feaa044595034f82.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00030517578125s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001289844512939453s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012683868408203125s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001266002655029297s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_2_ for ShardRange<'h' to MaxBound as of 1717347641.10871, (1000, 11111) as of 1717347641.10871, cleaved as of 1717347641.10871>, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003116130828857422s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016641616821289062s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_2_ for ShardRange<'h' to 'w' as of 1717347690.00000, (0, 0) as of 1717347679.00000, shrunk as of 1717347679.00000>, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717347641.10871, (1000, 11111) as of 1717347641.10871, cleaved as of 1717347641.10871> in 0.03s, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_2, db: /tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696_1717347680.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/696/d08603253b97a4fc8389d8100b181696/d08603253b97a4fc8389d8100b181696.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002315044403076172s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012540817260742188s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001289844512939453s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012683868408203125s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_3_ for ShardRange<'h' to MaxBound as of 1717347641.29979, (1000, 11111) as of 1717347641.29979, active as of 1717347641.29979>, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002613067626953125s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013685226440429688s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_3_ for ShardRange<'h' to 'w' as of 1717347717.00000, (0, 0) as of 1717347706.00000, shrunk as of 1717347706.00000>, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test INFO: Cleaved ShardRange<'h' to MaxBound as of 1717347641.29979, (1000, 11111) as of 1717347641.29979, active as of 1717347641.29979> in 0.036s, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_3, db: /tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1_1717347707.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/6c1/8156e3184a7f49e42b9074e4cb3516c1/8156e3184a7f49e42b9074e4cb3516c1.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022101402282714844s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001239776611328125s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001266002655029297s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000125885009765625s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_k for ShardRange<'d' to 'k' as of 1717347641.49453, (1000, 11111) as of 1717347641.49453, created as of 1717347641.49453>, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002465248107910156s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016355514526367188s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_k for ShardRange<'h' to 'w' as of 1717347733.00000, (0, 0) as of 1717347733.00000, shrinking as of 1717347733.00000>, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347641.49453, (1000, 11111) as of 1717347641.49453, created as of 1717347641.49453> in 0.042s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_4_ for ShardRange<'k' to MaxBound as of 1717347641.49460, (1000, 11111) as of 1717347641.49460, created as of 1717347641.49460>, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002663135528564453s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015425682067871094s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_4_ for ShardRange<'h' to 'w' as of 1717347733.00000, (0, 0) as of 1717347733.00000, shrinking as of 1717347733.00000>, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717347641.49460, (1000, 11111) as of 1717347641.49460, created as of 1717347641.49460> in 0.017s, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_4, db: /tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff_1717347734.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/4ff/7ab63f7c14211a36c77395c9374624ff/7ab63f7c14211a36c77395c9374624ff.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00030112266540527344s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013518333435058594s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012493133544921875s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012421607971191406s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_k for ShardRange<'d' to 'k' as of 1717347641.75219, (1000, 11111) as of 1717347641.75219, cleaved as of 1717347641.75219>, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002498626708984375s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016188621520996094s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_k for ShardRange<'h' to 'w' as of 1717347766.00000, (0, 0) as of 1717347766.00000, shrinking as of 1717347766.00000>, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347641.75219, (1000, 11111) as of 1717347641.75219, cleaved as of 1717347641.75219> in 0.023s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_5_ for ShardRange<'k' to MaxBound as of 1717347641.75226, (1000, 11111) as of 1717347641.75226, cleaved as of 1717347641.75226>, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002498626708984375s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015997886657714844s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_5_ for ShardRange<'h' to 'w' as of 1717347766.00000, (0, 0) as of 1717347766.00000, shrinking as of 1717347766.00000>, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717347641.75226, (1000, 11111) as of 1717347641.75226, cleaved as of 1717347641.75226> in 0.018s, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_5, db: /tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c_1717347767.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/e3c/3153d1a13e2a76238abc63bbf5347e3c/3153d1a13e2a76238abc63bbf5347e3c.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002167224884033203s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016880035400390625s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000125885009765625s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001277923583984375s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_k for ShardRange<'d' to 'k' as of 1717347641.97759, (1000, 11111) as of 1717347641.97759, active as of 1717347641.97759>, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026702880859375s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001735687255859375s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_k for ShardRange<'h' to 'w' as of 1717347799.00000, (0, 0) as of 1717347799.00000, shrinking as of 1717347799.00000>, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347641.97759, (1000, 11111) as of 1717347641.97759, active as of 1717347641.97759> in 0.024s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_6_ for ShardRange<'k' to MaxBound as of 1717347641.97767, (1000, 11111) as of 1717347641.97767, active as of 1717347641.97767>, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002796649932861328s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016236305236816406s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_6_ for ShardRange<'h' to 'w' as of 1717347799.00000, (0, 0) as of 1717347799.00000, shrinking as of 1717347799.00000>, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test INFO: Cleaved ShardRange<'k' to MaxBound as of 1717347641.97767, (1000, 11111) as of 1717347641.97767, active as of 1717347641.97767> in 0.021s, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_6, db: /tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9_1717347800.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/3c9/127479cb150cd6d703c13aa1fec193c9/127479cb150cd6d703c13aa1fec193c9.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002193450927734375s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012731552124023438s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001285076141357422s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019121170043945312s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_k for ShardRange<'d' to 'k' as of 1717347642.21469, (1000, 11111) as of 1717347642.21469, cleaved as of 1717347642.21469>, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002532005310058594s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001404285430908203s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_k for ShardRange<'h' to 'w' as of 1717347832.00000, (0, 0) as of 1717347832.00000, shrinking as of 1717347832.00000>, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347642.21469, (1000, 11111) as of 1717347642.21469, cleaved as of 1717347642.21469> in 0.02s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_t for ShardRange<'k' to 't' as of 1717347642.21479, (1000, 11111) as of 1717347642.21479, cleaved as of 1717347642.21479>, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002453327178955078s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013947486877441406s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_t for ShardRange<'h' to 'w' as of 1717347832.00000, (0, 0) as of 1717347832.00000, shrinking as of 1717347832.00000>, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347642.21479, (1000, 11111) as of 1717347642.21479, cleaved as of 1717347642.21479> in 0.019s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_7_ for ShardRange<'t' to MaxBound as of 1717347642.21482, (1000, 11111) as of 1717347642.21482, cleaved as of 1717347642.21482>, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000339508056640625s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018835067749023438s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_7_ for ShardRange<'h' to 'w' as of 1717347832.00000, (0, 0) as of 1717347832.00000, shrinking as of 1717347832.00000>, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717347642.21482, (1000, 11111) as of 1717347642.21482, cleaved as of 1717347642.21482> in 0.509s, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_7, db: /tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175_1717347833.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/175/be6981a6ebce34402af1287bbc13f175/be6981a6ebce34402af1287bbc13f175.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022268295288085938s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001246929168701172s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014925003051757812s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001327991485595703s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_k for ShardRange<'d' to 'k' as of 1717347643.16384, (1000, 11111) as of 1717347643.16384, created as of 1717347643.16384>, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002579689025878906s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001659393310546875s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_k for ShardRange<'h' to 'w' as of 1717347871.00000, (0, 0) as of 1717347871.00000, shrinking as of 1717347871.00000>, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347643.16384, (1000, 11111) as of 1717347643.16384, created as of 1717347643.16384> in 0.022s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_t for ShardRange<'k' to 't' as of 1717347643.16394, (1000, 11111) as of 1717347643.16394, created as of 1717347643.16394>, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002548694610595703s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001621246337890625s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_t for ShardRange<'h' to 'w' as of 1717347871.00000, (0, 0) as of 1717347871.00000, shrinking as of 1717347871.00000>, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347643.16394, (1000, 11111) as of 1717347643.16394, created as of 1717347643.16394> in 0.02s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_8_ for ShardRange<'t' to MaxBound as of 1717347643.16397, (1000, 11111) as of 1717347643.16397, created as of 1717347643.16397>, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002727508544921875s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001533031463623047s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_8_ for ShardRange<'h' to 'w' as of 1717347871.00000, (0, 0) as of 1717347871.00000, shrinking as of 1717347871.00000>, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717347643.16397, (1000, 11111) as of 1717347643.16397, created as of 1717347643.16397> in 0.02s, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_8, db: /tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25_1717347872.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/b25/cf40e8755399f5ee10bd8bdf3e03ab25/cf40e8755399f5ee10bd8bdf3e03ab25.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002491474151611328s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001277923583984375s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001251697540283203s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012636184692382812s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Starting to cleave (3 todo), path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_k for ShardRange<'d' to 'k' as of 1717347643.48095, (1000, 11111) as of 1717347643.48095, active as of 1717347643.48095>, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025773048400878906s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017380714416503906s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_k for ShardRange<'h' to 'w' as of 1717347910.00000, (0, 0) as of 1717347910.00000, shrinking as of 1717347910.00000>, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347643.48095, (1000, 11111) as of 1717347643.48095, active as of 1717347643.48095> in 0.019s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_t for ShardRange<'k' to 't' as of 1717347643.48102, (1000, 11111) as of 1717347643.48102, active as of 1717347643.48102>, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025177001953125s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000156402587890625s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_t for ShardRange<'h' to 'w' as of 1717347910.00000, (0, 0) as of 1717347910.00000, shrinking as of 1717347910.00000>, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347643.48102, (1000, 11111) as of 1717347643.48102, active as of 1717347643.48102> in 0.021s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_9_ for ShardRange<'t' to MaxBound as of 1717347643.48105, (1000, 11111) as of 1717347643.48105, active as of 1717347643.48105>, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.004772663116455078s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001666545867919922s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_9_ for ShardRange<'h' to 'w' as of 1717347910.00000, (0, 0) as of 1717347910.00000, shrinking as of 1717347910.00000>, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test INFO: Cleaved ShardRange<'t' to MaxBound as of 1717347643.48105, (1000, 11111) as of 1717347643.48105, active as of 1717347643.48105> in 0.026s, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: Cleaved 3 shard ranges, path: .shards_a/donor_9, db: /tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7_1717347911.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpzltdg07z/sda/containers/0/4a7/206c4587690950a47da43921490ac4a7/206c4587690950a47da43921490ac4a7.db' sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021958351135253906s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013017654418945312s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000133514404296875s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013446807861328125s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_10_k for ShardRange<'d' to 'k' as of 1717347643.82041, (1000, 11111) as of 1717347643.82041, created as of 1717347643.82041>, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002739429473876953s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001747608184814453s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_10_k for ShardRange<'h' to 'w' as of 1717347949.00000, (0, 0) as of 1717347949.00000, shrinking as of 1717347949.00000>, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347643.82041, (1000, 11111) as of 1717347643.82041, created as of 1717347643.82041> in 0.02s, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '3cf972f8-2763-4830-8859-206e5e7d665c-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 0}, path: .shards_a/donor_10, db: /tmp/tmpzltdg07z/sda/containers/0/a44/457ca7b014f2e631df7ba91ed2de5a44/457ca7b014f2e631df7ba91ed2de5a44_1717347950.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002281665802001953s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015020370483398438s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001304149627685547s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013446807861328125s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_k for ShardRange<'d' to 'k' as of 1717347644.00184, (1000, 11111) as of 1717347644.00184, cleaved as of 1717347644.00184>, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00038552284240722656s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023627281188964844s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_k for ShardRange<'h' to 'w' as of 1717347975.00000, (0, 0) as of 1717347975.00000, shrinking as of 1717347975.00000>, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347644.00184, (1000, 11111) as of 1717347644.00184, cleaved as of 1717347644.00184> in 0.023s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test INFO: Cleaving from row None into .shards_a/acceptor_11_t for ShardRange<'k' to 't' as of 1717347644.00192, (1000, 11111) as of 1717347644.00192, cleaved as of 1717347644.00192>, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00035643577575683594s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002200603485107422s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_11_t for ShardRange<'h' to 'w' as of 1717347975.00000, (0, 0) as of 1717347975.00000, shrinking as of 1717347975.00000>, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test INFO: Cleaved ShardRange<'k' to 't' as of 1717347644.00192, (1000, 11111) as of 1717347644.00192, cleaved as of 1717347644.00192> in 0.023s, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: Cleaved 2 shard ranges, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '1814da5c-2088-43c6-8a30-78cf246afe32-sda', 'cursor': 't', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 2, 'ranges_todo': 0}, path: .shards_a/donor_11, db: /tmp/tmpzltdg07z/sda/containers/0/254/300bf0ec46a195b626429b8fc0a35254/300bf0ec46a195b626429b8fc0a35254_1717347976.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023603439331054688s, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014853477478027344s, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001308917999267578s, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002243518829345703s, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'b40b95de-7a6a-44d1-acdc-2b84be18106f-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 1}, path: .shards_a/donor_12, db: /tmp/tmpzltdg07z/sda/containers/0/098/d625ed53f61d107bf15bc92e1e76f098/d625ed53f61d107bf15bc92e1e76f098_1717348008.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022411346435546875s, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012421607971191406s, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001289844512939453s, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012731552124023438s, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test INFO: Stopped cleave at gap: 'h' - 'k', path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '69a3f50c-6b17-452f-8e5c-7fab120cbc81-sda', 'cursor': 'h', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 2}, path: .shards_a/donor_13, db: /tmp/tmpzltdg07z/sda/containers/0/d3c/f437419f67d9f0514f459e6a3f90ad3c/f437419f67d9f0514f459e6a3f90ad3c_1717348028.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'h'), ('w', '')], path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021505355834960938s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012445449829101562s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013017654418945312s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012826919555664062s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking test.unit.container.test_sharder.TestSharder.test_cleave_shard_shrinking ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_active_root_range ... ok sharder-test INFO: Cleaving from row None into .shards_a/acceptor_14_k for ShardRange<'d' to 'k' as of 1717347645.66687, (1000, 11111) as of 1717347645.66687, cleaved as of 1717347645.66687>, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003376007080078125s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022172927856445312s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739.db sharder-test INFO: Replicating new shard container .shards_a/acceptor_14_k for ShardRange<'h' to 'w' as of 1717348047.00000, (0, 0) as of 1717348047.00000, shrinking as of 1717348047.00000>, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test INFO: Cleaved ShardRange<'d' to 'k' as of 1717347645.66687, (1000, 11111) as of 1717347645.66687, cleaved as of 1717347645.66687> in 0.041s, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test INFO: Stopped cleave at gap: 'k' - 't', path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'e4ae21d4-5295-4ef9-9903-92781dede4fc-sda', 'cursor': 'k', 'max_row': 3, 'cleave_to_row': 3, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 1, 'ranges_todo': 1}, path: .shards_a/donor_14, db: /tmp/tmpzltdg07z/sda/containers/0/739/a9dfb0212010e3d1ecef4bf40179b739/a9dfb0212010e3d1ecef4bf40179b739_1717348048.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003871917724609375s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00020194053649902344s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00025081634521484375s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022459030151367188s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test INFO: Cleaving from row None into .shards/other for ShardRange<'here' to 'where' as of 1717347648.00000, (0, 0) as of 1717347648.00000, active as of 1717347648.00000>, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003638267517089844s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001690387725830078s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards/other for ShardRange<'here' to 'there' as of 1717347646.01998, (0, 0) as of 1717347646.00000, shrunk as of 1717347646.00000>, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test INFO: Cleaved ShardRange<'here' to 'where' as of 1717347648.00000, (0, 0) as of 1717347648.00000, active as of 1717347648.00000> in 0.032s, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpspbtp4l9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000213623046875s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001232624053955078s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00012421607971191406s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001251697540283203s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test INFO: Cleaving from row None into .shards_a/shard_d for ShardRange<'here' to MaxBound as of 1717347649.00000, (0, 0) as of 1717347649.00000, active as of 1717347649.00000>, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_acceptor_with_sharded_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range ... ok test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor test.unit.container.test_sharder.TestSharder.test_cleave_shrinking_to_active_root_range_with_active_acceptor ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002543926239013672s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015687942504882812s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container .shards_a/shard_d for ShardRange<'here' to 'there' as of 1717347647.00000, (0, 0) as of 1717347647.00000, sharding as of 1717347647.00000>, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test INFO: Cleaved ShardRange<'here' to MaxBound as of 1717347649.00000, (0, 0) as of 1717347649.00000, active as of 1717347649.00000> in 0.019s, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmp2bztk01u/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002186298370361328s, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013971328735351562s, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018978118896484375s, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001308917999267578s, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test INFO: Starting to cleave (1 todo), path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002219676971435547s, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012493133544921875s, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpz6ipgi5i/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347647.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('there', '')], path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021219253540039062s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001366138458251953s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00015425682067871094s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001556873321533203s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test INFO: Starting to cleave (2 todo), path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test INFO: Cleaving from row None into a/c for ShardRange, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002465248107910156s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015091896057128906s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b.db sharder-test INFO: Replicating new shard container a/c for ShardRange<'here' to 'there' as of 1717347646.59770, (0, 0) as of 1717347647.00000, shrunk as of 1717347647.00000>, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test INFO: Cleaved ShardRange in 0.028s, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: Cleaved 1 shard ranges, path: .shards_a/shard_c, db: /tmp/tmpn2d6olu9/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347648.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found test.unit.container.test_sharder.TestSharder.test_cleave_skips_shrinking_and_stops_at_found ... ok sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaving from row None into .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00035953521728515625s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021195411682128906s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_b_0 for ShardRange, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaved ShardRange in 0.033s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaving from row None into .shards_a/c_d_2 for ShardRange<'b' to 'd' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002655982971191406s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001614093780517578s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_d_2 for ShardRange, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaved ShardRange<'b' to 'd' as of 1717347261.21455, (2, 3) as of 1717347647.07590, cleaved as of 1717347261.21455> in 0.032s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Continuing to cleave (2 done, 2 todo), path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaving from row None into .shards_a/c_f_3 for ShardRange<'d' to 'f' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0004253387451171875s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023317337036132812s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_f_3 for ShardRange, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaved ShardRange<'d' to 'f' as of 1717347261.21455, (2, 103) as of 1717347647.18394, cleaved as of 1717347261.21455> in 0.044s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Stopped cleave at unready ShardRange<'f' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, found as of 1717347261.21455>, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaving from row None into .shards_a/c__4 for ShardRange<'f' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347647.22097>, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00033974647521972656s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024509429931640625s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__4 for ShardRange, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test INFO: Cleaved ShardRange<'f' to MaxBound as of 1717347261.21455, (1, 1000) as of 1717347647.32155, cleaved as of 1717347647.22097> in 0.06s, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp_8moyudx/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347646.63245.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics test.unit.container.test_sharder.TestSharder.test_cleave_timing_metrics ... ok sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00035190582275390625s, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.0002665519714355469s, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj_004_0 for ShardRange, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test INFO: Cleaved ShardRange in 0.043s, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj_004' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0035698413848876953s, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00018930435180664062s, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test INFO: Cleaved ShardRange<'obj_004' to MaxBound as of 1717347261.21455, (1, 18432) as of 1717347647.61713, cleaved as of 1717347261.21455> in 0.046s, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmpjoy0g7md/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.38628.db REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, sync, (5, 'fbb04eeb78c2f5488c1e03480af5b77e', '3552b350-d28a-4d15-983f-9871330bade6-sdb', '1717347647.69748', '0000000000.00000', '0', '') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_items, ([{'ROWID': 1, 'name': 'obj002', 'created_at': '1717347649.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 2, 'name': 'obj003', 'created_at': '1717347650.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 3, 'name': 'obj004', 'created_at': '1717347651.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 4, 'name': 'obj005', 'created_at': '1717347652.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}, {'ROWID': 5, 'name': 'obj006', 'created_at': '1717347653.00000', 'size': 1, 'content_type': 'text/plain', 'etag': 'etag', 'deleted': 0, 'storage_policy_index': 0}], '3552b350-d28a-4d15-983f-9871330bade6-sdb') REPLICATE: /sda/0/4a8a08f09d37b73795649038408b5f33, merge_syncs, ([],) sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0003173351287841797s, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014829635620117188s, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaved ShardRange in 0.031s, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00030517578125s, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016355514526367188s, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717347261.21455, (3, 3) as of 1717347648.02578, cleaved as of 1717347261.21455> in 0.034s, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp5pt12qa7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, sync, (7, '00000000000000000000000000000000', 'cd8d93ae-8778-4e4b-9340-0a71233a1b47-', '1717347647.67063', '0000000000.00000', '0', '{"X-Container-Sysmeta-Shard-Context-18b82c25-77d1-45a7-add0-b7691a14304e-sda": ["{\\"ref\\": \\"18b82c25-77d1-45a7-add0-b7691a14304e-sda\\", \\"cursor\\": \\"\\", \\"max_row\\": 7, \\"cleave_to_row\\": 7, \\"last_cleave_to_row\\": null, \\"cleaving_done\\": true, \\"misplaced_done\\": true, \\"ranges_done\\": 2, \\"ranges_todo\\": 0}", "1717347648.05100"]}') REPLICATE: /sdb/0/4a8a08f09d37b73795649038408b5f33, merge_shard_ranges, ([{'name': '.shards_a/c_obj004_0', 'timestamp': '1717347261.21455', 'lower': '', 'upper': 'obj004', 'object_count': 4, 'bytes_used': 4, 'meta_timestamp': '1717347647.93853', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347261.21455', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': '.shards_a/c__1', 'timestamp': '1717347261.21455', 'lower': 'obj004', 'upper': '', 'object_count': 3, 'bytes_used': 3, 'meta_timestamp': '1717347648.02578', 'deleted': 0, 'state': 40, 'state_timestamp': '1717347261.21455', 'epoch': None, 'reported': 0, 'tombstones': -1}, {'name': 'a/c', 'timestamp': '1717347647.77675', 'lower': '', 'upper': '', 'object_count': 0, 'bytes_used': 0, 'meta_timestamp': '1717347648.06388', 'deleted': 0, 'state': 70, 'state_timestamp': '1717347647.77681', 'epoch': '1717347647.77681', 'reported': 0, 'tombstones': -1}], 'cd8d93ae-8778-4e4b-9340-0a71233a1b47-') sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026798248291015625s, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015044212341308594s, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_0 for ShardRange, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaved ShardRange in 0.04s, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'obj004' to MaxBound as of 1717347261.21455, (3, 3) as of 1717347648.02578, active as of 1717347261.21455>, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00030159950256347656s, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001723766326904297s, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db test.unit.container.test_sharder.TestSharder.test_cleave_to_existing_shard_db ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range test.unit.container.test_sharder.TestSharder.test_complete_sharding_missing_own_shard_range ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_root test.unit.container.test_sharder.TestSharder.test_complete_sharding_root ... ok test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard test.unit.container.test_sharder.TestSharder.test_complete_sharding_shard ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_data ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_bad_record_type ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_ok ... ok test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response test.unit.container.test_sharder.TestSharder.test_fetch_shard_ranges_unexpected_response ... ok sharder-test INFO: Replicating new shard container .shards_a/c__1 for ShardRange, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test INFO: Cleaved ShardRange<'obj004' to MaxBound as of 1717347261.21455, (3, 3) as of 1717347648.02578, active as of 1717347261.21455> in 0.021s, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp5pt12qa7/sdb/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347647.77681.db sharder-test WARNING: Failed to get own_shard_range, path: a/c, db: /tmp/tmprqhh5k3o/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347648.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '359c8ab9-d848-4226-8725-b4ae5b88026e-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpxabut_y6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347648.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '359c8ab9-d848-4226-8725-b4ae5b88026e-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpxabut_y6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347648.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '359c8ab9-d848-4226-8725-b4ae5b88026e-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpxabut_y6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347648.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '359c8ab9-d848-4226-8725-b4ae5b88026e-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpxabut_y6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347648.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'cf8e673b-e238-482e-8ba1-971cdc158a62-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpxabut_y6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347648.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '52d8ceb5-6b08-483f-a8fb-eaa0de983afe-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnh98p7hn/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347649.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '52d8ceb5-6b08-483f-a8fb-eaa0de983afe-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnh98p7hn/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347649.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '52d8ceb5-6b08-483f-a8fb-eaa0de983afe-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnh98p7hn/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347649.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '52d8ceb5-6b08-483f-a8fb-eaa0de983afe-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnh98p7hn/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347649.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '69df32e0-a236-4f10-ad53-df02018e5cb3-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: .shards_/shard_c, db: /tmp/tmpnh98p7hn/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347649.00000.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: TypeError('the JSON object must be str, bytes or bytearray, not dict'), path: a/c, db: /tmp/tmpdz_kiqxk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: JSONDecodeError('Expecting value: line 1 column 1 (char 0)'), path: a/c, db: /tmp/tmpdz_kiqxk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: ValueError('not a list'), path: a/c, db: /tmp/tmpdz_kiqxk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('name'), path: a/c, db: /tmp/tmpdz_kiqxk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('timestamp'), path: a/c, db: /tmp/tmpdz_kiqxk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: invalid data: KeyError('object_count'), path: a/c, db: /tmp/tmpdz_kiqxk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type None, path: a/c, db: /tmp/tmpi_xqacpb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'object', path: a/c, db: /tmp/tmpi_xqacpb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to get shard ranges from a/c: unexpected record type 'disco', path: a/c, db: /tmp/tmpi_xqacpb/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to get shard ranges from a/c: Unexpected response: 404, path: a/c, db: /tmp/tmp6etbv8iq/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpl3sfz3vo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_multiple_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap test.unit.container.test_sharder.TestSharder.test_find_and_enable_sharding_candidates_bootstrap ... ok test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates test.unit.container.test_sharder.TestSharder.test_find_and_enable_shrinking_candidates ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_three_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_root ... ok sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmpl3sfz3vo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347650.00000.db sharder-test DEBUG: Found 2 compactible sequences of length(s) [2, 2], path: a/c, db: /tmp/tmpl3sfz3vo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347650.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmpl3sfz3vo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347650.00000.db sharder-test DEBUG: shrinking 99999 objects from 1 shard ranges into ShardRange<'c' to 'e' as of 1717347650.90559, (100000, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmpl3sfz3vo/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347650.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7abkhs22/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp7abkhs22/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347650.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp7abkhs22/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347650.00000.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmpmmwe8hi6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Found 0 compactible sequences of length(s) [], path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: shrinking 99998 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: Found 1 compactible sequences of length(s) [2], path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test DEBUG: shrinking 0 objects from 1 shard ranges into ShardRange, path: a/c, db: /tmp/tmp__v7kz_0/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347651.00000.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp79ctcsnr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp79ctcsnr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp79ctcsnr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: a/c, db: /tmp/tmp79ctcsnr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp79ctcsnr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmp79ctcsnr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp2mj8_74i/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp2mj8_74i/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp2mj8_74i/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 1 found, path: .shards_a/c_, db: /tmp/tmp2mj8_74i/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp2mj8_74i/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmp2mj8_74i/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpqpqe9ndh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpqpqe9ndh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpqpqe9ndh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpqpqe9ndh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmpzekiewt5/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmpzekiewt5/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_root ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_none_found_shard ... ok test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size test.unit.container.test_sharder.TestSharder.test_find_shard_ranges_with_minimum_size ... ok test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part test.unit.container.test_sharder.TestSharder.test_get_shard_broker_no_local_handoff_for_part ... ok test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_candidate ... ok sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmpzekiewt5/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmpzekiewt5/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpkvmqgxhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmpkvmqgxhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpkvmqgxhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmpkvmqgxhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpw60wtwhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpw60wtwhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmpw60wtwhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmpw60wtwhh/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmp_8rfxfrk/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmp_8rfxfrk/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmp_8rfxfrk/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate test.unit.container.test_sharder.TestSharder.test_identify_sharding_old_style_candidate ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_sharding_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log test.unit.container.test_sharder.TestSharder.test_incomplete_shrinking_progress_warning_log ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats test.unit.container.test_sharder.TestSharder.test_increment_stats ... ok test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd test.unit.container.test_sharder.TestSharder.test_increment_stats_with_statsd ... ok test.unit.container.test_sharder.TestSharder.test_init test.unit.container.test_sharder.TestSharder.test_init ... ok test.unit.container.test_sharder.TestSharder.test_init_deprecated_options test.unit.container.test_sharder.TestSharder.test_init_deprecated_options ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error test.unit.container.test_sharder.TestSharder.test_init_internal_client_conf_loading_error ... ok test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name test.unit.container.test_sharder.TestSharder.test_init_internal_client_log_name ... ok test.unit.container.test_sharder.TestSharder.test_log_broker test.unit.container.test_sharder.TestSharder.test_log_broker ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception test.unit.container.test_sharder.TestSharder.test_log_broker_exception ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging test.unit.container.test_sharder.TestSharder.test_log_broker_exception_while_logging ... ok test.unit.container.test_sharder.TestSharder.test_log_broker_levels test.unit.container.test_sharder.TestSharder.test_log_broker_levels ... ok sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:00:52 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test WARNING: Repeat cleaving required, context: {'ref': '7aced48c-8a4b-40e0-bba4-4e9f5bb15e88-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpnexrrsik/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347652.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T17:00:52.000000. DB state: sharding, own_shard_range state: sharding, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 2, 'active': 0}, path: a/c, db: /tmp/tmpnexrrsik/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347652.00000.db sharder-test WARNING: Cleaving has not completed in 172801.00 seconds since 2024-06-02T17:00:54.000000. DB state: sharding, own_shard_range state: shrinking, state count of shard ranges: {'found': 0, 'created': 0, 'cleaved': 0, 'active': 1}, path: .shards_a/shard_c, db: /tmp/tmpev1x3skv/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347654.00000.db sharder-test WARNING: shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 4 exceeds replica count 3, reducing to 3 sharder-test DEBUG: bonjour mes amis, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test DEBUG: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: bonjour mes amis, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test INFO: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: bonjour mes amis, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test WARNING: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: bonjour mes amis, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: hello my friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: greetings friend%04ds, path: a/c%40d, db: /tmp/tmpykgl0qos/sda/containers/0/3b6/58f9120a67d9f5f79bd06fc8a7bda3b6/58f9120a67d9f5f79bd06fc8a7bda3b6.db sharder-test ERROR: exception: test, path: a/c, db: /tmp/tmpkhlc256h/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 448, in test_log_broker_exception raise ValueError('test') ValueError: test sharder-test DEBUG: bonjour mes amis, path: , db: /tmp/tmputn3kik1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: bonjour mes amis, path: a/c, db: sharder-test INFO: bonjour mes amis, path: , db: /tmp/tmputn3kik1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: bonjour mes amis, path: a/c, db: sharder-test WARNING: bonjour mes amis, path: , db: /tmp/tmputn3kik1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: bonjour mes amis, path: a/c, db: sharder-test ERROR: bonjour mes amis, path: , db: /tmp/tmputn3kik1/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: bonjour mes amis, path: a/c, db: sharder-test DEBUG: Unlinked retiring db '/tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00024962425231933594s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022649765014648438s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002536773681640625s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002512931823730469s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00028896331787109375s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018334388732910156s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018835067749023438s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016570091247558594s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed test.unit.container.test_sharder.TestSharder.test_misplaced_objects_db_id_changed ... ok sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018167495727539062s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00031685829162597656s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002288818359375s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347653.32193, (0, 0) as of 1717347653.32193, active as of 1717347653.32193>, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347653.32195, (0, 0) as of 1717347653.32195, active as of 1717347653.32195>, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/there-where in state active, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347653.32198, (0, 0) as of 1717347653.32198, active as of 1717347653.32198>, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test WARNING: Refused to remove misplaced objects for dest .shards_a/where-yonder in state active, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018715858459472656s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001690387725830078s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016379356384277344s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016832351684570312s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016808509826660156s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001773834228515625s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003025531768798828s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002396106719970703s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00025582313537597656s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019502639770507812s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016880035400390625s, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347653.32195, (0, 0) as of 1717347653.32195, active as of 1717347653.32195>, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347653.32198, (0, 0) as of 1717347653.32198, active as of 1717347653.32198>, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpww8a1s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.33314.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00026869773864746094s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002338886260986328s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022292137145996094s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00023317337036132812s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00032067298889160156s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00024175643920898438s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018286705017089844s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018072128295898438s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001995563507080078s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001876354217529297s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018405914306640625s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001876354217529297s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002105236053466797s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016736984252929688s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018262863159179688s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001842975616455078s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00018167495727539062s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00022459030151367188s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018739700317382812s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001842975616455078s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016546249389648438s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001647472381591797s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated test.unit.container.test_sharder.TestSharder.test_misplaced_objects_deleted_and_updated ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002300739288330078s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018453598022460938s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018310546875s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001697540283203125s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016498565673828125s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021886825561523438s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002167224884033203s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018906593322753906s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001690387725830078s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016736984252929688s, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Placed 1 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3ev4_p4k/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347653.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0009996891021728516s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022840499877929688s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00031113624572753906s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00025010108947753906s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00032138824462890625s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001850128173828125s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019240379333496094s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016570091247558594s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017714500427246094s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003216266632080078s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021839141845703125s, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347654.40853, (0, 0) as of 1717347654.40853, active as of 1717347654.40853>, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347654.40856, (0, 0) as of 1717347654.40856, active as of 1717347654.40856>, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_2_replicas ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_3_replicas ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347654.40858, (0, 0) as of 1717347654.40858, active as of 1717347654.40858>, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 1 required (not removing objects), shard db: /tmp/tmpt5euquy6/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpt5euquy6/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347654.41841.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002627372741699219s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022125244140625s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022459030151367188s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002455711364746094s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002560615539550781s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001766681671142578s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001747608184814453s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016427040100097656s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001862049102783203s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002598762512207031s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022745132446289062s, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347658.19893, (0, 0) as of 1717347658.19893, active as of 1717347658.19893>, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347658.19895, (0, 0) as of 1717347658.19895, active as of 1717347658.19895>, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/there-where in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmpi5x3rz0m/sda/containers/0/170/533f29a8ddb31e55151765e09ccbb170/533f29a8ddb31e55151765e09ccbb170.db, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347658.19898, (0, 0) as of 1717347658.19898, active as of 1717347658.19898>, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpi5x3rz0m/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.21001.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00029587745666503906s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002205371856689453s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023102760314941406s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00028967857360839844s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002522468566894531s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017595291137695312s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001780986785888672s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001652240753173828s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas test.unit.container.test_sharder.TestSharder.test_misplaced_objects_insufficient_replication_4_replicas ... ok sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017452239990234375s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003228187561035156s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002307891845703125s, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347658.45778, (0, 0) as of 1717347658.45778, active as of 1717347658.45778>, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/here-there in state active: 1 successes, 2 required (not removing objects), shard db: /tmp/tmp1v4cmtat/sda/containers/0/ed5/9b40bf3cdc484061806b8d0a545d2ed5/9b40bf3cdc484061806b8d0a545d2ed5.db, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347658.45780, (0, 0) as of 1717347658.45780, active as of 1717347658.45780>, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347658.45783, (0, 0) as of 1717347658.45783, active as of 1717347658.45783>, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test WARNING: Failed to sufficiently replicate misplaced objects shard .shards_a/where-yonder in state active: 0 successes, 2 required (not removing objects), shard db: /tmp/tmp1v4cmtat/sda/containers/0/e50/07b20f6b08408fde4e0bdce650938e50/07b20f6b08408fde4e0bdce650938e50.db, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp1v4cmtat/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.46525.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002503395080566406s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002696514129638672s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002295970916748047s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002636909484863281s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002582073211669922s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017976760864257812s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017642974853515625s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001773834228515625s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00019121170043945312s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024580955505371094s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002219676971435547s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347658.87621, (0, 0) as of 1717347658.87621, active as of 1717347658.87621>, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347658.87623, (0, 0) as of 1717347658.87623, active as of 1717347658.87623>, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347658.87626, (0, 0) as of 1717347658.87626, active as of 1717347658.87626>, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017952919006347656s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001671314239501953s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001647472381591797s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects test.unit.container.test_sharder.TestSharder.test_misplaced_objects_newer_objects ... ok sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017523765563964844s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00020694732666015625s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017023086547851562s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016307830810546875s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001652240753173828s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001633167266845703s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000209808349609375s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000171661376953125s, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347658.87621, (0, 0) as of 1717347658.87621, active as of 1717347658.87621>, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpfvc1ivce/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347658.88234.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003108978271484375s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016427040100097656s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there')], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000308990478515625s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002238750457763672s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022602081298828125s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002551078796386719s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002524852752685547s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'yonder')], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002129077911376953s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002770423889160156s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container test.unit.container.test_sharder.TestSharder.test_misplaced_objects_root_container ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.000186920166015625s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016736984252929688s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001728534698486328s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002110004425048828s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028252601623535156s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002548694610595703s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001652240753173828s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00017714500427246094s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023889541625976562s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00029349327087402344s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017881393432617188s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023746490478515625s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00023317337036132812s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00036215782165527344s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002474784851074219s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002696514129638672s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002658367156982422s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00017762184143066406s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00017309188842773438s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003914833068847656s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00033164024353027344s, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'yonder' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Placed 3 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpqx5i87bs/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347659.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002486705780029297s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00019311904907226562s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022029876708984375s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002288818359375s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00022172927856445312s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002281665802001953s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003101825714111328s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002498626708984375s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Placed 1 misplaced objects (2 unplaced), path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001938343048095703s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003554821014404297s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018143653869628906s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002951622009277344s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0008642673492431641s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: misplaced object source bounds [('', 'there'), ('where', '')], path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000335693359375s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028133392333984375s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002541542053222656s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0004153251647949219s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003676414489746094s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00031256675720214844s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000293731689453125s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_sharding ... ok sharder-test DEBUG: got 0 rows (deleted=True) in 0.00027060508728027344s, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Placed 5 misplaced objects (0 unplaced), path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/shard_c, db: /tmp/tmpkrssei2h/sda/containers/0/f3b/b3fac948f9f029603d4248b5e2d39f3b/b3fac948f9f029603d4248b5e2d39f3b_1717347660.00000.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00034165382385253906s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.010035991668701172s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001900196075439453s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018596649169921875s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021219253540039062s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00021791458129882812s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002732276916503906s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002505779266357422s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002536773681640625s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002677440643310547s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002498626708984375s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023651123046875s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00026106834411621094s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0004658699035644531s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001971721649169922s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016617774963378906s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016260147094726562s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded ... ok sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001571178436279297s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016045570373535156s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002970695495605469s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0007293224334716797s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018024444580078125s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002086162567138672s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003457069396972656s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00018334388732910156s, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp_js_at_y/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024366378784179688s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014162063598632812s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00013709068298339844s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001361370086669922s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018787384033203125s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021409988403320312s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00034737586975097656s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001938343048095703s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00027871131896972656s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001888275146484375s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016808509826660156s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001552104949951172s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00019550323486328125s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001537799835205078s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_one ... ok sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002117156982421875s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001575946807861328s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000171661376953125s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00016641616821289062s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.000194549560546875s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00021123886108398438s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001957416534423828s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016736984252929688s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001556873321533203s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00024318695068359375s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015234947204589844s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018668174743652344s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001666545867919922s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00018310546875s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00019979476928710938s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018358230590820312s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001671314239501953s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001804828643798828s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015544891357421875s, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmp37y4n6y4/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00023508071899414062s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014257431030273438s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00014352798461914062s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013709068298339844s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018739700317382812s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0001900196075439453s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00027370452880859375s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00015687942504882812s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00017952919006347656s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0008120536804199219s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016760826110839844s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test WARNING: Failed to find destination for at least 2 misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (2 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016069412231445312s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016117095947265625s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003650188446044922s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002155303955078125s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0002727508544921875s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002262592315673828s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0004467964172363281s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00034499168395996094s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002205371856689453s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Looking for misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: misplaced object source bounds [('', 'here'), ('where', '')], path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002562999725341797s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.000385284423828125s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002346038818359375s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two test.unit.container.test_sharder.TestSharder.test_misplaced_objects_shard_container_unsharded_limit_two ... ok test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication test.unit.container.test_sharder.TestSharder.test_misplaced_objects_sufficient_replication ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_root ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_finds_two_shard ... ok test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_root ... ok sharder-test DEBUG: moving misplaced objects found in range ShardRange, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00032830238342285156s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00023937225341796875s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0002949237823486328s, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, active as of 1717347261.21455>, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Placed 2 misplaced objects (0 unplaced), path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Finished handling misplaced objects, path: .shards_a/.shard_c, db: /tmp/tmpcd0k3hh_/sda/containers/0/15e/acd61f6ec4bfa17987465dba0a96315e/acd61f6ec4bfa17987465dba0a96315e.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: misplaced object source bounds [('', '')], path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0003249645233154297s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00028634071350097656s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00036263465881347656s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003666877746582031s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003743171691894531s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00018334388732910156s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0003104209899902344s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00016689300537109375s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001838207244873047s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0003943443298339844s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00023055076599121094s, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'here' to 'there' as of 1717347661.50735, (0, 0) as of 1717347661.50735, active as of 1717347661.50735>, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'there' to 'where' as of 1717347661.50739, (0, 0) as of 1717347661.50739, active as of 1717347661.50739>, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: moving misplaced objects found in range ShardRange<'where' to 'yonder' as of 1717347661.50741, (0, 0) as of 1717347661.50741, active as of 1717347661.50741>, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: Placed 4 misplaced objects (0 unplaced), path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp4jvwwbhk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347661.51458.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpckhigrd_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmpckhigrd_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmpckhigrd_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Scan for shard ranges already completed, path: a/c, db: /tmp/tmpckhigrd_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c_, db: /tmp/tmpoidh67dp/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: .shards_a/c_, db: /tmp/tmpoidh67dp/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Final shard range reached., path: .shards_a/c_, db: /tmp/tmpoidh67dp/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test DEBUG: Scan for shard ranges already completed, path: .shards_a/c_, db: /tmp/tmpoidh67dp/sda/containers/0/cce/04aa2bd2d2fcbc0614f53f00b2b85cce/04aa2bd2d2fcbc0614f53f00b2b85cce.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpg68yvdu7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmpg68yvdu7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmpg68yvdu7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: a/c, db: /tmp/tmpg68yvdu7/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard test.unit.container.test_sharder.TestSharder.test_old_style_find_shard_ranges_none_found_shard ... ok sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmp2qlhkclz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmp2qlhkclz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: .shards_a/c, db: /tmp/tmp2qlhkclz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: No shard ranges found, path: .shards_a/c, db: /tmp/tmp2qlhkclz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:2 success:2 failure:0 skipped:1 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test ERROR: Unhandled exception while processing: kapow!, path: a/c1, db: /tmp/tmppky_85qu/sdb/containers/0/ash/c1hash/c1hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2516, in _one_shard_cycle self._process_broker(broker, node, part) File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1245, in _execute_mock_call result = effect(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 886, in mock_processing raise Exception('kapow!') Exception: kapow! sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:2 failure:1 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: Unlinked retiring db '/tmp/tmppky_85qu/sda/containers/0/ash/c0hash/c0hash.db' sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test WARNING: Skipping xxxx as it is not mounted sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 test.unit.container.test_sharder.TestSharder.test_one_shard_cycle test.unit.container.test_sharder.TestSharder.test_one_shard_cycle ... ok test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers test.unit.container.test_sharder.TestSharder.test_one_shard_cycle_no_containers ... ok sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Found no containers directories sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:3 success:3 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.05s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 0) sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.01s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdc) sharder-test INFO: (Override partitions: 2, 0) sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters test.unit.container.test_sharder.TestSharder.test_partition_and_device_filters ... ok test.unit.container.test_sharder.TestSharder.test_periodic_warning test.unit.container.test_sharder.TestSharder.test_periodic_warning ... ok sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.01s sharder-test INFO: Begin container sharder "once" mode sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: (Override devices: sdb, sdc) sharder-test INFO: Since Sun Jun 2 05:01:02 2024 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:02 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test ERROR: Exception dumping recon cache: [Errno 2] No such file or directory: '/var/cache/swift/container.recon': Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 3433, in dump_recon_cache with lock_file(cache_file, lock_timeout, unlink=False) as cf: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/__init__.py", line 2393, in lock_file fd = os.open(filename, flags) ^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/swift/container.recon' sharder-test INFO: Container sharder "once" mode completed: 0.03s sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmp68wmgwc3/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmp68wmgwc3/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test WARNING: normal warning, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 1a, path: a/c1, db: /tmp/tmp68wmgwc3/sda/containers/0/8b9/a9f7e97965d6cf799a529102a973b8b9/a9f7e97965d6cf799a529102a973b8b9.db sharder-test WARNING: periodic warning 2, path: a/c2, db: /tmp/tmp68wmgwc3/sda/containers/0/229/9ab62b5ef34a985438bfdf7ee0102229/9ab62b5ef34a985438bfdf7ee0102229.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Identified 1 sharding candidates, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Starting scan for shard ranges, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test INFO: Completed scan for shard ranges: 2 found, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test INFO: Final shard range reached., path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard ... ok test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db test.unit.container.test_sharder.TestSharder.test_process_broker_leader_auto_shard_deleted_db ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp3jd2qtak/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347662.67168.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmpwijz961g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmpwijz961g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmpwijz961g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmpwijz961g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded (deleted), path: a/c, db: /tmp/tmpwijz961g/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrinking' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_no_others ... ok sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharding' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'sharded' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Own shard range in state 'shrunk' but no shard ranges and not leader; remaining unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmphtdkyki_/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others test.unit.container.test_sharder.TestSharder.test_process_broker_not_sharding_others ... ok sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Container 'a/c' cannot be set to sharding state: missing epoch sharder-test DEBUG: Finished processing, state unsharded, path: a/c, db: /tmp/tmptt_z6jyv/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022649765014648438s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001277923583984375s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002703666687011719s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00016808509826660156s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash60False/hash60False_1717347666.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002319812774658203s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000133514404296875s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022673606872558594s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013756752014160156s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash60True/hash60True_1717347685.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002560615539550781s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001289844512939453s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022220611572265625s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012946128845214844s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_completes_with_own_and_other_ranges ... ok sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/lse/hash50False/hash50False_1717347704.00000.db sharder-test DEBUG: Starting processing, state unsharded (deleted), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: PUT new shard range container for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test INFO: Completed creating 2 shard range containers, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c_m_0 for ShardRange, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00022864341735839844s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001308917999267578s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test INFO: Cleaving from row None into .shards_a/c__1 for ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002353191375732422s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001537799835205078s, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db sharder-test INFO: Cleaving ShardRange<'m' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - zero objects found, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Finished processing, state sharded (deleted), path: a/c, db: /tmp/tmp7xijksa5/sda/containers/0/rue/hash50True/hash50True_1717347720.00000.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharding', path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h60/hash60/hash60_1717347667.43953.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50.db sharder-test INFO: Kick off container cleaving, own shard range in state 'shrinking', path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges test.unit.container.test_sharder.TestSharder.test_process_broker_sharding_stalls_with_own_and_other_ranges ... ok test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin test.unit.container.test_sharder.TestSharder.test_ratelimited_roundrobin ... ok sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h50/hash50/hash50_1717347667.56367.db sharder-test DEBUG: Starting processing, state unsharded, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70.db sharder-test INFO: Kick off container cleaving, own shard range in state 'sharded', path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test INFO: Starting to cleave (2 todo), path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test INFO: Stopped cleave at unready ShardRange, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp3htobi19/sda/containers/0/h70/hash70/hash70_1717347667.67169.db sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:01 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:01 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 5.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:1 success:1 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 12:00:06 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 3604.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test INFO: Since Wed Dec 31 13:00:10 1969 visited - attempted:2 success:2 failure:0 skipped:0 completed:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 scanned - attempted:1 success:1 failure:0 found:2 min_time:99 max_time:123 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 created - attempted:1 success:1 failure:1 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 cleaved - attempted:1 success:1 failure:0 min_time:0.01 max_time:1.3 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 misplaced - attempted:1 success:1 failure:0 found:1 placed:1 unplaced:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_root - attempted:5 success:4 failure:1 has_overlap:0 num_overlap:0 sharder-test INFO: Since Wed Dec 31 13:00:10 1969 audit_shard - attempted:2 success:2 failure:0 sharder-test INFO: Container sharder cycle completed: 30.00s sharder-test INFO: Container sharder cycle starting, auto-sharding False sharder-test ERROR: Unhandled exception while dumping progress: Test over, path: a/c2, db: /tmp/tmpodh8na6j/sda/containers/0/ash/c2hash/c2hash.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over test.unit.container.test_sharder.TestSharder.test_run_forever test.unit.container.test_sharder.TestSharder.test_run_forever ... ok sharder-test ERROR: Exception in sharder: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2553, in run_forever self._one_shard_cycle(devices_to_shard=Everything(), File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2530, in _one_shard_cycle self._periodic_report_stats() File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1143, in _periodic_report_stats if (time.time() - self.reported) >= self.stats_interval: ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 2526, in _one_shard_cycle self._record_sharding_progress(broker, node, error) File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1053, in _record_sharding_progress own_shard_range = broker.get_own_shard_range() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/container/backend.py", line 2058, in get_own_shard_range self.path, Timestamp.now(), ShardRange.MIN, ShardRange.MAX, ^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/utils/timestamp.py", line 129, in now return cls(time.time(), offset=offset, delta=delta) ^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1178, in __call__ return _mock_self._mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1182, in _mock_call return _mock_self._execute_mock_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mock/mock.py", line 1239, in _execute_mock_call raise effect Exception: Test over sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test ERROR: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: 500, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: 500, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test WARNING: Failed to put shard ranges to 10.0.1.2:1102/sdc a/c: 500, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error test.unit.container.test_sharder.TestSharder.test_send_shard_ranges test.unit.container.test_sharder.TestSharder.test_send_shard_ranges ... ok sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: FakeStatus Error, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/swift/container/sharder.py", line 1206, in _put_container direct_put_container(node, part, account, container, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 400, in direct_put_container _make_req(node, part, 'PUT', path, headers_out, 'Container', conn_timeout, File "/build/reproducible-path/swift-2.33.0/swift/common/direct_client.py", line 122, in _make_req conn = http_connect(ip, port, node['device'], part, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/swift/common/bufferedhttp.py", line 280, in http_connect return http_connect_raw( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 1008, in connect conn = FakeConn(status, etag, body=body, timestamp=timestamp, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 802, in __init__ status = FakeStatus(status) ^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/swift-2.33.0/test/unit/__init__.py", line 704, in __init__ raise status('FakeStatus Error') Exception: FakeStatus Error sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test ERROR: Failed to put shard ranges to 10.0.1.0:1100/sda a/c: , path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test ERROR: Failed to put shard ranges to 10.0.1.1:1101/sdb a/c: , path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db: Timeout (Nones) sharder-test WARNING: Failed to put shard ranges to 10.0.1.3:1103/sdd a/c: 404, path: a/c, db: /tmp/tmpj59e23jr/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Moving any misplaced objects from sharding container, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003674030303955078s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00027751922607421875s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj002_0 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmp6nsqjdzz/sda/containers/0/8fc/857fe81dc763e7cff7e7cd987d6c28fc/857fe81dc763e7cff7e7cd987d6c28fc.db, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Starting to cleave (4 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaving ShardRange - shard db already in sync, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Replicating new shard container .shards_a/c_obj002_0 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaved ShardRange in 0.011s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003440380096435547s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.000217437744140625s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmp6nsqjdzz/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00020933151245117188s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00024056434631347656s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455> - shard db already in sync, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 1 required, shard db: /tmp/tmp6nsqjdzz/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0001399517059326172s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013303756713867188s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455> - shard db already in sync, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c_obj004_1 in state cleaved: 0 successes, 0 required, shard db: /tmp/tmp6nsqjdzz/sda/containers/0/d88/e4460cf9fd8281b0fc151a616970fd88/e4460cf9fd8281b0fc151a616970fd88.db, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [('', 'obj002')], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00021910667419433594s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00012230873107910156s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Continuing to cleave (1 done, 3 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj004_1 for ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaving ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455> - shard db already in sync, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Replicating new shard container .shards_a/c_obj004_1 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaved ShardRange<'obj002' to 'obj004' as of 1717347261.21455, (0, 0) as of 1717347261.21455, cleaved as of 1717347261.21455> in 0.004s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c_obj006_2 for ShardRange<'obj004' to 'obj006' as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0003352165222167969s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00021839141845703125s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c_obj006_2 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaved ShardRange<'obj004' to 'obj006' as of 1717347261.21455, (2, 2) as of 1717347668.48663, cleaved as of 1717347261.21455> in 0.023s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 2 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00019431114196777344s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=True) in 9.703636169433594e-05s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0003592967987060547s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.0001971721649169922s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: Failed to sufficiently replicate cleaved shard .shards_a/c__3 in state cleaved: 2 successes, 3 required, shard db: /tmp/tmp6nsqjdzz/sda/containers/0/683/f7a6bb6d9367b7ae7b6b30a833289683/f7a6bb6d9367b7ae7b6b30a833289683.db, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 0 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 sharder-test WARNING: existing_shard_replication_quorum of 99 exceeds replica count 3, reducing to 3 test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures test.unit.container.test_sharder.TestSharder.test_shard_replication_quorum_failures ... ok test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts test.unit.container.test_sharder.TestSharder.test_sharded_record_sharding_progress_missing_contexts ... ok test.unit.container.test_sharder.TestSharder.test_sharding_enabled test.unit.container.test_sharder.TestSharder.test_sharding_enabled ... ok sharder-test DEBUG: Starting processing, state sharding, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Looking for misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: misplaced object source bounds [('', 'obj006')], path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.00010609626770019531s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00013875961303710938s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished handling misplaced objects, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Continuing to cleave (3 done, 1 todo), path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaving from row None into .shards_a/c__3 for ShardRange<'obj006' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455>, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaving ShardRange<'obj006' to MaxBound as of 1717347261.21455, (0, 0) as of 1717347261.21455, created as of 1717347261.21455> - shard db already in sync, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Replicating new shard container .shards_a/c__3 for ShardRange, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test INFO: Cleaved ShardRange<'obj006' to MaxBound as of 1717347261.21455, (3, 3) as of 1717347668.62959, cleaved as of 1717347261.21455> in 0.009s, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Cleaved 1 shard ranges, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db' sharder-test INFO: Completed cleaving, DB set to sharded state, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test DEBUG: Finished processing, state sharded, path: a/c, db: /tmp/tmp6nsqjdzz/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00064.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b470a72-13ce-41fc-b6c8-5e0050b5f645-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpdmaqq8b5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b470a72-13ce-41fc-b6c8-5e0050b5f645-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpdmaqq8b5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b470a72-13ce-41fc-b6c8-5e0050b5f645-sda', 'cursor': '', 'max_row': 1, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpdmaqq8b5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': '0b470a72-13ce-41fc-b6c8-5e0050b5f645-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': 1, 'last_cleave_to_row': None, 'cleaving_done': True, 'misplaced_done': True, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpdmaqq8b5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00000.db sharder-test WARNING: Repeat cleaving required, context: {'ref': 'ee212e9a-aad9-487c-9e61-0a56c4c9360a-sda', 'cursor': '', 'max_row': 2, 'cleave_to_row': None, 'last_cleave_to_row': None, 'cleaving_done': False, 'misplaced_done': False, 'ranges_done': 0, 'ranges_todo': 0}, path: a/c, db: /tmp/tmpdmaqq8b5/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33_1717347668.00000.db sharder-test DEBUG: Unlinked retiring db '/tmp/tmpomwa6rva/sda/containers/0/ash/c2hash/c2hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmpomwa6rva/sda/containers/0/ash/c1hash/c1hash.db' sharder-test DEBUG: Unlinked retiring db '/tmp/tmpomwa6rva/sda/containers/0/ash/c3hash/c3hash.db' sharder-test INFO: Since Sun Jun 2 05:01:09 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_shard - attempted:0 success:0 failure:0 test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump test.unit.container.test_sharder.TestSharder.test_shrinking_candidate_recon_dump ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges test.unit.container.test_sharder.TestSharder.test_update_root_container_all_ranges ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported test.unit.container.test_sharder.TestSharder.test_update_root_container_already_reported ... ok test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range ... FAIL test.unit.container.test_sharder.TestSharder.test_update_stat test.unit.container.test_sharder.TestSharder.test_update_stat ... ok sharder-test INFO: Since Sun Jun 2 05:01:09 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 visited - attempted:0 success:0 failure:0 skipped:0 completed:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 scanned - attempted:0 success:0 failure:0 found:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 created - attempted:0 success:0 failure:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 cleaved - attempted:0 success:0 failure:0 min_time:0 max_time:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 misplaced - attempted:0 success:0 failure:0 found:0 placed:0 unplaced:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_root - attempted:0 success:0 failure:0 has_overlap:0 num_overlap:0 sharder-test INFO: Since Sun Jun 2 05:01:09 2024 audit_shard - attempted:0 success:0 failure:0 sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=-1, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=8, tombstones=0, path: a/c, db: /tmp/tmpqoct7vl8/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpd2l8_s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpd2l8_s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpd2l8_s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpd2l8_s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpd2l8_s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpd2l8_s6l/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=0, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=3, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=4, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=5, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=6, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: tombstones = 0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: updated root objs=7, tombstones=0, path: a/c, db: /tmp/tmpbojx06up/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=False) in 0.0004966259002685547s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 15 rows (deleted=True) in 0.00037217140197753906s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=False) in 0.0002846717834472656s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00030541419982910156s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.000324249267578125s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00027108192443847656s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects test.unit.container.test_sharder.TestSharder.test_yield_objects ... ok sharder-test DEBUG: got 5 rows (deleted=False) in 0.0004265308380126953s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.0002925395965576172s, path: a/c, db: /tmp/tmpv0zr2zse/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00028514862060546875s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0002639293670654297s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002243518829345703s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.0002486705780029297s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00027370452880859375s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0006816387176513672s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00021982192993164062s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.0002205371856689453s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00022935867309570312s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=False) in 0.0003921985626220703s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001697540283203125s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 4 rows (deleted=True) in 0.0002505779266357422s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.0001671314239501953s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0002052783966064453s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00020360946655273438s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00022912025451660156s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0001971721649169922s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00028133392333984375s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00025200843811035156s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00018095970153808594s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=True) in 0.00016617774963378906s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001747608184814453s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.0002079010009765625s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00027561187744140625s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002715587615966797s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00023984909057617188s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.0002167224884033203s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00021386146545410156s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.00036144256591796875s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=False) in 0.00020241737365722656s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 2 rows (deleted=True) in 0.00018835067749023438s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.00020265579223632812s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=True) in 0.00020241737365722656s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=False) in 0.00021123886108398438s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 3 rows (deleted=True) in 0.00021958351135253906s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 1 rows (deleted=False) in 0.0001842975616455078s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=False) in 0.0002334117889404297s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 5 rows (deleted=False) in 0.0002181529998779297s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 10 rows (deleted=True) in 0.0004706382751464844s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range ... ok test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects test.unit.container.test_sharder.TestSharder.test_yield_objects_to_shard_range_no_objects ... ok test.unit.obj.test_diskfile.TestECDiskFile.test_close_error test.unit.obj.test_diskfile.TestECDiskFile.test_close_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit test.unit.obj.test_diskfile.TestECDiskFile.test_commit ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync test.unit.obj.test_diskfile.TestECDiskFile.test_commit_fsync ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error test.unit.obj.test_diskfile.TestECDiskFile.test_commit_ignores_cleanup_ondisk_files_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_fsync_dir_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_EDQUOT_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_ENOSPC_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_OSError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError test.unit.obj.test_diskfile.TestECDiskFile.test_commit_raises_DiskFileError_for_rename_other_IOError ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_and_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_content_type_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_close_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space test.unit.obj.test_diskfile.TestECDiskFile.test_create_mkstemp_no_space ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_create_prealloc_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_DiskFileNoSpace ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_logging ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails test.unit.obj.test_diskfile.TestECDiskFile.test_create_unlink_cleanup_renamer_fails ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called test.unit.obj.test_diskfile.TestECDiskFile.test_create_use_linkat_renamer_not_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_errors_bad_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_data_file_has_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_data_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_delete test.unit.obj.test_diskfile.TestECDiskFile.test_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_corners ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_partial_closes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_range_w_none ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_checks_only_aligned_frag_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_edges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_empty ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_no_etag_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_app_iter_ranges_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_marked_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_concurrent_writes ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_default_disallowed_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_large_app_iter_ranges ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_mkstemp_creates_dir ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_slo ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_preserves_sysmeta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_io_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_reader_iter_w_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer test.unit.obj.test_diskfile.TestECDiskFile.test_disk_file_writer ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_content_length_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_names ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_diskfile_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_missing_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_newer_non_durable_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_no_data_file ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_durable_timestamp_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files test.unit.obj.test_diskfile.TestECDiskFile.test_error_in_cleanup_ondisk_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_exception_in_handle_close_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments test.unit.obj.test_diskfile.TestECDiskFile.test_fragments ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_available_when_not_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open test.unit.obj.test_diskfile.TestECDiskFile.test_fragments_not_open ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash test.unit.obj.test_diskfile.TestECDiskFile.test_from_audit_location_with_mismatched_hash ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_datafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path test.unit.obj.test_diskfile.TestECDiskFile.test_get_dev_path ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_get_metafile_metadata_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_fallback_to_mkstemp ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called test.unit.obj.test_diskfile.TestECDiskFile.test_get_tempfile_use_linkat_os_open_called ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised test.unit.obj.test_diskfile.TestECDiskFile.test_get_tmpfile_os_open_other_exceptions_are_raised ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache test.unit.obj.test_diskfile.TestECDiskFile.test_keep_cache ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_killed_before_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking test.unit.obj.test_diskfile.TestECDiskFile.test_mount_checking ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened test.unit.obj.test_diskfile.TestECDiskFile.test_not_opened ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_create ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete test.unit.obj.test_diskfile.TestECDiskFile.test_number_calls_to_cleanup_ondisk_files_during_delete ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_data_info_has_durable_key ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_data_ts_meta_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_listdir_error_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_data_ts_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_meta_ts_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_multiple_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_stale_meta_data_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_ts_meta_data ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_ondisk_search_loop_wayward_files_ignored_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_open_deleted_with_corrupt_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_durable_legacy ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_most_recent_missing_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_exist ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired test.unit.obj.test_diskfile.TestECDiskFile.test_open_not_expired ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_bad_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_open_with_fragment_preferences_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed test.unit.obj.test_diskfile.TestECDiskFile.test_purge_already_removed ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta test.unit.obj.test_diskfile.TestECDiskFile.test_purge_final_fragment_index_and_not_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_purge_last_fragment_index_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_non_existent_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_timestamp_frag_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_old_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index test.unit.obj.test_diskfile.TestECDiskFile.test_purge_one_fragment_index ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone test.unit.obj.test_diskfile.TestECDiskFile.test_purge_tombstone ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag test.unit.obj.test_diskfile.TestECDiskFile.test_purge_without_frag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_bad_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_deleted_files ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_fstat_oserror ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_a_directory ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_hashdir_not_listable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_bad_x_delete_at ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_content_length_bad ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_etag ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_missing_name ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_invalid_zero_byte ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_ioerror_enodata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_missing_content_length ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_subtly_corrupt_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_truncated_xattrs ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids test.unit.obj.test_diskfile.TestECDiskFile.test_quarantine_valids ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_read_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_causes_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine test.unit.obj.test_diskfile.TestECDiskFile.test_reader_ec_exception_does_not_cause_quarantine ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary test.unit.obj.test_diskfile.TestECDiskFile.test_reader_frag_check_does_not_quarantine_if_its_not_binary ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive test.unit.obj.test_diskfile.TestECDiskFile.test_reader_quarantines_corrupted_ec_archive ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks test.unit.obj.test_diskfile.TestECDiskFile.test_splice_to_wsockfd_blocks ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable test.unit.obj.test_diskfile.TestECDiskFile.test_sysmeta_frag_index_is_immutable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch test.unit.obj.test_diskfile.TestECDiskFile.test_tee_to_md5_pipe_length_mismatch ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_legacy_durable ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase test.unit.obj.test_diskfile.TestECDiskFile.test_write_cleanup_part_power_increase ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_disk_full ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_no_xattr ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_multiple_metas ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_content_type_removes_same_time_meta ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type test.unit.obj.test_diskfile.TestECDiskFile.test_write_metadata_with_older_content_type ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_cache_dropping ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported test.unit.obj.test_diskfile.TestECDiskFile.test_zero_copy_turns_off_when_md5_sockets_not_supported ... skipped 'Large xattrs not supported in `/tmp`. Skipping test' test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_ignore_corrupted_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes test.unit.obj.test_diskfile.TestHashesHelpers.test_read_legacy_hashes ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_invalid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality test.unit.obj.test_diskfile.TestHashesHelpers.test_read_write_valid_hashes_mutation_and_transative_equality ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_invalid_updated ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_safe_default ... ok test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated test.unit.obj.test_diskfile.TestHashesHelpers.test_write_hashes_valid_updated ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account test.unit.proxy.test_server.TestProxyServerConfiguration.test_auto_create_account ... ok sharder-test DEBUG: got 5 rows (deleted=True) in 0.0003097057342529297s, path: a/c, db: /tmp/tmplrjkkxzk/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=False) in 0.0002601146697998047s, path: a/c, db: /tmp/tmpt5g7k1yy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db sharder-test DEBUG: got 0 rows (deleted=True) in 0.00014090538024902344s, path: a/c, db: /tmp/tmpt5g7k1yy/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db proxy DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) cont1 DEBUG: Could not load '/tmp/tmptrnnkhbe/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmptrnnkhbe/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont1 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. cont2 DEBUG: Could not load '/tmp/tmptrnnkhbe/tmp_test_proxy_server_chunked/container-sync-realms.conf': [Errno 2] No such file or directory: '/tmp/tmptrnnkhbe/tmp_test_proxy_server_chunked/container-sync-realms.conf' cont2 WARNING: Option allow_versions is deprecated. Configure the versioned_writes middleware in the proxy-server instead. This option will be ignored in a future release. acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a" 201 - "-" "test" "-" 0.0144 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a" 201 - "-" "test" "-" 0.0122 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/2/a1" 201 - "-" "test" "-" 0.0111 "-" 3656192 - acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/2/a1" 201 - "-" "test" "-" 0.0138 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "HEAD /sda1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "tx1ba35e975100455abbff8-00665ca556" "proxy-server 3656192" 0.0033 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://127.0.0.1:43955/sda1/1/a/c" "tx1ba35e975100455abbff8-00665ca556" "container-server 3656192" 0.0024 "-" 3656192 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://127.0.0.1:49855/sdb1/1/a/c" "tx1ba35e975100455abbff8-00665ca556" "container-server 3656192" 0.0017 "-" 3656192 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx1ba35e975100455abbff8-00665ca556" "proxy-server 3656192" 0.0423 "-" 3656192 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a/c" 201 - "PUT http://localhost/v1/a/c" "tx1ba35e975100455abbff8-00665ca556" "proxy-server 3656192" 0.0261 "-" 3656192 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/17/01/10 PUT /v1/a/c HTTP/1.0 201 - - t - - - tx1ba35e975100455abbff8-00665ca556 - 0.0592 - - 1717347670.575749636 1717347670.634943724 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "HEAD /sdb1/2/a1" 204 - "HEAD http://localhost/v1/a1?format=json" "tx78323723a0a54b2ebb7a0-00665ca556" "proxy-server 3656192" 0.0018 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/2/a1/c1" 201 - "PUT http://127.0.0.1:43955/sda1/1/a1/c1" "tx78323723a0a54b2ebb7a0-00665ca556" "container-server 3656192" 0.0065 "-" 3656192 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/2/a1/c1" 201 - "PUT http://127.0.0.1:49855/sdb1/1/a1/c1" "tx78323723a0a54b2ebb7a0-00665ca556" "container-server 3656192" 0.0019 "-" 3656192 0 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx78323723a0a54b2ebb7a0-00665ca556" "proxy-server 3656192" 0.0411 "-" 3656192 0 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a1/c1" 201 - "PUT http://localhost/v1/a1/c1" "tx78323723a0a54b2ebb7a0-00665ca556" "proxy-server 3656192" 0.0282 "-" 3656192 0 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/17/01/10 PUT /v1/a1/c1 HTTP/1.0 201 - - t - - - tx78323723a0a54b2ebb7a0-00665ca556 - 0.0550 - - 1717347670.637368202 1717347670.692354679 0 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txa64b0fe30af34832b9e87-00665ca556" "proxy-server 3656192" 0.0283 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a/c1" 201 - "PUT http://127.0.0.1:49855/sdb1/2/a/c1" "txa64b0fe30af34832b9e87-00665ca556" "container-server 3656192" 0.0108 "-" 3656192 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a/c1" 201 - "PUT http://127.0.0.1:43955/sda1/2/a/c1" "txa64b0fe30af34832b9e87-00665ca556" "container-server 3656192" 0.0018 "-" 3656192 1 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa64b0fe30af34832b9e87-00665ca556" "proxy-server 3656192" 0.0292 "-" 3656192 1 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/2/a/c1" 201 - "PUT http://localhost/v1/a/c1" "txa64b0fe30af34832b9e87-00665ca556" "proxy-server 3656192" 0.0470 "-" 3656192 1 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/17/01/10 PUT /v1/a/c1 HTTP/1.0 201 - - t - - - txa64b0fe30af34832b9e87-00665ca556 - 0.0871 - - 1717347670.694678545 1717347670.781741142 1 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "HEAD /sdb1/1/a" 204 - "HEAD http://localhost/v1/a?format=json" "txffa63bc7e52c42428c073-00665ca556" "proxy-server 3656192" 0.0106 "-" 3656192 - acct1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://127.0.0.1:43955/sda1/1/a/c2" "txffa63bc7e52c42428c073-00665ca556" "container-server 3656192" 0.0111 "-" 3656192 2 acct2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://127.0.0.1:49855/sdb1/1/a/c2" "txffa63bc7e52c42428c073-00665ca556" "container-server 3656192" 0.0021 "-" 3656192 2 cont2 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sdb1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txffa63bc7e52c42428c073-00665ca556" "proxy-server 3656192" 0.0329 "-" 3656192 2 cont1 INFO: 127.0.0.1 - - [02/Jun/2024:17:01:10 +0000] "PUT /sda1/1/a/c2" 201 - "PUT http://localhost/v1/a/c2" "txffa63bc7e52c42428c073-00665ca556" "proxy-server 3656192" 0.0500 "-" 3656192 2 proxy INFO: 127.0.0.1 127.0.0.1 02/Jun/2024/17/01/10 PUT /v1/a/c2 HTTP/1.0 201 - - t - - - txffa63bc7e52c42428c073-00665ca556 - 0.0729 - - 1717347670.784274578 1717347670.857201099 2 test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_cors_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_recheck_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options test.unit.proxy.test_server.TestProxyServerConfiguration.test_memcache_skip_options ... ok test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout test.unit.proxy.test_server.TestProxyServerConfiguration.test_node_timeout ... ok test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) test-proxy-config DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None, 'rebalance_missing_suppression_count': 1, 'concurrent_gets': False, 'concurrency_timeout': 0.5, 'concurrent_ec_extra_requests': 0}, app) ====================================================================== FAIL: test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range test.unit.container.test_sharder.TestSharder.test_update_root_container_own_range ---------------------------------------------------------------------- testtools.testresult.real._StringException: Traceback (most recent call last): File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 5956, in test_update_root_container_own_range check_only_own_shard_range_sent(state) File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 5944, in check_only_own_shard_range_sent self.check_shard_ranges_sent(broker, expected_sent) File "/build/reproducible-path/swift-2.33.0/test/unit/container/test_sharder.py", line 5900, in check_shard_ranges_sent self.assertEqual(servers, [ File "/usr/lib/python3.11/unittest/case.py", line 873, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.11/unittest/case.py", line 1079, in assertListEqual self.assertSequenceEqual(list1, list2, msg, seq_type=list) File "/usr/lib/python3.11/unittest/case.py", line 1061, in assertSequenceEqual self.fail(msg) File "/usr/lib/python3.11/unittest/case.py", line 703, in fail raise self.failureException(msg) AssertionError: 70 Failed with Lists differ: [('10.0.1.0', 1100), ('10.0.1.2', 1102), ('10.0.1.1', 1101)] != [('10.0.1.0', 1100), ('10.0.1.1', 1101), ('10.0.1.2', 1102)] First differing element 1: ('10.0.1.2', 1102) ('10.0.1.1', 1101) - [('10.0.1.0', 1100), ('10.0.1.2', 1102), ('10.0.1.1', 1101)] + [('10.0.1.0', 1100), ('10.0.1.1', 1101), ('10.0.1.2', 1102)] ---------------------------------------------------------------------- Ran 8924 tests in 430.905s FAILED (failures=1, skipped=1814) + echo ======> STESTR TEST SUITE FAILED FOR python3.11: displaying pip3 freeze output... ======> STESTR TEST SUITE FAILED FOR python3.11: displaying pip3 freeze output... + [ -x /usr/bin/pip3 ] + pip3 freeze WARNING: The directory '/nonexistent/first-build/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. alabaster==0.7.12 attrs==23.2.0 autopage==0.4.0 Babel==2.14.0 blinker==1.8.2 boto3==1.34.46 botocore==1.34.46 certifi==2023.11.17 cffi==1.16.0 chardet==5.2.0 charset-normalizer==3.3.2 cliff==4.6.0 cmd2==2.4.3+ds coverage==7.2.7 cryptography==42.0.5 debtcollector==3.0.0 decorator==5.1.1 deprecation==2.0.7 dnspython==2.6.1 docutils==0.20.1 dogpile.cache==1.3.1 dulwich==0.21.6 eventlet==0.0.0 extras==1.0.0 fixtures==4.1.0 greenlet==3.0.1 hatch-vcs==0.4.0 hatchling==1.24.2 idna==3.6 imagesize==1.4.1 importlib-metadata==7.1.0 iso8601==1.0.2 jaraco.classes==3.2.1 jaraco.context==4.3.0 jaraco.functools==4.0.0 jeepney==0.8.0 Jinja2==3.1.3 jmespath==1.0.1 jsonpatch==1.32 jsonpointer==2.4 jsonschema==4.19.2 jsonschema-specifications==2023.12.1 keyring==25.2.1 keystoneauth1==5.6.0 keystonemiddleware==10.6.0 lxml==5.2.1 Mako==1.3.2.dev0 MarkupSafe==2.1.5 mock==5.1.0 more-itertools==10.2.0 msgpack==1.0.3 netaddr==0.10.1 netifaces==0.11.0 oauthlib==3.2.2 openstackdocstheme==3.2.0 openstacksdk==3.0.0 os-api-ref==2.3.0 os-client-config==2.1.0 os-service-types==1.7.0 os-testr==3.0.0 osc-lib==3.0.1 oslo.cache==3.7.0 oslo.config==9.4.0 oslo.context==5.5.0 oslo.i18n==6.3.0 oslo.log==5.5.1 oslo.serialization==5.4.0 oslo.utils==7.1.0 oslosphinx==4.18.0 packaging==24.0 Paste==3.10.0 PasteDeploy==3.1.0 pathspec==0.12.1 pbr==6.0.0 platformdirs==4.2.1 pluggy==1.5.0 ply==3.11 prettytable==3.6.0 pyasyncore==1.0.2 pycadf==3.1.1 pycparser==2.22 pyeclib==1.6.1 Pygments==2.17.2 pyinotify==0.9.6 PyJWT==2.7.0 pyOpenSSL==24.1.0 pyparsing==3.1.2 pyperclip==1.8.2 python-binary-memcached==0.31.2 python-cinderclient==9.5.0 python-dateutil==2.9.0 python-glanceclient==4.5.0 python-keystoneclient==5.4.0 python-memcached==1.60 python-neutronclient==11.2.0 python-novaclient==18.6.0 python-openstackclient==6.6.0 python-subunit==1.4.2 python-swiftclient==4.5.0 pytz==2024.1 PyYAML==6.0.1 referencing==0.31.0 requests==2.31.0 requests-mock==1.11.0 requestsexceptions==1.4.0 rfc3986==2.0.0 roman==3.3 rpds-py==0.12.0 s3transfer==0.10.1 SecretStorage==3.3.3 setuptools-scm==8.0.4 simplejson==3.19.2 six==1.16.0 snowballstemmer==2.2.0 Sphinx==7.2.6 stestr==4.1.0 stevedore==5.2.0 swift==2.33.0 Tempita==0.5.2 testtools==2.7.1 tomlkit==0.12.5 trove-classifiers==2024.1.31 typing_extensions==4.12.0 uhashring==2.3 urllib3==1.26.18 voluptuous==0.14.2 warlock==2.0.1 wcwidth==0.2.13 WebOb==1.8.7 wrapt==1.15.0 xattr==0.10.1 zipp==1.0.0 + exit 1 make[1]: *** [debian/rules:63: override_dh_auto_install] Error 1 make[1]: Leaving directory '/build/reproducible-path/swift-2.33.0' make: *** [debian/rules:8: binary] Error 2 dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2 I: copying local configuration E: Failed autobuilding of package I: unmounting dev/ptmx filesystem I: unmounting dev/pts filesystem I: unmounting dev/shm filesystem I: unmounting proc filesystem I: unmounting sys filesystem I: cleaning the build env I: removing directory /srv/workspace/pbuilder/3586756 and its subdirectories